Compare commits

...
Author SHA1 Message Date
Garry TanandClaude Fable 5 7e1c4f312f Merge origin/master — reconcile #1144 composition-aware brain_score with #3023 orphan-policy getHealth rewrite
Master's #3023 moved orphan_pages out of SQL into a TS filter over islanded
rows (shared orphan-reporting policy). Kept that structure and threaded the
#1144 narrative exclusion into the islanded query instead: code/calendar-index
pages no longer enter the orphan numerator, matching the narrative_pages
denominator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 11:12:32 -07:00
Garry TanandClaude Fable 5 e5b3e7aba7 fix(health): composition-aware brain_score + integrity/recipe fixes (#1098 #1099 #1143 #1144)
- recipes: ClawVisor health checks hit /ready (surfaces db/vault degradation)
  instead of bare-liveness /health across email/calendar/credential-gateway (#1098)
- recipes: ship canonical contacts-to-brain.md, the Google Contacts consumer
  the credential-gateway recipe already promises (#1099)
- integrity: skip type:code pages in both scan paths and in integrity auto so
  'tweet' in source identifiers/comments can't be flagged or auto-repaired (#1143)
- health: exclude code / calendar-index pages from the orphan, link-density and
  timeline denominators of brain_score in BOTH engines (engine parity), so bulk
  code/calendar imports no longer crater the score (#1144)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:35:01 -07:00
12 changed files with 530 additions and 20 deletions
+2 -2
View File
@@ -23,7 +23,7 @@ health_checks:
label: "Auth provider"
checks:
- type: http
url: "$CLAWVISOR_URL/health"
url: "$CLAWVISOR_URL/ready"
label: "ClawVisor"
- type: env_exists
name: GOOGLE_CLIENT_ID
@@ -135,7 +135,7 @@ Tell the user:
Validate:
```bash
curl -sf "$CLAWVISOR_URL/health" && echo "PASS: ClawVisor reachable" || echo "FAIL"
curl -sf "$CLAWVISOR_URL/ready" && echo "PASS: ClawVisor reachable" || echo "FAIL"
```
**STOP until ClawVisor validates.**
+308
View File
@@ -0,0 +1,308 @@
---
id: contacts-to-brain
name: Contacts-to-Brain
version: 0.1.0
description: Google Contacts become canonical people/ pages, enriching brain entities with ground-truth name/email/phone/org data.
category: sense
requires: [credential-gateway]
secrets:
- name: CLAWVISOR_URL
description: ClawVisor gateway URL (Option A — recommended, handles OAuth for you)
where: https://clawvisor.com — create an agent, activate Google Contacts service
- name: CLAWVISOR_AGENT_TOKEN
description: ClawVisor agent token (Option A)
where: https://clawvisor.com — agent settings, copy the agent token
- name: GOOGLE_CLIENT_ID
description: Google OAuth2 client ID (Option B — direct API access, you manage tokens)
where: https://console.cloud.google.com/apis/credentials — create OAuth 2.0 Client ID
- name: GOOGLE_CLIENT_SECRET
description: Google OAuth2 client secret (Option B)
where: https://console.cloud.google.com/apis/credentials — same page as client ID
health_checks:
- type: any_of
label: "Auth provider"
checks:
- type: http
url: "$CLAWVISOR_URL/ready"
label: "ClawVisor"
- type: env_exists
name: GOOGLE_CLIENT_ID
label: "Google OAuth"
setup_time: 15 min
cost_estimate: "$0 (both options are free)"
---
# Contacts-to-Brain: Your Address Book Becomes Ground Truth
Calendar attendees and email senders are the bulk of `people/<slug>` brain pages.
Your Google Contacts is the ground-truth directory — canonical name, email, phone,
organization — for those same entities. Syncing it closes the "who is this person"
loop automatically and feeds enrichment.
## IMPORTANT: Instructions for the Agent
**You are the installer.** Follow these steps precisely.
**Why this matters:** email-to-brain and calendar-to-brain create people pages
from whatever name string the API returned — sometimes an email prefix, sometimes
a nickname. Contacts carries the authoritative record. After this recipe runs,
"j.smith@acme-example.com" and "Jon S" resolve to the same person page with the
right display name, phone, and company.
**The output is staging files, not direct writes:** the deterministic collector
dumps contact records to `brain/contacts/.staging/`; YOU (the agent) merge them
into `people/<slug>` pages using judgment — the Notability Gate in
`skills/_brain-filing-rules.md` applies. Not every contact deserves a page.
**Do not skip steps. Verify after each step.**
## Architecture
```
Google Contacts (People API, paginated)
↓ (ClawVisor credential gateway: list_contacts / get_contact / search_contacts)
Contacts Sync Script (deterministic Node.js)
↓ Outputs:
├── brain/contacts/.raw/contacts-{date}.json (raw API responses, provenance)
└── brain/contacts/.staging/{slug}.md (one markdown record per contact)
Agent reads staging files
↓ Judgment calls (Notability Gate):
├── Merge into existing people/<slug> pages (name/email/phone/org enrichment)
├── Create new pages ONLY for notable contacts not already in brain
└── Skip the rest (staging is not the brain)
```
## Opinionated Defaults
**Staging record format** (one file per contact, deterministic):
```markdown
# Alice Example
- **Emails:** alice@acme-example.com, alice@gmail.com
- **Phone:** +1 555 0100
- **Organization:** Acme Example — VP Engineering
- **Source:** Google Contacts (resourceName people/c123, synced 2026-07-21)
```
**Enrichment, not duplication:** if `people/alice-example.md` already exists,
append missing fields to it with a `[Source: Google Contacts]` citation. Do NOT
create a second page. Slug-match by normalized name, then by email against
existing page content.
**Notability Gate (from `skills/_brain-filing-rules.md`):** a contact with no
brain presence gets a new page only if they appear elsewhere in the brain
(calendar attendee, email correspondent) or the user confirms they matter.
When in doubt, DON'T create — a junk page degrades search quality.
## Prerequisites
1. **GBrain installed and configured** (`gbrain doctor` passes)
2. **Node.js 18+** (for the sync script)
3. **Google Contacts access** via ONE of:
- **Option A: ClawVisor** (recommended, handles OAuth for you, no token management)
- **Option B: Google OAuth2 directly** (you manage tokens, no extra service needed)
## Setup Flow
### Step 1: Choose and Configure Contacts Access
Ask the user: "How do you want to connect to Google Contacts?
**Option A: ClawVisor (recommended)**
ClawVisor handles OAuth, token refresh, and encryption. If you already use
ClawVisor for email-to-brain or calendar-to-brain, this uses the same setup —
just activate the Google Contacts service on your existing agent.
**Option B: Google OAuth2 directly**
Connect to the Google People API directly. No extra service needed, but you
manage OAuth tokens yourself."
#### Option A: ClawVisor Setup
Tell the user:
"I need your ClawVisor URL and agent token.
1. Go to https://clawvisor.com
2. Create an agent (or use existing)
3. Activate the **Google Contacts** service
4. Create a standing task with purpose: 'Full contacts access for people
enrichment: list contacts, read contact details, search contacts across
all connected Google accounts.'
IMPORTANT: Be EXPANSIVE in the task purpose. Narrow purposes block requests.
5. Copy the gateway URL and agent token"
Validate:
```bash
curl -sf "$CLAWVISOR_URL/ready" && echo "PASS: ClawVisor reachable" || echo "FAIL"
```
**STOP until ClawVisor validates.**
#### Option B: Google OAuth2 Setup
Same flow as `recipes/credential-gateway.md` Option B, with the contacts scope:
1. https://console.cloud.google.com/apis/credentials — create an OAuth client ID
(Desktop app), consent screen scope: `https://www.googleapis.com/auth/contacts.readonly`
2. Enable the People API: https://console.cloud.google.com/apis/library/people.googleapis.com
3. Run the OAuth flow; store tokens in `~/.gbrain/google-tokens.json` (auto-refresh on expiry)
Validate:
```bash
[ -n "$GOOGLE_CLIENT_ID" ] && [ -n "$GOOGLE_CLIENT_SECRET" ] \
&& echo "PASS: Google OAuth credentials set" \
|| echo "FAIL: Missing GOOGLE_CLIENT_ID or GOOGLE_CLIENT_SECRET"
```
**STOP until OAuth flow completes and tokens are stored.**
### Step 2: Set Up the Contacts Sync Script
```bash
mkdir -p contacts-sync
cd contacts-sync
npm init -y
```
The sync script needs these capabilities:
1. **Paginated retrieval**`list_contacts` (People API `people.connections.list`)
returns pages of up to 100; follow `nextPageToken` until exhausted. Request
fields: names, emailAddresses, phoneNumbers, organizations, metadata.
2. **Deterministic staging output** — one markdown file per contact at
`brain/contacts/.staging/{slug}.md`, slug from normalized display name
(fall back to email prefix). Same contact = same file on every run (idempotent).
3. **Raw JSON preservation** — save raw API responses to
`brain/contacts/.raw/contacts-{date}.json` for provenance.
4. **Skip empty records** — contacts with no name AND no email are noise; drop them.
### Step 3: Run the Full Sync
```bash
node contacts-sync.mjs
```
Verify:
```bash
ls brain/contacts/.staging/ | head -10
```
Should show one file per contact, e.g. `alice-example.md`, `charlie-example.md`.
### Step 4: Enrich People Pages (Agent Judgment)
This is YOUR job (the agent). For each staging record:
1. **Check brain**: `gbrain search "contact name"` — do they have a
`people/<slug>` page? Also search by email address.
2. **Existing page** → merge the ground-truth fields (canonical name, emails,
phone, organization) into the page, each with a
`[Source: Google Contacts]` citation. Fix a wrong/partial display name.
3. **No page** → apply the Notability Gate: create a page only if the contact
already appears in the brain (calendar, email) or is clearly relevant.
Otherwise skip.
4. **Back-link** per the Iron Law in `skills/_brain-filing-rules.md`: an
organization with a brain page gets a link from the person's page and back.
After enrichment, import and embed:
```bash
gbrain sync --no-pull --no-embed && gbrain embed --stale
```
Verify:
```bash
gbrain search "alice-example" --limit 3
```
Should return the enriched people page with contact details.
### Step 5: Set Up Weekly Sync
Contacts change slowly; once a week is plenty:
```bash
# Cron: every Sunday at 9 AM
0 9 * * 0 cd /path/to/contacts-sync && node contacts-sync.mjs
```
After each sync, re-run the Step 4 enrichment pass over CHANGED staging files
only (compare mtime or diff against git), then:
```bash
gbrain sync --no-pull --no-embed && gbrain embed --stale
```
### Step 6: Log Setup Completion
```bash
mkdir -p ~/.gbrain/integrations/contacts-to-brain
echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.1.0","status":"ok","details":{"contacts":"CONTACT_COUNT"}}' >> ~/.gbrain/integrations/contacts-to-brain/heartbeat.jsonl
```
Tell the user: "Contacts-to-brain is set up. [N] contacts staged; [M] people
pages enriched with ground-truth contact data. Weekly sync keeps it current."
## Implementation Guide
### Pagination
```
list_all_contacts():
contacts = []
token = null
do:
page = list_contacts({ pageSize: 100, pageToken: token,
personFields: 'names,emailAddresses,phoneNumbers,organizations,metadata' })
contacts += page.connections
token = page.nextPageToken
while token
return contacts
```
### Slug Normalization
```
slugify(contact):
name = contact.names?[0]?.displayName
if not name:
name = contact.emailAddresses?[0]?.value.split('@')[0]
return name.toLowerCase().normalize('NFD')
.replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '')
```
Collision (two contacts, same slug): suffix with the email domain
(`alice-example-acme-example`) rather than overwrite.
### What the Agent Should Test After Setup
1. **Idempotency:** run the sync twice. `git status` on `brain/contacts/.staging/`
shows no changes on the second run.
2. **Pagination:** with 250+ contacts, verify the staging count matches the
Google Contacts count (not capped at 100).
3. **Notability Gate:** verify a one-off contact with no brain presence did NOT
get a `people/` page.
4. **Enrichment merge:** verify an existing people page gained contact fields
without losing its prior content, each with a `[Source: Google Contacts]`
citation.
## Cost Estimate
| Component | Monthly Cost |
|-----------|-------------|
| ClawVisor (free tier) | $0 |
| Google People API | $0 (within free quota) |
| **Total** | **$0** |
## Troubleshooting
**No contacts returned:**
- Check ClawVisor has the Google Contacts service activated
- Check the standing task purpose is expansive enough
- Option B: verify the People API is enabled and the token carries
`contacts.readonly`
**Duplicate people pages after enrichment:**
- The agent matched by name but the brain page slug differs — search by email
address too before creating, then merge and delete the duplicate
**Contacts with no name:**
- The sync script falls back to the email prefix; records with neither name
nor email are dropped as noise
+4 -4
View File
@@ -23,7 +23,7 @@ health_checks:
label: "Auth provider"
checks:
- type: http
url: "$CLAWVISOR_URL/health"
url: "$CLAWVISOR_URL/ready"
label: "ClawVisor"
- type: env_exists
name: GOOGLE_CLIENT_ID
@@ -75,7 +75,7 @@ Tell the user:
3. Activate the services you need:
- **Gmail** (for email-to-brain)
- **Google Calendar** (for calendar-to-brain)
- **Google Contacts** (for enrichment)
- **Google Contacts** (for contacts-to-brain)
4. Create a standing task with a broad purpose. CRITICAL: be EXPANSIVE.
Good purpose: 'Full executive assistant access to Gmail, Calendar, and
@@ -88,7 +88,7 @@ Tell the user:
Validate:
```bash
curl -sf "$CLAWVISOR_URL/health" \
curl -sf "$CLAWVISOR_URL/ready" \
&& echo "PASS: ClawVisor reachable" \
|| echo "FAIL: ClawVisor not reachable — check the URL"
```
@@ -171,7 +171,7 @@ can now access your Google services."
## How to Verify
1. **ClawVisor:** `curl $CLAWVISOR_URL/health` returns OK.
1. **ClawVisor:** `curl $CLAWVISOR_URL/ready` returns OK.
2. **Google OAuth:** Tokens exist at `~/.gbrain/google-tokens.json`.
3. **Gmail access:** Run the email collector — it should pull recent messages.
4. **Calendar access:** Run the calendar sync — it should pull today's events.
+3 -3
View File
@@ -23,7 +23,7 @@ health_checks:
label: "Auth provider"
checks:
- type: http
url: "$CLAWVISOR_URL/health"
url: "$CLAWVISOR_URL/ready"
label: "ClawVisor"
- type: env_exists
name: GOOGLE_CLIENT_ID
@@ -130,7 +130,7 @@ Tell the user:
Validate:
```bash
curl -sf "$CLAWVISOR_URL/health" && echo "PASS: ClawVisor reachable" || echo "FAIL"
curl -sf "$CLAWVISOR_URL/ready" && echo "PASS: ClawVisor reachable" || echo "FAIL"
```
**STOP until ClawVisor validates.**
@@ -328,7 +328,7 @@ threads you already replied to. Sent mail acts as a negative filter.
## Troubleshooting
**No emails collected:**
- Check ClawVisor health: `curl $CLAWVISOR_URL/health`
- Check ClawVisor health: `curl $CLAWVISOR_URL/ready`
- Check standing task is active and has Gmail service enabled
- Check task purpose is expansive enough (narrow purposes block requests)
+11 -1
View File
@@ -334,6 +334,10 @@ export async function scanIntegrity(
if (!page) continue;
// Skip grandfathered pages (opted out of brain-integrity enforcement)
if ((page.frontmatter as Record<string, unknown> | undefined)?.validate === false) continue;
// Skip code pages: indexed source files aren't prose. 'tweet' in an
// identifier or comment is not a bare-tweet citation gap, and auto-repair
// would inject wikilink brackets into source code.
if (page.type === 'code') continue;
pagesScanned++;
bareHits.push(...findBareTweetHits(page.compiled_truth, slug));
externalHits.push(...findExternalLinks(page.compiled_truth, slug));
@@ -363,6 +367,9 @@ async function scanIntegrityBatch(
// YAML) diverges from the sequential path's strict === false check. Intentional
// — gbrain lint should reject stringly-typed validate at write time.
const validateCondition = sql`AND (frontmatter->>'validate' IS NULL OR frontmatter->>'validate' != 'false')`;
// Mirror of the sequential path's `page.type === 'code'` skip: code pages
// (indexed source files) are never prose-integrity candidates.
const codeCondition = sql`AND type IS DISTINCT FROM 'code'`;
// v0.32.8: scan ONE row per (source_id, slug) pair, not one per slug.
// Pre-fix used DISTINCT ON (slug) which collapsed multi-source rows into
@@ -372,7 +379,7 @@ async function scanIntegrityBatch(
const rows = await sql`
SELECT slug, compiled_truth, frontmatter
FROM pages
WHERE 1=1 ${typeCondition} ${validateCondition}
WHERE 1=1 ${typeCondition} ${validateCondition} ${codeCondition}
ORDER BY source_id, slug
LIMIT ${limit}
`;
@@ -461,6 +468,9 @@ async function cmdAuto(args: string[]): Promise<void> {
const page = await engine.getPage(slug, { sourceId: source_id });
if (!page) continue;
// Never auto-repair code pages — injecting tweet citations into
// indexed source files corrupts them. Same gate as scanIntegrity.
if (page.type === 'code') continue;
pagesProcessed++;
progress.tick(1, slug);
+22 -5
View File
@@ -5208,9 +5208,18 @@ export class PGLiteEngine implements BrainEngine {
const { rows: [h] } = await this.db.query(`
WITH entity_pages AS (
SELECT id, slug FROM pages WHERE type IN ('person', 'company')
),
narrative_pages AS (
-- Composition-aware score: code source files and calendar daily
-- files are orphans-by-design (no inbound wikilinks, no Timeline
-- fence). Excluding them from the orphan/link-density/timeline
-- denominators keeps a bulk code/calendar import from cratering
-- brain_score.
SELECT id FROM pages WHERE type IS NULL OR type NOT IN ('code', 'calendar-index')
)
SELECT
(SELECT count(*) FROM pages) as page_count,
(SELECT count(*) FROM narrative_pages) as narrative_page_count,
(SELECT count(*) FROM content_chunks WHERE embedded_at IS NOT NULL)::float /
GREATEST((SELECT count(*) FROM content_chunks), 1)::float as embed_coverage,
0 as stale_pages,
@@ -5222,7 +5231,8 @@ export class PGLiteEngine implements BrainEngine {
) as dead_links,
(SELECT count(*) FROM content_chunks WHERE embedded_at IS NULL) as missing_embeddings,
(SELECT count(*) FROM links) as link_count,
(SELECT count(DISTINCT page_id) FROM timeline_entries) as pages_with_timeline,
(SELECT count(DISTINCT te.page_id) FROM timeline_entries te
WHERE te.page_id IN (SELECT id FROM narrative_pages)) as pages_with_timeline,
(SELECT count(*) FROM entity_pages e
WHERE EXISTS (SELECT 1 FROM links l WHERE l.to_page_id = e.id))::float /
GREATEST((SELECT count(*) FROM entity_pages), 1)::float as link_coverage,
@@ -5244,12 +5254,19 @@ export class PGLiteEngine implements BrainEngine {
const { rows: islandedRows } = await this.db.query(`
SELECT p.slug
FROM pages p
WHERE NOT EXISTS (SELECT 1 FROM links l WHERE l.to_page_id = p.id)
-- Narrative pages only (same type filter as the narrative_pages CTE):
-- code/calendar-index pages are orphans-by-design and must not count
-- against the noOrphans component (#1144).
WHERE (p.type IS NULL OR p.type NOT IN ('code', 'calendar-index'))
AND NOT EXISTS (SELECT 1 FROM links l WHERE l.to_page_id = p.id)
AND NOT EXISTS (SELECT 1 FROM links l WHERE l.from_page_id = p.id)
`);
const r = h as Record<string, unknown>;
const pageCount = Number(r.page_count);
// Composition-aware denominators (excludes code/calendar-index pages);
// a code-only brain has nothing narrative to penalize → full marks.
const narrativePageCount = Number(r.narrative_page_count);
const embedCoverage = Number(r.embed_coverage);
const stalePages = await this.countStalePagesForExtraction({ versionTs: LINK_EXTRACTOR_VERSION_TS });
const orphanOverrides = await loadOrphanPolicyOverrides(this);
@@ -5259,9 +5276,9 @@ export class PGLiteEngine implements BrainEngine {
const linkCount = Number(r.link_count);
const pagesWithTimeline = Number(r.pages_with_timeline);
const linkDensity = pageCount > 0 ? Math.min(linkCount / pageCount, 1) : 0;
const timelineCoverageDensity = pageCount > 0 ? Math.min(pagesWithTimeline / pageCount, 1) : 0;
const noOrphans = pageCount > 0 ? 1 - (orphanPages / pageCount) : 1;
const linkDensity = narrativePageCount > 0 ? Math.min(linkCount / narrativePageCount, 1) : 1;
const timelineCoverageDensity = narrativePageCount > 0 ? Math.min(pagesWithTimeline / narrativePageCount, 1) : 1;
const noOrphans = narrativePageCount > 0 ? 1 - (orphanPages / narrativePageCount) : 1;
const noDeadLinks = pageCount > 0 ? 1 - Math.min(deadLinks / pageCount, 1) : 1;
// Bug 11 — per-component points. Sum equals brainScore by construction
// so `doctor` can render a breakdown that adds up to the total.
+22 -5
View File
@@ -5328,9 +5328,18 @@ export class PostgresEngine implements BrainEngine {
const [h] = await sql`
WITH entity_pages AS (
SELECT id, slug FROM pages WHERE type IN ('person', 'company')
),
narrative_pages AS (
-- Composition-aware score: code source files and calendar daily
-- files are orphans-by-design (no inbound wikilinks, no Timeline
-- fence). Excluding them from the orphan/link-density/timeline
-- denominators keeps a bulk code/calendar import from cratering
-- brain_score.
SELECT id FROM pages WHERE type IS NULL OR type NOT IN ('code', 'calendar-index')
)
SELECT
(SELECT count(*) FROM pages) as page_count,
(SELECT count(*) FROM narrative_pages) as narrative_page_count,
(SELECT count(*) FROM content_chunks WHERE embedded_at IS NOT NULL)::float /
GREATEST((SELECT count(*) FROM content_chunks), 1)::float as embed_coverage,
0 as stale_pages,
@@ -5340,7 +5349,8 @@ export class PostgresEngine implements BrainEngine {
) as dead_links,
(SELECT count(*) FROM content_chunks WHERE embedded_at IS NULL) as missing_embeddings,
(SELECT count(*) FROM links) as link_count,
(SELECT count(DISTINCT page_id) FROM timeline_entries) as pages_with_timeline,
(SELECT count(DISTINCT te.page_id) FROM timeline_entries te
WHERE te.page_id IN (SELECT id FROM narrative_pages)) as pages_with_timeline,
(SELECT count(*) FROM entity_pages e
WHERE EXISTS (SELECT 1 FROM links l WHERE l.to_page_id = e.id))::float /
GREATEST((SELECT count(*) FROM entity_pages), 1)::float as link_coverage,
@@ -5361,11 +5371,18 @@ export class PostgresEngine implements BrainEngine {
const islandedRows = await sql<{ slug: string }[]>`
SELECT p.slug
FROM pages p
WHERE NOT EXISTS (SELECT 1 FROM links l WHERE l.to_page_id = p.id)
-- Narrative pages only (same type filter as the narrative_pages CTE):
-- code/calendar-index pages are orphans-by-design and must not count
-- against the noOrphans component (#1144).
WHERE (p.type IS NULL OR p.type NOT IN ('code', 'calendar-index'))
AND NOT EXISTS (SELECT 1 FROM links l WHERE l.to_page_id = p.id)
AND NOT EXISTS (SELECT 1 FROM links l WHERE l.from_page_id = p.id)
`;
const pageCount = Number(h.page_count);
// Composition-aware denominators (excludes code/calendar-index pages);
// a code-only brain has nothing narrative to penalize → full marks.
const narrativePageCount = Number(h.narrative_page_count);
const embedCoverage = Number(h.embed_coverage);
const stalePages = await this.countStalePagesForExtraction({ versionTs: LINK_EXTRACTOR_VERSION_TS });
const orphanOverrides = await loadOrphanPolicyOverrides(this);
@@ -5375,9 +5392,9 @@ export class PostgresEngine implements BrainEngine {
const pagesWithTimeline = Number(h.pages_with_timeline);
// brain_score: 0-100 weighted average
const linkDensity = pageCount > 0 ? Math.min(linkCount / pageCount, 1) : 0;
const timelineCoverageWhole = pageCount > 0 ? Math.min(pagesWithTimeline / pageCount, 1) : 0;
const noOrphans = pageCount > 0 ? 1 - (orphanPages / pageCount) : 1;
const linkDensity = narrativePageCount > 0 ? Math.min(linkCount / narrativePageCount, 1) : 1;
const timelineCoverageWhole = narrativePageCount > 0 ? Math.min(pagesWithTimeline / narrativePageCount, 1) : 1;
const noOrphans = narrativePageCount > 0 ? 1 - (orphanPages / narrativePageCount) : 1;
const noDeadLinks = pageCount > 0 ? 1 - Math.min(deadLinks / pageCount, 1) : 1;
// Per-component points. Sum equals brainScore by construction.
//
+34
View File
@@ -119,6 +119,40 @@ describe('Bug 11 — orphan_pages is "no inbound links"', () => {
});
});
describe('#1144 — brain_score is composition-aware', () => {
test('code and calendar-index pages do not count as orphans or dilute density metrics', async () => {
// Two linked narrative pages + a flood of orphan-by-design pages.
await engine.putPage('people/alice', { type: 'person', title: 'Alice', compiled_truth: 'a', frontmatter: {} });
await engine.putPage('people/bob', { type: 'person', title: 'Bob', compiled_truth: 'b', frontmatter: {} });
const aId = (await (engine as any).db.query(`SELECT id FROM pages WHERE slug='people/alice'`)).rows[0].id;
const bId = (await (engine as any).db.query(`SELECT id FROM pages WHERE slug='people/bob'`)).rows[0].id;
await (engine as any).db.query(
`INSERT INTO links (from_page_id, to_page_id, link_type) VALUES ($1, $2, 'mentions')`,
[aId, bId],
);
for (let i = 0; i < 10; i++) {
await engine.putPage(`code/src/f${i}.py`, { type: 'code', title: `f${i}.py`, compiled_truth: 'def f(): pass', frontmatter: {} });
}
await engine.putPage('daily/calendar/2026-01-01', { type: 'calendar-index', title: '2026-01-01', compiled_truth: 'events', frontmatter: {} });
const h = await engine.getHealth();
// 11 unlinked code/calendar pages exist, but both narrative pages are
// linked → 0 orphans, full no-orphans marks.
expect(h.orphan_pages).toBe(0);
expect(h.no_orphans_score).toBe(15);
// Link density: 1 link / 2 narrative pages, not 1 / 13 total pages.
expect(h.link_density_score).toBe(Math.round(0.5 * 25));
});
test('a brain with ONLY non-narrative pages gets full composition marks', async () => {
await engine.putPage('code/src/only.py', { type: 'code', title: 'only.py', compiled_truth: 'x = 1', frontmatter: {} });
const h = await engine.getHealth();
expect(h.no_orphans_score).toBe(15);
expect(h.link_density_score).toBe(25);
expect(h.timeline_coverage_score).toBe(15);
});
});
describe('Bug 11 — doctor renders brain_score breakdown', () => {
test('doctor source contains brain_score breakdown rendering', async () => {
const source = await Bun.file(new URL('../src/commands/doctor.ts', import.meta.url)).text();
+45
View File
@@ -763,3 +763,48 @@ describeBoth('Engine parity — federated sourceIds[] secondary reads (#2200)',
}
});
});
// #1144 — brain_score is composition-aware: code / calendar-index pages are
// orphans-by-design and must not feed the orphan/link-density/timeline
// denominators. Both engines must agree.
async function seedComposition(eng: BrainEngine) {
await eng.putPage('people/comp-alice', { type: 'person', title: 'Alice', compiled_truth: 'a', timeline: '' });
await eng.putPage('people/comp-bob', { type: 'person', title: 'Bob', compiled_truth: 'b', timeline: '' });
await eng.addLink('people/comp-alice', 'people/comp-bob', 'knows', 'mentions', 'markdown');
for (let i = 0; i < 5; i++) {
await eng.putPage(`code/src/comp-f${i}.py`, { type: 'code', title: `f${i}.py`, compiled_truth: 'def f(): pass', timeline: '' });
}
await eng.putPage('daily/calendar/2026-01-01', { type: 'calendar-index', title: '2026-01-01', compiled_truth: 'events', timeline: '' });
}
describeBoth('Engine parity — getHealth composition-aware brain_score (#1144)', () => {
let pgEngine: BrainEngine;
let pgliteEngine: PGLiteEngine;
beforeAll(async () => {
pgEngine = await setupDB();
await seedComposition(pgEngine);
pgliteEngine = new PGLiteEngine();
await pgliteEngine.connect({});
await pgliteEngine.initSchema();
await seedComposition(pgliteEngine);
}, 90_000);
afterAll(async () => {
await pgliteEngine.disconnect();
await teardownDB();
}, 30_000);
test('orphan/link-density/timeline components identical and exclude code + calendar-index pages', async () => {
const pg = await pgEngine.getHealth();
const pglite = await pgliteEngine.getHealth();
for (const k of ['orphan_pages', 'no_orphans_score', 'link_density_score', 'timeline_coverage_score', 'brain_score'] as const) {
expect(pg[k]).toBe(pglite[k]);
}
// 6 non-narrative pages exist but both narrative pages are linked → 0 orphans.
expect(pg.orphan_pages).toBe(0);
expect(pg.no_orphans_score).toBe(15);
// 1 link / 2 narrative pages, not / 8 total pages.
expect(pg.link_density_score).toBe(Math.round(0.5 * 25));
});
});
+29
View File
@@ -140,6 +140,35 @@ describeE2E('scanIntegrity batch parity (E2E, Postgres-only)', () => {
});
});
describe('code pages', () => {
test('type:code page is skipped on both paths (#1143)', async () => {
const engine = getEngine();
await engine.putPage('people/alice', {
type: 'person',
title: 'Alice',
compiled_truth: 'Alice tweeted about something.',
timeline: '',
frontmatter: {},
});
await engine.putPage('code/src/bot.py', {
type: 'code',
title: 'src/bot.py (python)',
compiled_truth: '# the user tweeted about this feature\ndef send_tweet():\n pass',
timeline: '',
frontmatter: { language: 'python', file: 'src/bot.py' },
});
const batchResult = await scanIntegrity(engine, { limit: 100, batchLoad: true });
const seqResult = await scanIntegrity(engine, { limit: 100, batchLoad: false });
expect(batchResult.pagesScanned).toBe(seqResult.pagesScanned);
expect(batchResult.pagesScanned).toBe(1);
expect(batchResult.bareHits.map(h => h.slug)).not.toContain('code/src/bot.py');
expect(seqResult.bareHits.map(h => h.slug)).not.toContain('code/src/bot.py');
});
});
describe('topPages', () => {
test('topPages ordering matches between paths', async () => {
const engine = getEngine();
+16
View File
@@ -201,6 +201,15 @@ describe('scanIntegrity', () => {
timeline: '',
frontmatter: { validate: false },
});
// Indexed source file — 'tweeted about' in a comment must NOT be flagged
// as a bare-tweet citation gap (#1143: auto-repair would corrupt source).
await engine.putPage('code/src/bot.py', {
type: 'code',
title: 'src/bot.py (python)',
compiled_truth: '# the user tweeted about this feature\ndef send_tweet():\n pass',
timeline: '',
frontmatter: { language: 'python', file: 'src/bot.py' },
});
}, 60_000);
afterAll(async () => {
@@ -223,6 +232,13 @@ describe('scanIntegrity', () => {
expect(slugs).not.toContain('people/legacy');
});
test('skips type:code pages (#1143 — bare-tweet false positives on source files)', async () => {
const res = await scanIntegrity(engine);
const slugs = res.bareHits.map(h => h.slug);
expect(slugs).not.toContain('code/src/bot.py');
expect(res.pagesScanned).toBe(2);
});
test('honors limit', async () => {
const res = await scanIntegrity(engine, { limit: 1 });
expect(res.pagesScanned).toBe(1);
+34
View File
@@ -0,0 +1,34 @@
/**
* #1098 ClawVisor health checks must hit /ready (surfaces db/vault
* degradation), never /health (bare liveness, masks those failure modes).
* #1099 the credential-gateway recipe tells users to activate Google
* Contacts; the canonical consumer recipe must actually ship.
*/
import { describe, test, expect } from 'bun:test';
import { readFileSync, existsSync } from 'fs';
import { join } from 'path';
const RECIPES = join(import.meta.dir, '../recipes');
describe('#1098 — ClawVisor health checks use /ready', () => {
for (const f of ['email-to-brain.md', 'calendar-to-brain.md', 'credential-gateway.md', 'contacts-to-brain.md']) {
test(`${f} references $CLAWVISOR_URL/ready, never /health`, () => {
const text = readFileSync(join(RECIPES, f), 'utf-8');
expect(text).not.toContain('$CLAWVISOR_URL/health');
expect(text).toContain('$CLAWVISOR_URL/ready');
});
}
});
describe('#1099 — canonical contacts-to-brain recipe ships', () => {
test('recipes/contacts-to-brain.md exists with the sibling frontmatter shape', () => {
const p = join(RECIPES, 'contacts-to-brain.md');
expect(existsSync(p)).toBe(true);
const text = readFileSync(p, 'utf-8');
expect(text).toContain('id: contacts-to-brain');
expect(text).toContain('requires: [credential-gateway]');
expect(text).toContain('health_checks:');
expect(text).toContain('contacts.readonly');
});
});