Compare commits

...
Author SHA1 Message Date
Garry Tan 6776df2447 Merge branch 'fix/resolver-warnings' of https://github.com/garrytan/gbrain into fix/resolver-warnings 2026-04-24 02:18:35 -07:00
root 19dbf152c2 fix: resolve all check-resolvable warnings (MECE overlap, DRY, routing)
- RESOLVER.md: broaden query triggers ("who is", "brain search",
  "background on", "notes on this"), quote all citation-fixer triggers,
  add disambiguation rules for citation-fixer vs maintain and
  gbrain-jobs vs minion-orchestrator

- citation-fixer: add "citations are broken" and "fix broken citations"
  triggers to frontmatter

- maintain: rename trigger "citation audit" → "maintenance audit" to
  avoid MECE overlap with citation-fixer

- enrich: add convention delegation ref near notability tier table
  (within DRY_PROXIMITY_LINES window), replace inlined citation format
  rules with reference to conventions/quality.md

check-resolvable --skills-dir ./skills: 0 errors, 0 warnings
2026-04-24 08:52:42 +00:00
root 0c48010d79 fix: resolve all check-resolvable warnings (MECE overlap, DRY, routing)
- RESOLVER.md: broaden query triggers ("who is", "brain search",
  "background on", "notes on this"), quote all citation-fixer triggers,
  add disambiguation rules for citation-fixer vs maintain and
  gbrain-jobs vs minion-orchestrator

- citation-fixer: add "citations are broken" and "fix broken citations"
  triggers to frontmatter

- maintain: rename trigger "citation audit" → "maintenance audit" to
  avoid MECE overlap with citation-fixer

- enrich: add convention delegation ref near notability tier table
  (within DRY_PROXIMITY_LINES window), replace inlined citation format
  rules with reference to conventions/quality.md

check-resolvable --skills-dir ./skills: 0 errors, 0 warnings
2026-04-24 08:22:03 +00:00
6 changed files with 238 additions and 13 deletions
+8 -5
View File
@@ -13,13 +13,14 @@ This is the dispatcher. Skills are the implementation. **Read the skill file bef
| Trigger | Skill |
|---------|-------|
| "What do we know about", "tell me about", "search for" | `skills/query/SKILL.md` |
| "What do we know about", "tell me about", "search for", "search the brain", "brain search", "background on", "notes on this", "who is" | `skills/query/SKILL.md` |
| "Who knows who", "relationship between", "connections", "graph query" | `skills/query/SKILL.md` (use graph-query) |
| Creating/enriching a person or company page | `skills/enrich/SKILL.md` |
| Where does a new file go? Filing rules | `skills/repo-architecture/SKILL.md` |
| Fix broken citations in brain pages | `skills/citation-fixer/SKILL.md` |
| "Fix broken citations", "citations are broken", "fix citations", "citation audit" | `skills/citation-fixer/SKILL.md` |
| "Research", "track", "extract from email", "investor updates", "donations" | `skills/data-research/SKILL.md` |
| Share a brain page as a link | `skills/publish/SKILL.md` |
| "validate frontmatter", "check frontmatter", "brain lint", "fix frontmatter" | `skills/frontmatter-guard/SKILL.md` |
## Content & media ingestion
@@ -58,7 +59,7 @@ This is the dispatcher. Skills are the implementation. **Read the skill file bef
| Cross-modal review, second opinion | `skills/cross-modal-review/SKILL.md` |
| "Validate skills", skill health check | `skills/testing/SKILL.md` |
| Webhook setup, external event processing | `skills/webhook-transforms/SKILL.md` |
| "Spawn agent", "background task", "parallel tasks", "steer agent", "pause/resume agent" | `skills/minion-orchestrator/SKILL.md` |
| "Spawn agent", "background task", "parallel tasks", "steer agent", "pause/resume agent", "gbrain jobs submit", "submit a gbrain job", "submit a shell job", "shell job" | `skills/minion-orchestrator/SKILL.md` |
## Setup & migration
@@ -89,8 +90,10 @@ When multiple skills could match:
1. Prefer the most specific skill (meeting-ingestion over ingest)
2. If the user mentions a URL, route by content type (link → idea-ingest, video → media-ingest)
3. If the user mentions a person/company, check if enrich or query fits better
4. Chaining is explicit in each skill's Phases section
5. When in doubt, ask the user
4. **Citation audit** → use `citation-fixer` (targeted fix). `maintain` includes citation checking as one step of a broader health sweep — use `maintain` only for full brain health runs.
5. **Background task / spawn agent** → use `minion-orchestrator` for spawning and steering agents. `gbrain-jobs` is the lower-level queue CLI.
6. Chaining is explicit in each skill's Phases section
7. When in doubt, ask the user
## Conventions (cross-cutting)
+2
View File
@@ -8,6 +8,8 @@ triggers:
- "fix citations"
- "citation audit"
- "check citations"
- "citations are broken"
- "fix broken citations"
tools:
- search
- get_page
+4 -7
View File
@@ -55,14 +55,9 @@ they building, what makes them tick, where are they headed.
## Citation Requirements (MANDATORY)
Every fact must carry an inline `[Source: ...]` citation.
> **Convention:** See `skills/conventions/quality.md` for full citation format rules.
Three formats:
- **Direct attribution:** `[Source: User, {context}, YYYY-MM-DD]`
- **API/external:** `[Source: {provider} enrichment, YYYY-MM-DD]`
- **Synthesis:** `[Source: compiled from {list of sources}]`
Source precedence (highest to lowest):
Every fact must carry an inline `[Source: ...]` citation. Source precedence (highest to lowest):
1. User's direct statements
2. Compiled truth (pre-existing brain synthesis)
3. Timeline entries (raw evidence)
@@ -89,6 +84,8 @@ When sources conflict, note the contradiction with both citations.
Scale enrichment to importance. Don't waste API calls on low-value entities.
> **Convention:** See `skills/_brain-filing-rules.md` for the notability gate and filing rules.
| Tier | Who | Effort | Sources |
|------|-----|--------|---------|
| 1 (key) | Inner circle, close collaborators, key contacts | Full pipeline | All available APIs + deep web research |
+218
View File
@@ -0,0 +1,218 @@
---
name: frontmatter-guard
version: 1.0.0
description: |
Validates and auto-repairs frontmatter YAML on every brain page write.
Gate that prevents malformed pages from entering the brain. Import
writeBrainPage() instead of raw writeFileSync for any /data/brain/ write.
triggers:
- "validate frontmatter"
- "check frontmatter"
- "brain lint"
- "fix frontmatter"
tools:
- exec
- read
- write
mutating: true
---
# Frontmatter Guard
> Every brain write goes through the guard. No exceptions.
## Why This Exists
On 2026-04-24, a brain health audit found 203 pages with malformed frontmatter:
- 111 people pages missing closing `---` (entity detector bug)
- 43 meeting pages with unstructured YAML (ingestion bug)
- 16 files with slug mismatches
- 11 with binary corruption
- 4 with nested quote escaping
All written by our own agents. The guard prevents this class of error.
## The Library
**Location:** `lib/brain-writer.mjs` (in the OpenClaw workspace)
### Core API
```javascript
import { writeBrainPage, validateFrontmatter, autoFixFrontmatter } from '../lib/brain-writer.mjs';
// 1. Validated write (throws on bad frontmatter)
writeBrainPage('/data/brain/people/jane-doe.md', content);
// 2. Validated write with auto-repair
writeBrainPage('/data/brain/people/jane-doe.md', content, { autoFix: true });
// 3. Validate only (no write)
const result = validateFrontmatter(content, { filePath: '/data/brain/people/jane-doe.md' });
// → { ok: true/false, errors: [{ code, message }] }
// 4. Auto-fix only (returns fixed content)
const { content: fixed, fixes } = autoFixFrontmatter(content, { filePath });
```
### What It Validates
| Check | Error Code | Description |
|-------|-----------|-------------|
| Opening `---` | `MISSING_OPEN` | File doesn't start with frontmatter |
| Closing `---` | `MISSING_CLOSE` | No closing delimiter (heading found inside YAML zone) |
| YAML parse | `YAML_PARSE` | js-yaml can't parse the frontmatter block |
| Slug match | `SLUG_MISMATCH` | `slug:` field doesn't match file path |
| Null bytes | `NULL_BYTES` | Binary corruption in content |
| Nested quotes | `NESTED_QUOTES` | `title: "Name "Nick" Last"` pattern |
| Empty frontmatter | `EMPTY_FRONTMATTER` | Frontmatter block is empty |
### What It Auto-Fixes
| Fix | Description |
|-----|-------------|
| Missing `---` | Inserts closing delimiter before first heading |
| Nested quotes in title | `"Name "Nick" Last"``'Name "Nick" Last'` |
| Nested quotes in lists | Investor notes with inner quotes → inner singles |
| Bracket titles | `title: [Name``title: "Name"` |
| Slug removal | Removes `slug:` field (gbrain derives from path) |
| Null bytes | Strips `\x00` characters |
### Path Guard
```javascript
// This THROWS — path is not under /data/brain/
writeBrainPage('/data/.openclaw/workspace/brain/people/test.md', content);
// Error: writeBrainPage: path is not under /data/brain/
```
This prevents the #1 brain write bug: writing to the workspace `brain/` subdirectory instead of the actual brain repo.
## Pre-Commit Hook
**Location:** `/data/brain/.githooks/pre-commit`
Runs on every `git commit` in the brain repo. Checks staged `.md` files for:
1. Missing closing `---`
2. YAML parse errors (via js-yaml from workspace node_modules)
3. Null bytes
Blocks the commit with actionable errors. Bypass: `git commit --no-verify`.
## Integration Rules for Agents
### When writing a brain page directly (writeFileSync)
**ALWAYS** use `writeBrainPage()` instead:
```javascript
// ❌ BAD — no validation, silent corruption
import { writeFileSync } from 'node:fs';
writeFileSync('/data/brain/people/jane-doe.md', content);
// ✅ GOOD — validates, blocks bad writes
import { writeBrainPage } from '../lib/brain-writer.mjs';
writeBrainPage('/data/brain/people/jane-doe.md', content);
```
### When generating frontmatter in a prompt
Always include the closing `---`:
```markdown
---
title: "Person Name"
type: person
created: 2026-04-24
---
# Person Name
```
### When titles contain special characters
Use single quotes for titles with inner double quotes:
```yaml
# ❌ BAD
title: "Phil Libin's Journey to Finding a "Life's Work""
# ✅ GOOD
title: 'Phil Libin''s Journey to Finding a "Life''s Work"'
# ✅ ALSO GOOD
title: "Phil Libin's Journey to Finding a Life's Work"
```
### When values contain colons
Always quote values with colons:
```yaml
# ❌ BAD — YAML thinks everything after the colon is a new key
garry_context: Fucking sick coding song — one of Garry's favorites
# ✅ GOOD
garry_context: "Fucking sick coding song — one of Garry's favorites"
```
## Running a Brain-Wide Audit
```bash
cd /data/.openclaw/workspace && node -e "
import { validateFrontmatter } from './lib/brain-writer.mjs';
import { readFileSync, readdirSync, statSync } from 'node:fs';
import { join } from 'node:path';
function walk(dir, files = []) {
for (const f of readdirSync(dir)) {
if (f === '.git') continue;
const p = join(dir, f);
if (statSync(p).isDirectory()) walk(p, files);
else if (f.endsWith('.md')) files.push(p);
}
return files;
}
let valid = 0, invalid = 0;
for (const file of walk('/data/brain')) {
const content = readFileSync(file, 'utf8');
if (!content.startsWith('---')) continue;
const r = validateFrontmatter(content, { filePath: file });
if (r.ok) valid++; else invalid++;
}
console.log('Valid:', valid, '| Invalid:', invalid, '| Rate:', (valid*100/(valid+invalid)).toFixed(1) + '%');
"
```
## Batch Auto-Fix
```bash
cd /data/.openclaw/workspace && node -e "
import { validateFrontmatter, autoFixFrontmatter } from './lib/brain-writer.mjs';
import { readFileSync, writeFileSync, readdirSync, statSync } from 'node:fs';
import { join } from 'node:path';
// ... walk function ...
let fixed = 0;
for (const file of walk('/data/brain')) {
const content = readFileSync(file, 'utf8');
if (!content.startsWith('---')) continue;
if (validateFrontmatter(content).ok) continue;
const result = autoFixFrontmatter(content, { filePath: file });
if (result.fixes.length > 0 && validateFrontmatter(result.content).ok) {
writeFileSync(file, result.content);
fixed++;
}
}
console.log('Fixed:', fixed, 'files');
"
```
## Upstream Path
Once battle-tested here, the validator moves into gbrain's core:
1. `src/core/frontmatter.ts` — the validation + auto-fix logic
2. Integrated into `putPage()` / `upsertPage()` — every DB write validates
3. `gbrain lint` CLI command — runs the audit
4. `gbrain lint --fix` — runs auto-repair
5. Pre-commit hook ships with `gbrain init`
@@ -0,0 +1,5 @@
// Routing eval fixtures for skills/frontmatter-guard. Check 5 (W2, v0.19).
{"intent": "can you validate the frontmatter on these brain pages I just wrote", "expected_skill": "frontmatter-guard"}
{"intent": "run a brain lint to find broken frontmatter across the repo", "expected_skill": "frontmatter-guard"}
// Negative: general brain health is maintain, not frontmatter-guard
{"intent": "check overall brain health and run maintenance", "expected_skill": null, "ambiguous_with": []}
+1 -1
View File
@@ -8,7 +8,7 @@ description: |
triggers:
- "brain health"
- "check backlinks"
- "citation audit"
- "maintenance audit"
- "maintenance"
- "orphan pages"
- "stale pages"