admin/gbrain
1
0
Fork 0
mirror of https://github.com/garrytan/gbrain.git synced 2026-08-14 00:48:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
963 Commits 590 Branches 27 Tags
master
Commit Graph
39 Commits
Author SHA1 Message Date
Garry TanandClaude Opus 4.8 a996e42856 v0.45.2.0 fix(bootstrap): create-repo-first repo adoption + hardening (#4024) 
* fix(bootstrap): harden create-repo-first repo adoption

`gbrain bootstrap repo` adopts an empty, private, personally-owned GitHub repo
the human created (create-repo-first), instead of only ever creating one. This
hardens the existing adoption branch:

- Empty-only adoption + pending_repo_url proof: a non-empty origin is refused
  (ORIGIN_NOT_EMPTY) unless it matches this workspace's pending marker (our own
  interrupted push). Never adopts a user's existing project from a git-ancestry
  guess, and never silently no-ops without pushing.
- Repo-local git identity is set on the adopt path too (fresh-machine commits).
- repo_url is recorded only AFTER a successful push (pending marker before);
  a failed push no longer looks "done" to `bootstrap status`.
- Pre-push secret scan also covers an already-committed tree; ls-files failure
  fails closed.
- assertOriginMatches binds BOTH the fetch URL and a configured push URL to the
  verified-private repo, so a foreign pushurl can't leak the workspace.
- disposition: 'created' | 'adopted' | 'reused' replaces the overloaded flag.
- Hook push-gate: the no-daemon session-end / recovery push is deferred until
  the repo phase records repo_url AND the current origin still matches it, so
  nothing is published to an unverified or redirected remote.

Adds ORIGIN_NOT_EMPTY / REMOTE_CHECK_FAILED error codes.

* docs(bootstrap): lead with the repo, document create-repo-first

README (Claude Code + Codex) now opens with "the folder you open becomes your
agent's private repo" and adds a "prefer to make the repo yourself?" callout for
the create-repo-first path (empty, personal-account repo). Updates the bootstrap
guide, the Claude Code MCP note, and the KEY_FILES / AGENT_BOOTSTRAP_PLAN
invariants to describe adoption instead of "foreign origins refused".

* v0.45.1.0 fix(bootstrap): create-repo-first repo adoption + hardening

Bumps VERSION/package.json to 0.45.1.0, adds the CHANGELOG entry, refreshes the
runbook + template-repo version stamps, and regenerates the llms bundle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(todos): file P2 follow-up — index-blob secret scan for bootstrap pushes

* ci(gitleaks): run the free CLI instead of the license-gated v2 action

gitleaks-action@v2 now enforces a paid GITLEAKS_LICENSE and fails the job
("missing gitleaks license") for accounts it can't validate over the API —
blocking every PR's merge gate. Replace it with the open-source gitleaks CLI
(pinned 8.30.1, checksum-verified against the release's own checksums file),
scanning the PR/push commit range with the committed .gitleaks.toml allowlist.
Same secret-scan coverage, no license wall.

* v0.45.2.0 chore(release): re-bump 0.45.1.0 -> 0.45.2.0

Re-target the release version at the user's request. Updates VERSION,
package.json, the CHANGELOG header + self-repair block, the runbook +
template-repo version stamps, the TODOS follow-up reference, and the llms bundle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(perf): raise entity-card ratio ceiling 50x -> 100x (CI flake)

The RATIO GUARD asserted entity p99 <= 50x max(getPage p50, 1ms). On a fast
runner getPage p50 floors to 1ms and a normal entity p99 (~50ms) reads as ~52x,
tripping the gate even though absolute p99 (52ms) is well under the 100ms budget
— a p99 tail divided by a sub-ms median. At the 1ms floor, 50x also made the
ratio STRICTER than the test's own 100ms absolute budget. Raise the ceiling to
100x: still far below the >=200x O(N)-regression signal the guard exists to
catch, and consistent with (never stricter than) the absolute budget.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-11 23:01:22 -07:00
Garry TanandClaude Fable 5 d35c9c9e44 v0.45.0.0 feat(bootstrap): paste-in personal-agent install for Codex + Claude Code (#3975) 
* docs(designs): agent-bootstrap plan + design docs (normative, review-absorbed)

The scrubbed, in-repo sources of truth for the gbrain bootstrap wave:
AGENT_BOOTSTRAP_DESIGN.md (product scope/sequencing) and
AGENT_BOOTSTRAP_PLAN.md (implementation; all review-finding IDs inlined).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): format spec, question bank, identity templates, bundled assets

agent.json manifest (format_version 1, initialized sentinel) + machine-local
install receipt [CX2-1, CX2-12]; 12-question/6-required interview bank with
consent keys and a persist:false sink for the optional provider key [CX2-13];
ten {{TOKEN}} identity templates (generic, adapted to gbrain ops — gates call
recall/query/put_page, write-through-ops rule, keyless agent-authored facts,
silence contract); assets embedded compiled-binary-safe via file-type imports
[ENG-6].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): runbook, README paste block, bootstrap guide, TODOS entries

BOOTSTRAP_FOR_AGENTS.md (agent-driven install runbook: CLI phase list is the
source of truth, never-invent rules, Codex approvals preflight, keyless posture,
failure-modes table, version stamp for the skew check); README gains the
full-agent paste block pinned to latest-stable inside the Claude Code/Codex
quick start (memory-only tier stays); docs/guides/bootstrap.md carries the full
install/security/consent/degradation/uninstall contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(designs): spike instrument for the bootstrap wave (build order 0)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): interview + render engines

Interview gate with read-back confirm-hash (any later answer change clears the
confirmation — the hostile single-batch case is structurally impossible),
per-answer provenance, caps + escaping at set time, config-sink routing for the
provider key; renderer with hard-fail token sweep, subordinate fencing of
principal input, never-clobber + backups, deterministic minimal mode for the
template repo, scaled byte floors. 58 unit tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): private-repo lifecycle — repo create, attach, uninstall, run lock

gh-gated private repo creation with API-verified privacy (rate-limit distinct
from public), refuse-foreign-origin with attach as the sanctioned path, atomic
bootstrap mutex (pid liveness + age + token), receipt-keyed uninstall that
never wholesale-deletes the gbrain home and only offers --delete-brain for a
brain it created; read-only PGLite lock probe (never opens the engine).
54 unit tests, injectable exec seam throughout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(release): latest-stable ref, template-repo publish job, bootstrap CI guards

release.yml advances the latest-stable tag only after assets publish (the paste
block's permanent ref — copies in the wild never rot) and gains a PAT-gated
publish-template job verified against the vendored tree; two skip-graceful
guards (sanctioned-ref + runbook stamp; template/token bijection + placeholder
assertion + generator byte-diff) wired into verify; README + runbook re-admitted
to the CI cache hash; vendored deterministic template tree generated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(context): IPC v2 turn_context + 8KB assembly + visibility resolver + session identity

Discriminated-union IPC with handler map, protocol echo (stale-serve detection),
shared-secret gate, server-side source binding, per-kind budgets; turn-context
assembly (reflex pointers + volunteered pages + world-only hot facts) under a
data-not-instructions envelope trimmed to the harness's 10KB hook-output cap;
facts.default_visibility resolved through one helper at all four sites (explicit
caller wins, typos fail closed); typed sessionId threads _meta.session_id into
the hot-memory cache key. 50 new tests; 180 adjacent tests confirmed green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(persistence): secret-scan, gbrain sources push, durability unification

Pattern secret scanner (own runtime allowlist, redacted previews, corpus-write
redaction mode); sources push runs the whole scan→stage→commit→pull→push
sequence under one cross-platform lock (mkdir-atomic, pid+age+token) with a
deny-glob backstop, commit-first divergence-safe pull, refuse-unverifiable
visibility, and push-status telemetry; gbrain-home choke point unifies
GBRAIN_HOME semantics with config (0700); durability is parent-repo-aware and
rotates its push log at 0600. 35 new tests; 200 existing green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sources): harden/pull gates accept sources inside a parent git repo

The bootstrap workspace registers brain/ (a subdirectory) as the source; the
durability core already resolves the repo root, so the command gates now check
inside-a-repo rather than .git-right-here [CX2-3].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(serve): resident maintenance sweep + keyless capability probe

The lock-owning serve process now closes the persistence loop: startup (3s
post-connect, best-effort, unref'd) and idle (10-min quiet intervals through
the injectable timer seam) sweeps run facts-fence reconciliation, deterministic
link/timeline extraction over recent workspace pages, and spend-gated corpus
ingest (skipped keyless — agent-authored fences cover it). gbrain sweep --once
is the trusted CLI seam bootstrap verify uses. Capability probe renders the
honest keyless/keyed report. Full reuse of the cycle extractor + extract cores;
26 new tests, neighbors green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(hooks): engine-free gbrain hook command, settings writers, transcript parser

Four hook events (session-start digest + crashed-session recovery push,
user-prompt turn-context injection under an 800ms deadline and the 10KB cap,
stop buffers, session-end corpus write with redaction/retention/dedup +
best-effort push); structural JSON settings merger keyed by a _gbrain marker
(foreign hooks and permissions survive); dated host-spec registry; Claude Code
.jsonl parser as a spec-target with a scrubbed 7-shape fixture. Heartbeat is
counters-only by construction. 59 tests; zero engine modules in the import
graph.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(bootstrap): cross-link the full-agent path from the connection docs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): dispatcher, verify, status — the command assembled

gbrain bootstrap {status,interview,render,repo,hooks,verify,uninstall,attach}:
engine-free except verify (owns its engine, in-process sweep — no live-serve
conflict); phase list is the TS source of truth with install.jsonl telemetry
and the support blob; verify's fail-soft check suite covers the real write path
(put_page → write-through file → sweep → graph floor → recall), passes keyless,
persists snapshots, and ends with the first-run tour. cli.ts wired per the
three-touchpoint rule; doctor gains the bootstrap check group (silent on
machines with no bootstrap state). 28 new tests; 353 adjacent green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: KEY_FILES bootstrap cluster + CLAUDE.md dispatcher row (+ build:llms)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(bootstrap): e2e pins — hook-under-live-serve, attach, degraded modes, compiled binary, Docker harness

The permanent pins: a real serve holds the PGLite lock while the engine-free
hook completes (and a direct engine open provably throws LiveServeLockError);
stale-socket fail-open; machine-2 attach with marker-keyed hook repair;
decline-everything installs verify green with every degradation named; the
compiled binary renders bundled templates in an empty cwd. Offline Docker
harness (networkless, read-only) gated into heavy-tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bootstrap): register doctor check categories + system-of-record allow comments

The six bootstrap doctor checks join OPS_CHECK_NAMES; the sweep's batch link/
timeline inserts carry the explicit extract-path allow comments (the sweep IS
the extraction path for workspace pages).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): shard wedge cap tracks suite growth (1500s -> 1800s)

At ~9000 tests a healthy shard finished at 1466s and two progressing shards
were false-killed at the old cap; 1800s restores ~25% headroom over the
slowest observed healthy shard. Real hangs still hit it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(ci): cache-hash policy — README + runbook edits must invalidate [C2]

The old deny-list assertion predates the paste block; README.md and
BOOTSTRAP_FOR_AGENTS.md are policy-doc re-admissions now, so their edits must
change the hash (a paste-block edit shipping under a cached green was the C2
hole).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): classify post-suite exit-hangs as warn-pass; file the leak forensics

A shard killed by the wedge watchdog with every assigned file started and zero
fail markers did all its work and leaked a handle at exit — pre-existing and
master-reproducible (P1 TODO carries the full bisect forensics). Bun's per-test
timeout turns a hung test into a (fail), so the classifier cannot mask one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): shard cap 2400s — the count-balanced heavy shard needs it under contention

Observed: the heavy shard still progressing 22s before an 1800s kill while
siblings finish at 1150-1550s (split balances file count, not weight). Filed
the load-sensitive WAL-repair flake (pre-existing, master's v0.42.75.0 wave).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(bootstrap): quarantine env-mutating suites to the serial lane

check-test-isolation R1: six new files mutate GBRAIN_HOME/env at module scope —
the serial lane (one process per file) is the guard's prescribed home for them.
All 114 tests pass post-rename.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(readme): per-harness install sections — Codex, Claude Code, then OpenClaw/Hermes

Each harness gets its own complete paste-block section (desktop app first,
terminal noted — Claude Code CLI is the identical harness; Codex CLI works
pull-based today); the OpenClaw/Hermes platform path keeps equal weight with
its one-click deploys and INSTALL_FOR_AGENTS block intact; memory-only and
remote-connect tiers consolidated under 'Lighter ways in'. Supersedes the
review's D5 ordering by user direction; stale heading references updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(readme): Codex as the recommended first step; OpenClaw/Hermes framed as-intended, high-cost

The install section now routes newcomers explicitly: Codex first
(subscription-priced, nothing to deploy), OpenClaw/Hermes as GBrain used the
way it was designed — always on, at real server + API cost.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: doc-audit code chasers — broken recovery hints, stale op description, auto_link key

Four small code fixes surfaced by the markdown accuracy audit:
- doctor's auto-RLS recovery hint pointed at `apply-migrations --force-retry 35`,
  which cannot work (--force-retry targets the vX.Y.Z orchestrator registry, not
  the numeric schema MIGRATIONS array). Hint now points at the recreate SQL in
  docs/guides/rls-and-you.md; test pins against regression.
- v0_11_0 migration printed the same broken-mechanism class of hint
  (`config set minion_mode` writes DB config nothing reads); now names
  `apply-migrations --mode` + preferences.json, the real setter.
- submit_job's op description hardcoded a stale handler list; now points at
  registerBuiltinHandlers as the source plus the --follow discovery trick.
- `auto_link` added to KNOWN_CONFIG_KEYS: read by link-extraction, reconcile-links,
  and sweep, and documented as the off-switch in brain-ops/maintain, but the
  allowlist rejected `config set auto_link false`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: repo-wide accuracy + MECE reform from the 9-bucket markdown audit

A code-grounded audit of every markdown file (root, architecture, guides,
mcp, tutorials, docs-root, operations/eval/designs, skills, recipes) followed
by a fix wave with per-bucket ownership. Four classes of change:

Accuracy — every documented command/flag verified against src/ before writing:
dead commands replaced with working ones (pages purge-deleted, jobs watch
--follow, gbrain restore, import-based Obsidian flow, space-separated --scopes,
real thin-client recipes, working isolation verification, real supervisor
restart procedure, curl-based ngrok health check, real minion_mode setter);
count drift fixed with rot-proof phrasing (100+ ops, 50+ bundled skills via
skills/manifest.json, 140+ engine methods, KNOBS_HASH_VERSION pointer instead
of hardcoded versions); stale claims corrected (search-mode defaults, RETRIEVAL
pipeline order incl. autocut, sentinel rules, refusal-list mechanism, engine
snapshot, shard cap 2400s + EXIT-HANG classifier in TESTING.md, latest-stable +
publish-template documented in RELEASING.md as release.yml promises).

MECE — one home per concept, pointers elsewhere: test isolation → TESTING.md;
OAuth registration + --bind/--public-url lore → DEPLOY.md; mode bundles →
guides/search-modes.md (the home the CLAUDE.md dispatcher always promised);
merge contract → schema-packs.md; WAL ladder → ENGINES.md; quiet-hours →
quiet-hours.md; capture taxonomy → entity-detection.md; person-page taxonomy →
compiled-truth.md; brain-first protocol → brain-first-lookup.md; refresh
semantics → refresh-algorithm.md; KEY_FILES.md deduplicated (58 extension
entries merged, one entry per file); infra-layer.md rewritten as a pointer page.

Privacy — placeholder sweep across guides, docs, skills, and recipes per the
iron rule; per-release narration stripped from reference docs (current-state
prose only).

Bootstrap coverage — AGENTS.md pointer, RESOLVER routing row, INSTALL.md path,
tutorial cross-links, keyless-mode sections in spend-controls/headless-install.

skills.lock.json regenerated; llms.txt/llms-full.txt rebuilt. Gates: verify
36/36, typecheck clean, doctor 96/96, skills-integrity + resolver + build-llms
+ config-set + migrations all green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(readme): refresh production-brain stats to current brain-repo counts

155,795 pages / 24,589 people / 5,340 companies, counted from the brain
repo's current HEAD; the "100K-page brain" framing moves to 150K to match.
Cron-fleet count unchanged (its store lives on the deployment host, not in
the repos available for verification).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(scan,push): modern OpenAI/Voyage key patterns, scan staged blobs not disk

- secret-scan matches sk-proj-/sk-svcacct-/sk-None- and pa- Voyage keys
  (the bare sk- pattern missed every current OpenAI key format).
- workspacePush stages first, then scans the staged index blobs via
  git cat-file, closing the scan-then-stage TOCTOU where a file changed
  between snapshot and commit shipped unscanned.
- shared binary-sniff helper, memoized glob regexes, atomic push-status
  write, and tests for pull_conflict + gitignored deny-match paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ci): regenerate flag registry for new commands, harden shard classifier + release token

- cli-flag-registry.generated.ts regenerated: bootstrap/hook/sweep and
  sources push --message/--allow-unverified-remote were missing, so the
  strict #2185 validator rejected real invocations and skipped the new
  commands entirely.
- EXIT-HANG shard classifier now requires every assigned file to have
  started before warn-passing a watchdog kill (was fail-open).
- release.yml passes TEMPLATE_REPO_PAT via http.extraheader, off the argv.
- compiled-binary e2e fails loud in CI instead of a silent permanent skip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(hook,sweep,ipc): non-blocking hook pushes, bounded sweep + cache, source-bound resolve

- session-start/session-end no longer run synchronous git + inline push
  inside their self-deadline; a detached child does the push and the hook
  returns immediately (blocked Claude Code startup for minutes on a dirty
  tree before).
- serve sweep drops the unbounded listAllPageRefs, resolves only candidate
  targets, claims corpus files atomically (no double-LLM-spend race), and
  caps the fence LIKE scan; heartbeat writes are O_APPEND with rare compaction.
- hot-memory cache evicts expired entries and bounds entry count (the key is
  caller-controlled via _meta.session_id).
- v1 resolve IPC honors boundSourceId like turn_context; turn-context runs
  its arms concurrently. doctor reads push/heartbeat thresholds from hook.ts.
- new tests: doctor bootstrap checks, hook push-gate + deadline, concurrent
  sweep claims, cache eviction, bound-source resolve.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bootstrap): origin-ownership gate, world visibility, collision-safe source id, consent + templates

- repo adoption requires an exact receipt repo_url match or authed-owner
  check (undefined repo_url was a wildcard); create verifies privacy BEFORE
  the first push.
- verify sets facts.default_visibility=world if unset, so agent-authored
  facts surface in per-turn context (they defaulted private before).
- source_id derives a path-hash suffix when 'workspace' is taken by another
  checkout; every consumer reads manifest.source_id.
- skipped HOOKS_CONSENT now declines (was falling through to default yes);
  --minimal refuses on an initialized manifest; tilde fences escaped.
- MCP registration pins --surface full; status hard-fails a public origin
  (template door); receipt writers guard against newer/corrupt receipts;
  uninstall only claims brain-deleted after a real rm.
- templates ship jobs disabled + provider-consent + support-relay lines;
  soul-audit re-runs over the shared interview bank. TODOS: 11 follow-ups.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(security): close adversarial-review findings — scan fails closed, whole-PEM redaction, bound repo push

Cross-model adversarial pass (Claude + Codex) on the bootstrap wave:

- secret scan fails CLOSED: an unreadable, oversized, or binary staged blob
  now blocks the push (blocked_unscannable, exit 5) instead of committing
  unscanned; only a confirmed staged deletion is skipped. This was the
  headline "block secrets before they leave the machine" property failing open.
- private-key redaction spans the whole PEM block (header+body+footer), not
  just the header line — the base64 body no longer survives into the corpus
  the sweep sends to an extraction provider.
- bootstrap repo commits the workspace (secret-scan-gated) before the first
  push and verifies the remote actually received it, so a push-fail retry
  can't adopt an empty remote as success.
- privacy verify is re-bound to origin immediately before push (a concurrent
  origin rewrite between verify and push is refused).
- session-end corpus write is atomic and clears the stale ingested/in-progress
  sidecars so a resumed session's appended transcript is re-ingested.
- public-origin refusal enforced at render (not only status); MCP "already
  registered" is verified to target this workspace, not blessed blindly;
  verify probe cleanup scopes deletes to its own slugs, not a token substring;
  allowlist fingerprint floor raised 8→16 hex.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* v0.45.0.0 feat(bootstrap): paste-in personal-agent install for Codex + Claude Code

Turns a Codex or Claude Code session into a persistent personal agent:
interview-rendered identity files, a local PGLite brain, per-turn context via
serve IPC (Claude Code hooks / Codex pull protocol), session-triggered
persistence, and a private GitHub repo as the agent's portable body. Keyless-
first (the harness model is the LLM; one optional key adds embeddings +
extraction). New `gbrain bootstrap` command family + `gbrain hook` + `gbrain
sweep`; doctor bootstrap health checks; latest-stable distribution ref +
template-repo publish job. Opt-in, additive — existing installs untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: regenerate flag registry for security-fix flags; drop fabricated gbrain capabilities doc ref

CI caught two real failures under the merged state:
- the flag registry lagged the blocked_unscannable/exit-5 flags the security
  round added, tripping the #2185 freshness guard.
- headless-install.md described the keyless capability report as a
  `gbrain capabilities` command, which the #3502 doc-command resolver
  rejects — reworded to prose (the real surface is bootstrap verify's report).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: sync KEY_FILES + bootstrap plan to security-fix behavior

Cross-referenced the security-fix round against the reference docs and
corrected the drift those commits introduced:

- workspace-push.ts entry: stage-FIRST-then-scan order (the TOCTOU fix),
  fail-closed blocked_unscannable, and the sources-push status -> exit-code map.
- hooks.ts entry: MCP registration pins `serve --surface full`.
- hook.ts entry: session-start/session-end pushes run in a detached child
  (non-blocking); atomic corpus write clears stale sidecars.
- bootstrap.ts entry: render hard-refuses a public origin (template door).
- verify.ts entry: source_id collision resolution (workspace-<path-hash>).
- AGENT_BOOTSTRAP_PLAN as-shipped delta note for the scan/stage reorder.

llms bundle unchanged (KEY_FILES is link-only); build:llms and
test/build-llms.test.ts green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ci): silence SC2016 on the intentional askpass literal in release.yml

The one-shot GIT_ASKPASS script must contain literal $1 and
$TEMPLATE_REPO_PAT so they expand when /bin/sh runs it at git's credential
prompt, not when the outer shell writes the file — single quotes are correct.
Add a scoped shellcheck disable so actionlint passes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(skills): declare 'bootstrap my data' trigger in cold-start frontmatter

The doc reform added 'bootstrap my data' to cold-start's RESOLVER.md row (to
disambiguate data-bootstrap from agent-bootstrap) but not to the skill's own
frontmatter triggers, tripping the RESOLVER↔frontmatter round-trip contract
(resolver.test.ts). Declare it; regenerate skills.lock.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): default per-turn hooks + search mode ON without a prompt

Installing gbrain for your coding agent IS the consent for the behaviors that
make it work, so stop re-litigating them with install-time questions whose
"no" defeats the product:

- Per-turn hooks (Claude Code) install ON by default — no prompt. Off-ramps:
  `--no-hooks` at install, `GBRAIN_HOOKS=0` at runtime, `bootstrap uninstall`.
  The "hooks installed" line now surfaces the kill switch so default-on is
  never silent. A persisted HOOKS_CONSENT=no (interview --skip) still declines.
- Search mode defaults to `balanced` silently (nobody knows the modes at
  install; `gbrain search modes` changes it any time).
- MCP scope stays the ONE deliberate prompt — project vs user is a real
  cross-repo privacy choice, not friction.

Marks the two consents `silent: true` in the question bank (new QuestionSpec
field), rewrites the runbook phases so the agent no longer asks them, adds the
`--no-hooks` flag (+ registry regen), and adds default-on / opt-out tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(bootstrap): real end-to-end coverage — cross-session recall, per-turn content, Codex door, realistic corpus

Closes the seven e2e gaps a coverage audit surfaced: the plumbing was
well-unit-tested but the product claims ("Codex works, context shows up every
turn with real content, it remembers across restarts, machine two recovers,
Postgres works") were unproven end to end. Test-only wave — zero src changes.

- Hermetic synthetic corpus (test/fixtures/bootstrap-corpus/ + a loader helper):
  12 interlinked pages (52 edges, timelines), 12 world/private beliefs, 8 gold
  queries — curated from the gbrain-evals synthetic corpora, 100% placeholder
  names, so recall is asserted on a real multi-entity brain instead of a
  2-node self-planted probe.
- GAP1 magic moment: author a fact via the real write path, disconnect the
  engine, reopen against the same DB, recall it — a real session boundary, not
  verify.ts's same-connection SQL read-back. Plus a source-isolation assertion.
- GAP2 per-turn content: hook-under-serve Pin 1 now seeds a known fact and
  asserts its text lands in the injected block AND private beliefs never do
  (was: empty brain, empty_block accepted as a pass).
- GAP3 Codex door: assert the rendered AGENTS.md carries the Gate-3 brain-first
  pull protocol; make the fake codex shim implement `mcp get` so the [FIX7]
  target-verification can actually fail; the Docker cold-machine harness now
  exercises the hooks/MCP registration step instead of skipping it.
- GAP4 corpus recall: turn-context + verify graph-floor/qrels run on the real
  multi-entity brain with real edges.
- GAP5 attach: machine-two now re-ingests the cloned brain/ into a fresh DB and
  recalls a fact authored only on machine one — the multi-device payoff.
- GAP6 keyed + Postgres (env-gated): real embeddings prove semantic recall a
  paraphrase query can reach but keyless BM25 cannot; bootstrap verify drives a
  real Postgres engine (skipIf DATABASE_URL/keys absent).
- GAP7 persistence: session-end runs the REAL push (not the mocked seam) to a
  local bare remote and the remote receives the content; a planted secret is
  blocked at the gate; the 15-min cron installs and fires a scan-gated push.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(bootstrap): real-agent e2e — drive the actual claude + codex binaries end to end

Closes the audit's biggest gap ("no real harness ever drives a turn"). Adapts
gstack's PTY/headless agent harness to prove the bootstrap install + smoke work
against the REAL binaries, not PATH shims. Test/CI/docs only — zero src changes.

- test/helpers/agent-harness.ts: hermetic clean-room child env (ported from
  gstack; drops CONDUCTOR_/CLAUDE_/GSTACK_/MCP_/GBRAIN_, promotes
  GSTACK_ANTHROPIC_API_KEY→ANTHROPIC_API_KEY), real-binary resolvers + auth
  probes, headless `claude -p --output-format stream-json` and `codex exec
  --json` turn runners, a gbrain stdio MCP-config writer, and a keyless brain
  seeder. + a fixture-parse unit test (no binary needed).
- test/e2e/bootstrap-real-claude.serial.test.ts: real `gbrain bootstrap`
  install → REAL `claude mcp add` (verified via `claude mcp get`) → verify
  exit 0 → a real `claude -p --mcp-config --strict-mcp-config` turn that
  invokes mcp__gbrain__search and answers from the brain (proven: toolCalls
  include mcp__gbrain__search, final text carries the seeded fact).
- test/e2e/bootstrap-real-codex.serial.test.ts: same install with REAL `codex
  mcp add` into a real ~/.codex/config.toml + Gate-3 pull-protocol assertion,
  then a real `codex exec --json` turn surfacing the fact (MCP or the pull-
  protocol shell path). Bounded retry absorbs codex's occasional MCP-call
  cancellation without softening the fact-requiring assertion.
- Everything hermetic (temp HOME/CLAUDE_CONFIG_DIR/CODEX_HOME/GBRAIN_HOME;
  real ~/.codex auth copied read-only) and skipIf-gated so it self-skips
  cleanly where the binaries/auth are absent.
- heavy-tests.yml: gated `real-agent-e2e` job (nightly/label, never the PR
  shard; no-op on a runner without authed binaries).
- TODOS: compiled `gbrain` binary can't serve a PGLite brain (bun compile
  omits the WASM/extension payloads); harness falls back to `bun run` serve.

Verified against live claude 4.6 + codex 0.147.0: 15 pass / 0 fail; verify
36/36; typecheck clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ci): real-agent-e2e job — bash array + --timeout (actionlint SC2086 + bun-test-timeout guard)

The real-agent-e2e job's file loop used an unquoted $FILES (SC2086) and ran
`bun test` without --timeout (check-bun-test-timeout guard). Switch to a bash
array and add --timeout=600000 (real-agent turns are slow; the tests self-skip
without authed binaries so it's a no-op elsewhere).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(pglite): embed WASM + extension assets so the compiled binary can serve

A `bun build --compile` gbrain binary could not `serve` a PGLite brain: the
compile bundles JS but not PGLite's runtime payload (pglite.wasm, initdb.wasm,
pglite.data, vector/pg_trgm tarballs), so `serve` on PGLite died with a
bunfs/ENOENT. Now the assets ride inside the binary.

- src/core/pglite-embedded-assets.ts: embeds the five assets via
  `import … with { type: 'file' }` (the ENG-6 idiom) and exposes
  getEmbeddedPgliteOptions() → { pgliteWasmModule, initdbWasmModule, fsBundle,
  extensions:{vector,pg_trgm} }. WASM/fsBundle are consumed as bytes; the two
  extension tarballs are materialized to a content-addressed temp file (atomic,
  size-verified reuse) because PGLite reads them via fs.createReadStream, which
  cannot read a /$bunfs path. Unconditional (works in bun-run and compiled),
  so no fragile mode branch.
- src/core/pglite-engine.ts: static-import getEmbeddedPgliteOptions (engine
  path stays static per the engine-dynamic-import invariant); spread into both
  PGlite.create sites (initial + WAL-repair retry). The bunfs classifier stays
  as a backstop but no longer fires for a correct binary.
- scripts/check-pglite-embedded.sh (+ smoketest): compiles a focused binary and
  asserts it boots PGLite, CREATE EXTENSION vector/pg_trgm, and round-trips a
  page — wired into `bun run verify` (now 37 checks), check:all, and
  check:pglite-embedded. Fail-soft only when compile is unavailable.
- agent-harness.ts probeCompiledPglite now passes → the real-agent e2e uses the
  fast compiled MCP server. TODOS: the P2 "can't serve PGLite" item is closed.

Verified: fresh compiled binary ran `search`/`query` against a PGLite brain and
returned the seeded row (no bunfs/ENOENT); verify 37/37; pglite-engine 120/0
source-mode; typecheck clean; engine-dynamic-import + parity guards pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-11 16:42:11 -07:00
Sina MatianClaude Fable 5Garry Tan
 		fc310db3ea v0.44.1.0 feat(ai): recipes resolve any model — remove the native-recipe runtime allowlist (#4014) 
* feat(ai): recipes resolve any model — remove the native-recipe runtime allowlist

Frontier models ship weekly; a curated models: array can never stay current.
assertTouchpoint now checks only the provider's touchpoint capability (anthropic
has no embeddings, voyage has no chat) and never gates on the model id. Any id
the user names goes to the provider, which is the real authority on what
exists — a nonexistent model surfaces as the provider's own model_not_found at
call time, and gbrain models doctor live-probes the configured models for a
pre-flight check.

With the gate gone, the entire extendedModels bypass machinery is dead and
deleted: the _extendedModels registry, registerExtendedModel,
registerConfigSelectedChatModel (+ its one caller in the contextual-reindex
handler), both registration loops, and the tier-resolution loop that existed
only to feed them. This also structurally closes the per-task-key gap where
models.think / models.dream.* / facts.extraction_model selections were rejected
while identical models.default selections worked.

Recipe models: arrays remain informational — models[0] default selection for
--model <provider> shorthand, guard-test fixtures pinning the repo's own
hardcoded defaults, and gbrain providers list display. gateway.rerank() keeps
its own model-list check deliberately: each listed reranker id maps to a known
request/response wire shape.

unknown_model still fires for providers lacking the touchpoint, so every probe
reason stays reachable; tests that pinned the allowlist rejection now pin the
pass-through contract (or the missing-touchpoint trigger).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* v0.44.1.0 feat(ai): recipes resolve any model — remove the native-recipe runtime allowlist

Frontier models ship weekly; a curated models: array can never stay current.
assertTouchpoint now checks only the provider's touchpoint capability and never
gates on the model id — any id the user names goes to the provider, and a
nonexistent one surfaces as the provider's own model_not_found at call time
(gbrain models doctor stays the token-free pre-flight).

The extendedModels bypass machinery is deleted end to end, which also closes
the per-task-key gap: models.think / models.dream.* / facts.extraction_model
selections now behave exactly like models.default. think's graceful sentinel
surfaces the thrown AIConfigError's own message + fix instead of generic key
advice, so a provider 4xx is never key-blamed. gateway.rerank() keeps its own
model-list check (each listed id maps to a known wire shape).

Ship chores: VERSION/package.json → 0.44.1.0, CHANGELOG entry, gitleaks action
pin refreshed to current v2.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-11 12:41:24 -07:00
Garry Tan 75fae742d5 v0.44.0.0 feat(eval): BrainBench — cross-harness memory conformance suite (Cathedral 2) (#2158) 
Public, reproducible, cross-harness benchmark for the four ways agent memory fails (know-to-ask, push precision/recall, write-back fidelity, cross-session continuity). Hermetic PGLite, sealed gold, deterministic corpus, CI gate vs master's committed baseline, published JSON-schema foreign-runner contract. See docs/eval/BRAINBENCH.md.
 2026-08-10 09:44:46 -07:00
Garry TanandClaude Fable 5 c5952b8714 v0.43.0.0 feat(mcp,search): MEMORY_VERBS v1 five-verb protocol + concept-question query steering (cathedral 1, #2416) (#2172) 
* feat(verbs): MEMORY_VERBS v1 core — five frozen verbs over existing machinery

recall extended in place (G1B additive superset: protocol_version + string
fact_id/provenance on every response; query arm via cheap-hybrid with
keyword-only degradation when no embedding provider; op-layer budget packing
facts-first via the new generic packToBudget). remember/entity/synthesize/
forget as first-class Operations in src/core/verbs.ts (runtime leaf — TDZ-safe
both import directions) with MEMORY VERB descriptions, ToolAnnotations,
populated suggestions on every error (verbError + protocol ErrorCodes +
additive detail/protocol_version on OperationError.toJSON).

writeSingleFact (facts/write-single.ts): zero-LLM single-fact seam reusing the
pipeline's dedup; X1 deterministic supersession (same entity+kind, similar,
text differs -> supersedeId / fence expire+link); validUntil threaded through
FenceInputFact (was silently dropped). buildEntityCard (verbs/entity-card.ts):
reflex resolution arms + parallel depth-1 reads, per-arm degradation on
pre-page_aliases brains, near-miss suggestions with create_safety. runThink
gains additive usage aggregation; synthesize converts the no-LLM graceful
result to the unavailable protocol error and prices the cost block via
canonical model-pricing. RESPONSE_SCHEMAS registry + ERROR_SCHEMA for the
conformance kit.

Cathedral 1 stage 1/4 (#cathedral-1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(mcp): MEMORY_VERBS v1 surface mode + fail-closed dispatch + usage sidecar

gbrain serve --surface verbs|full (config mcp_surface registered in the merge
allowlist; default full — existing installs unchanged). Enforcement is
two-layer and fail-closed: filtered ListTools AND dispatchToolCall's new
allowedOps set, applied on stdio + BOTH HTTP paths (serve-http.ts OAuth and
the second http-transport.ts). ToolAnnotations passthrough in buildToolDefs +
the HTTP inline mapper, emitted only when an op defines them (byte-stability
for existing tools). Verb validation/internal errors speak the protocol
envelope (suggestion + protocol_version) at the dispatch layer.

E4 observability: per-verb JSONL sidecar (usage-log.ts — local only, never
uploaded; O_APPEND line-atomic, lock-free 10MB rotation) written from dispatch
so param-validation failures count; brain_id via gbrainPath; doctor gains the
memory_verbs_usage check. CLI: human-readable rendering for remember/entity/
synthesize with trailing --json escape; gbrain recall gains --query +
--budget-tokens routed through the recall OP (same arm MCP exercises) with the
budget footer; the forget verb carries no cliHint (CLI_ONLY forget dispatches
first and would shadow it).

Cathedral 1 stage 2/4 (#cathedral-1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(protocol): gbrain protocol — schemas, conformance kit, stats + TTHW

gbrain protocol [--json] emits the live verb input schemas + the hand-authored
RESPONSE_SCHEMAS + error contract + versioning policy. gbrain protocol
conformance certifies ANY MCP endpoint (default: self-spawned stdio; --target
http://...[--token] or a stdio command) against the embedded fixture set —
shape/enum/behavior/round-trip only, deterministic; entity-page cases seed via
put_page when available and skip honestly on verbs-only targets; synthesize is
cost-gated behind --synthesize (asserts the clean unavailable error on key-less
CI). Fixtures double as BrainBench seeds (test/fixtures/memory-verbs/cases.json
mirror). Self-certification passes 23/23 live.

gbrain protocol stats [--days N]: per-verb calls/error-rate/latency, budget
drops, entity hit rate, and measured TTHW (init stamps protocol_installed_at;
delta to the sidecar's first verb call) — local JSONL only, never uploaded.
init epilogues print the 3-command quickstart (mcp add --surface verbs +
key-free resurrection demo) with the visibility consent line. entity-card fix:
raw namespaced-slug input now matches exactly (slugify flattens slashes).

Cathedral 1 stage 3/4 (#cathedral-1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test+docs(protocol): conformance suite, p99 gate, MEMORY_VERBS_v1 doc, funnel

test/memory-verbs-conformance.test.ts: 26 tests through dispatchToolCall —
G1B superset regression (legacy fields byte-equal, additions allowed), budget
math incl. budget<first-item, keyword degradation (never an error), remember
provenance/ttl-trap/null-echo, remote round-trip via the world default +
private facts hidden, entity card schema-validated with the chat gateway
rigged to throw (zero-LLM guard) + all arms + fence test, synthesize
[EXPENSIVE + annotations + clean unavailable, forget idempotency/not_found,
writeSingleFact X1 supersession via the embed-transport seam + degraded
dedup, the F3 negative self-test (the runner FAILS lying servers: dropped
fields, bad enums, re-typed ids), fixture-mirror drift guard.
test/mcp-surface.test.ts: verbs=exactly 5, full=identity, dispatch-layer
fail-closed, strict flag parsing. test/entity-card-perf.slow.test.ts: 20K
pages/100K links/30K aliases/40K facts — measured p99 22.9ms vs the 100ms
contract + a recalibrated 50x ratio guard; wired into the CI perf job along
with stdio self-certification (23/23 CONFORMANT live). tool-defs byte-
stability test now pins per-op: ops WITHOUT annotations keep the exact
pre-v1 shape.

docs/protocol/MEMORY_VERBS_v1.md: the frozen contract — envelopes, error
codes, additive-forever policy, ttl forms (P30D rejected), status-not-
status_text, surface modes + why default full, per-harness installs, consent
line, conformance pass criteria + copyable --target examples, local-only
stats privacy, 200K manual bench recipe. README quickstart + docs/mcp pages
moved to --surface verbs with the claude-not-found recovery; llms-config
registered + bundles rebuilt; KEY_FILES entries for the 7 new modules.

Cathedral 1 stage 4/4 (#cathedral-1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(verbs): verify-gate compliance — test seam, allow-comment placement, import hygiene

usage-log gains __setUsageLogPathForTests so the verb tests redirect the
sidecar without mutating process.env.GBRAIN_HOME (test-isolation lint);
the gbrain-allow-direct-insert rationale moves onto the insertFact call line
(system-of-record guard greps per-line); protocol.ts drops its unused
`operations` import (operations-filter-bypass guard — operationsByName only).
The synthesize unit test now drives the chat-transport seam (deterministic
success envelope + priced cost block, zero real API spend); the keyless
`unavailable` conversion stays covered end-to-end by the CI conformance step,
which runs credential-free.

Cathedral 1 verify pass (#cathedral-1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(verbs): close 3 ship-review P1s — forget cross-source, entity backlink leak, conformance exit code

Pre-landing review (codex, on the committed diff) caught three P1s the
plan-stage reviews flagged only in the abstract:

P1.1 trust boundary — forget expired by GLOBAL fact id with no source/visibility
scope: a remote caller in source A could expire facts in source B (incl.
private) by guessing ids. forgetFactInFence gains sourceId + worldOnly opts
(scope-check before any state inspection; out-of-scope → not_found, no
existence leak); the forget verb passes ctx.sourceId + worldOnly=remote.

P1.2 source isolation — entity card incoming edges + backlink_count leaked
across sources: getBacklinks(slug,{sourceId}) scopes only the TARGET page and
getBacklinkCounts has no source param, so a foreign-source page's slug surfaced
in a same-named entity's card. Replaced both with a both-sides-scoped query
(f.source_id = t.source_id = entity source, mentions excluded) inside the card
module — no engine-interface change.

P1.3 CLI exit code — protocol conformance/unknown-subcommand set
process.exitCode directly, which the PGLite/WASM force-exit seam ignores
(it reads the gbrain-owned setCliExitVerdict). A NOT CONFORMANT target printed
the banner but exited 0 — CI would green-light a non-conformant server. Now
routes through setCliExitVerdict; proven: unknown subcommand exits 1.

Regression tests: cross-source forget → not_found + foreign fact stays active;
remote can't forget a private fact; entity card omits foreign-source backlinks
from both edges and the count.

Cathedral 1 ship-review fixes (#cathedral-1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* v0.43.0.0 feat(mcp): MEMORY_VERBS v1 — frozen five-verb memory protocol + conformance kit (cathedral 1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: surface --surface verbs in local-stdio quickstart paths for v0.43.0.0

The MEMORY_VERBS v1 quickstart flag (gbrain serve --surface verbs) was added to
the README quickstart, CLAUDE_CODE.md, and CODEX.md during the build but three
other docs that show the local-stdio wire-up still presented bare `gbrain serve`:
the connect-coding-agent Path B walkthrough (the doc the README links to),
INSTALL.md, and DEPLOY.md's Local stdio path. Brought them in line so the
five-verb surface is consistent across every local-quickstart entry point.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(doctor): assign memory_verbs_usage check to the SKILL category set

The new memory_verbs_usage doctor check (Cathedral 1, E4) was added without a
category, tripping the doctor-categories drift guard (every check name must
belong to exactly one category set). It's an agent-capability adoption signal,
a sibling of retrieval_reflex_health — assigned to SKILL_CHECK_NAMES.

Cathedral 1 ship-review fix (#cathedral-1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(verbs): propagate MEMORY_VERBS v1 to existing harnesses on upgrade

The verbs shipped reachable + self-describing, but an EXISTING install that
upgrades to 0.43 and keeps its skills had no active push toward them: no
post-upgrade notice (default surface stays 'full'), and no skill/guidance
routing — agents kept calling search/query/get_page/extract_facts.

Two additive propagation fixes:

- Post-upgrade feature pitch: src/commands/migrations/v0_43_0.ts (pitch-only,
  no-op idempotent orchestrator — the verbs ride existing tables, no schema/
  data migration). Registered in the migrations index so
  and the self-upgrade NOTIFY channel announce the five verbs +  +  on any upgrade past 0.43.
  Verified live: the pitch fires for an install upgrading from 0.42.43.0.
- Skill routing: skills/query (memory reads) and skills/brain-ops (read/write
  cycle) now teach the verbs — recall/entity/synthesize vs search/get_page/
  query, remember vs extract_facts, forget — with a fall-back-to-classic-ops
  note for older brains / full surface. The in-MCP tool descriptions already
  steered tool-list readers; this adds the narrative layer the audit found
  missing across all 29 skills.

apply-migrations future-list fixtures updated for the new 0.43.0.0 registry
entry. No DB migration; no contract change; verbs unaffected.

Cathedral 1 propagation (#cathedral-1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(verbs): green the 3 failing CI shards — 3-seg migration version, hermetic synthesize test, fresh llms

Three real CI-shard failures from the propagation commit + master merges:

- migrations-registry (shard 10): v0_43_0 used a 4-segment version 0.43.0.0,
  but the registry enforces 3-segment semver (/^\d+\.\d+\.\d+$/). Changed
  the migration version to 0.43.0 (the RELEASE stays 0.43.0.0; migrations have
  always used 3-seg, e.g. 0.32.2). apply-migrations future-list fixtures
  updated to match.
- synthesize conformance test (shard 5): asserted a successful answer via the
  chat seam, but runThink builds its client through a real-key check (not the
  seam), so CI (credential-free) took the NO_ANTHROPIC_API_KEY path → the
  verb's unavailable conversion → isError. Now wraps the call in withEnv with a
  fake key AND the chat seam, so it's deterministic regardless of ambient
  credentials.
- build-llms (shard 8): the v0.42.44.0 master merge changed an inlined doc;
  regenerated llms.txt/llms-full.txt.

Cathedral 1 CI fixes (#cathedral-1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(search): steer concept/landscape questions to query — descriptions, docs, lookup chain, CLI nudge (#2416)

The search/query routing guidance shipped a stale and backwards model:
SEARCH_DESCRIPTION still led with "Keyword search using full-text search"
(search has been cheap-hybrid — vector + keyword + RRF, expansion off —
since D4/D15), and the mandatory lookup chain in brain-first.md ordered
search-first with query only "if search is thin". For concept/landscape
questions ("all the X that do Y"), keyword+vector recall always returns
something, so the fallback never fires and expansion-recoverable
synonym-phrased matches drop silently.

- operations-descriptions.ts: fix the stale SEARCH lead (cheap-hybrid, no
  LLM expansion); both constants now route concept/synonym/landscape
  questions to query, cap the claim honestly (both are top-K — exhaustive
  enumeration belongs to list_pages), and keep every pinned redirect.
- brain-first.md: lookup chain is intent-driven (exact token → search;
  concept/landscape → query FIRST), cost claims corrected, and the
  anti-pattern is explicit: a nonzero search count is not a completeness
  signal. Same sweep in brain-ops/SKILL.md and query/SKILL.md, which
  duplicated the backwards order.
- search-modes.md: decision tree + quick-ref updated to the cheap-hybrid
  reality, new Tricky Spot #6 (count ≠ completeness), example names moved
  to placeholder people per the privacy convention. RETRIEVAL.md gets the
  matching one-liner.
- query-intent.ts: looksConceptShaped + conceptNudge — deterministic
  concept-shape classifier in the existing classifier home, tuned to favor
  false-negatives; cues owned by other routers (find_experts, salience
  ops) deliberately excluded. cli.ts prints the hint-only stderr nudge on
  BOTH result paths (local engine + thin-client routed), --quiet-gated;
  search stays the cheap hot path (never auto-reroutes).
- salience-llm-routing e2e: new concept-phrasings block asserts the
  descriptions actually route concept questions to query (verified live:
  20/20 — all 12 personal-query regressions still route to salience ops,
  all 8 concept phrasings route to query).
- KEY_FILES.md: query-intent entry updated to current state (the old
  entry pointed at a renamed file); llms bundle regenerated (no diff —
  entry is link-only).
- TODOS.md: MCP-envelope hint follow-up filed (remote agents are the
  higher-value half; needs a return-contract change kept out of this
  commit).

Verification: typecheck clean; 70 targeted unit tests + 20 live routing
evals green; all 24 check:all guards green; failing full-suite shards
triaged — think-pipeline.serial.test.ts:434 fails identically on the
clean tree (pre-existing on this branch, filed separately), the rest are
PGLite parallel-load contention that pass in isolation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): pin think's stub-path usage at null per the E2 accounting contract

The MEMORY_VERBS [E2] usage-accounting change deliberately moved "no LLM
ran" from `undefined` to `null` (the frozen cost-block spelling), but the
older #1698 stub-path test still pinned `undefined` and failed on every
run of this branch. The protected semantic is unchanged: no fake usage or
cost on the stub path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): cover the #2416 query-steering wave under v0.43.0.0

The 0.43.0.0 entry described only MEMORY_VERBS; the #2416 retrieval-routing
wave rides the same release. Adds its user-facing summary and refreshes the
release date to the actual ship date.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(protocol): reword so the #3502 docs-lint doesn't read a phantom command

Master's docs-lint (landed after this branch forked) scans for
`gbrain <verb>` references; the MEMORY_VERBS diagram's "gbrain reference
implementation" phrasing parsed as a nonexistent `gbrain reference`
command. Post-merge-only failure; both sides were green alone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: update project documentation for v0.43.0.0

Post-ship /document-release drift pass over the MEMORY_VERBS v1 +
#2416 query-steering waves:

- docs/tutorials/connect-coding-agent.md: the pasteable brain-first
  protocol still taught the pre-#2416 "Search first" rule and named
  only full-surface tools; step 1 now routes by question shape
  (exact tokens -> search, concept/landscape -> query first) and both
  steps map the five-verb surface names (recall/synthesize/remember).
  find_experts and query callouts marked with their surface.
- README.md + tutorial: "90-tool wall" count refreshed to 110 (the
  actual operation catalog).
- CLAUDE.md: contract-first op count ~90 -> ~110 with the v0.43.0.0
  MEMORY_VERBS clause; reference-map row for the memory-verbs
  protocol / --surface / conformance docs.
- TODOS.md: blank line before the v0.42.74.0 section header.
- llms-full.txt regenerated (build:llms; freshness test green).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: apply cross-model doc-review fixes for v0.43.0.0

Verified findings from the /document-release Codex pass:

- MEMORY_VERBS_v1.md: disclose that conformance WRITES (marker-suffixed
  seed page + remember/forget facts, write-scope creds, artifacts not
  auto-deleted); quickstart header says 4 commands because it has 4.
- CHANGELOG v0.43.0.0: cost claim scoped per verb (budget/latency/cost
  block); remember "updates in place" -> supersedes-with-history (what
  write-single.ts actually does).
- DEPLOY.md: legacy bearer fallback works on BOTH engines now (both
  schemas carry access_tokens per http-transport.ts) — was Postgres-only.
- CLAUDE_CODE.md: list_skills/core-tools verify note marked full-surface
  (on --surface verbs those tools aren't on the surface).
- skills/query/SKILL.md: --surface full INCLUDES the verbs; fallback is
  for pre-0.43 servers only.
- skills/brain-ops/SKILL.md: search/query tool labels updated to the
  #2416 reality (cheap hybrid vs hybrid + LLM expansion).
- search-modes.md: stale "keyword search returns chunks" label.
- RETRIEVAL.md: note the query op defaults expand:true per call.
- Counts: 43 skills -> 52 (README x2, INSTALL), 30 skills -> 52
  (CLAUDE.md), "30+ tools over MCP" -> 110 (README).
- TODOS.md: filed 2 code-vs-spec follow-ups (recall filter composition +
  limit cap; synthesize unavailable-mapping breadth) instead of silently
  editing the frozen contract.
- llms-full.txt + skills.lock.json regenerated; freshness guards green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cli): regenerate the flag registry for the cathedral-1 command set

Master's #2185 strict flag validation ships a committed, generated
CLI_FLAG_REGISTRY with drift + freshness guards keyed off CLI_ONLY. The
merge added this branch's `protocol` command to CLI_ONLY, but the
committed registry was generated on master without it, failing CI shard
7's drift guard. Regenerated via `bun run build:flag-registry` on the
merged tree (95 commands; picks up `protocol` and the `--surface`
help-text mentions from the verbs work).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-09 17:25:00 -07:00
Sina MatianClaude Fable 5Garry Tan
 		aecb33e795 v0.42.73.1 revert(ci): remove the PR gate and withdraw the v0.42.72.1 contribution requirements (#3805) 
* v0.42.73.1 revert(ci): remove the PR gate — it cannot function on this repository

The gate needs an ANTHROPIC_API_KEY Actions secret for its verdict and
read-write workflow permissions to post a comment or set a label. This
repository grants neither, and both are admin-only, so it can only ever skip.

It also caused a real incident before that was understood: a read-only token
turned every API call into a 403, the code treated that as a crash, and the
check went red on an outside contributor's PR four times with no comment
explaining why. v0.42.73.0 fixed the crash, but a check that runs on every PR
and can never reach a verdict does not earn a place in the repo.

Removes the workflow, the script, its type surface, and its test file. The code
is preserved in git history at v0.42.73.0. If it is ever restored, the
mechanical half (intent/screenshot policy, title rule, red flags) should render
to the Actions job summary rather than a comment — that needs no token
permission and no API key.

CONTRIBUTING.md's intent-paragraph and screenshot requirement is unchanged and
stands as written; it is enforced by maintainers reading PRs, as before.

typecheck clean, verify 34/34, llms bundles regenerated.

* v0.42.73.1 revert(docs): withdraw the human-intent-paragraph + screenshot contribution requirement

This reverts commit 6d1232d5a6 (v0.42.72.1).

CONTRIBUTING.md, both issue templates, and the pull-request template return
to their pre-2026-08-02 state. VERSION/package.json/CHANGELOG keep moving
forward (0.42.73.1); the v0.42.72.1 CHANGELOG entry stays as historical
record, and the 0.42.73.1 entry now describes the withdrawal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-05 04:20:30 +07:00
Garry TanandClaude Fable 5 88731d8cf3 fix(ci): gate — sanitize mechanical flag details, anchor path regexes, exempt maintainer/bot/draft from the policy check (blind review round 2) 
Blind review round 2 rejected the branch. Five findings, all fixed.

BLOCKING 1 — the verdict was forgeable through a PR filename.
renderComment wrote mechanical red-flag details raw, and two of them
interpolate filenames (adds_recipe, deletes_tests). git allows a newline
inside a filename and JS `[^/]` matches one, so RECIPE_RE's anchors were
decorative: a PR adding `src/core/ai/recipes/x\n## PR Gate — ...\ncc
@octocat\n<!-- ...state... -->\nz.ts` put that text verbatim into the bot's
comment — forged heading, live third-party mention, and on the NEUTRAL
render (which writes no state block of its own) parseState() returned the
ATTACKER's block, so the next run hit the spend guard and silently skipped
the verdict with no label and exit 0. Three layers:
  (a) flag details go through sanitizeList, exactly like the model's
      strings. Audited every other interpolation into the comment; the rest
      are literals in the file, Number()-coerced, or already sanitized.
  (b) every path regex spells its segment class [^/\n], not [^/] —
      RECIPE_RE, SOURCE_EXT_RE and the test-path check.
  (c) parseState reads the state block only off line 2 of a marker-leading
      comment (where renderComment writes it) and STATE_RE is whole-line
      anchored. A block anywhere else is somebody else's text.

BLOCKING 2 — no exemption, so every release PR was close-lane.
Measured: 40 of the last 40 merged PRs would be close-lane on
missing_screenshot, including every /ship release PR. A check that is red
on every release gets switched off within a week, and then it filters
nothing. The #3745 policy exists to filter INCOMING OUTSIDE CONTRIBUTIONS;
release automation cannot take a screenshot of itself. It is now waived for
OWNER/MEMBER/COLLABORATOR, bot authors and drafts — the usefulness verdict,
the title rule and every mechanical red flag still run, and the sticky
comment says the check was skipped. author_association / draft / user.type
are read from the pr.json the workflow already fetches: no new API call, one
source of truth. `draft` is the one author-settable input, so
ready_for_review joins the trigger list and the exemption is folded into the
spend-guard hash — the draft-era verdict cannot be reused after the flip.
Stated as a deliberate decision in both the workflow header and the script.

3 — DOWNGRADE_FLAG_IDS omitted deletes_tests, adds_symlink and
adds_node_modules, so a PR deleting test/e2e/engine-parity.test.ts kept
merge-lane and a green check on the strength of its prose. All three added.
The old test used deletes_tests as its example of a NON-downgrading flag;
rewritten to pin the stronger invariant instead — every id detectRedFlags
can emit is a downgrade trigger (derived from the detector, so a new flag
fails until it is classified on purpose), and the set is still an allowlist
(an unrecognized id changes nothing).

4 — FENCE_RE backtracks superlinearly on a hostile body: 65KB of backticks
(GitHub's max body length) measured 8.2s across the two policy scans on a
pull_request_target runner. stripCodeFences now caps the scan at 16KB —
same input, 0.40s. Tradeoff documented at the constant: the intent
paragraph and the screenshot both sit near the top in practice (the PR
template puts them in the first two sections, and the model payload already
caps the same body at 6KB), so a real contributor is not judged on a
truncated tail.

Verified: test/pr-gate-workflow.test.ts 126 pass / 0 fail (was 95),
typecheck clean, actionlint clean, check-privacy /
check-no-tracked-symlinks / check-progress-to-stdout /
check-bun-test-timeout / check-key-files-current-state all exit 0. Each new
pin was mutation-tested against the pre-fix behavior: all six mutants fail
the suite.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 10:46:09 +08:00
Garry TanandClaude Fable 5 4a00c31b12 fix(ci): gate policy check survives an API outage; merge master; drop NUL separators 
Three fixes on the strict PR usefulness gate (#3698), plus the merge that
brings in the policy it enforces.

1. The mechanical policy check now outlives the model. The
   ANTHROPIC_API_KEY guard used to sit above detectPolicyMisses, so a PR
   with no intent paragraph and no screenshot got a green NEUTRAL skip
   whenever the key was absent or Anthropic was down — "wait for a 500"
   was a documented way past the one hard requirement. The #3745 branch
   now sits above the key guard and the spend guard: a policy miss is
   close-lane + the friendly fix-it comment + exit 1 with no API
   dependency at all. A compliant PR that hits a missing key or a dead
   API keeps the round-1 NEUTRAL behavior unchanged (loud comment,
   ::warning::, exit 0, stale gate:* labels cleared) — and the NEUTRAL
   comment now says plainly that the *usefulness verdict* did not run,
   while still reporting the title check and mechanical red flags it was
   able to compute without a model.

2. Merged origin/master, which carries #3745's CONTRIBUTING.md section
   and .github/pull_request_template.md. No conflicts: this branch never
   touched VERSION / package.json / CHANGELOG.md, so master's 0.42.72.1
   carried through untouched — the feature branch adds no version bump.
   The test's inlined pull_request_template fallback (only needed while
   the branch predated the merge) is gone; it now reads the real file, so
   growing the template's own prose past the 40-word bar fails here
   instead of silently letting an untouched template through. The
   CONTRIBUTING_URL deep link is pinned against a GitHub-style slug of
   every heading in the merged CONTRIBUTING.md, with the slugger itself
   pinned so it cannot "pass" against an anchor GitHub never generates.

3. hashInputs joined its three fields with literal NUL bytes, which made
   grep treat the whole of scripts/pr-gate.mjs as binary — any future
   grep-based CI guard over that file would have matched nothing and
   passed silently. Replaced with JSON.stringify of the tuple: still
   unforgeable (each field is quoted and escaped), still stable by
   construction, and printable. `grep -c hashInputs scripts/pr-gate.mjs`
   now returns 2 instead of nothing. Existing sticky-comment state hashes
   are invalidated once, costing one re-verdict per open PR.

Tests: 95 pass / 0 fail in test/pr-gate-workflow.test.ts. The no-API-key
policy-miss case was verified to fail against the pre-fix ordering.
Verified live against the Anthropic API: HTTP 200, strict JSON, all seven
required keys, merge-lane on a compliant fixture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 10:46:09 +08:00
Garry TanandClaude Fable 5 2f65ed8da6 fix(ci): harden the PR gate — comment-ownership, output sanitization, deterministic lane downgrades (blind review round 1) 
Two independent blind reviews of #3698 (one APPROVE, one REJECT). Every
confirmed finding from the reject side, plus the cheap hardening both
reviews flagged.

BLOCKING

1. Sticky-comment hijack. upsertStickyComment adopted ANY comment containing
   the marker, so a contributor could pre-post `<!-- gbrain-pr-gate -->`, have
   the gate PATCH it, then edit it into a fake green verdict. isOwnComment()
   now requires user.type === 'Bot' AND login === 'github-actions[bot]' AND
   the body to START WITH the marker; anything else gets a fresh comment.

2. LLM output injected as raw Markdown. Model-produced reasons[] and
   reviewer_checklist[] reached the comment unescaped — PR-body-driven
   injection could forge headings, a second marker, and live @mentions.
   sanitizeModelText()/sanitizeList() are now the single choke point in
   renderComment(): HTML comments stripped, mentions zero-width-broken,
   leading block markers removed, newlines collapsed, 300 chars per string,
   8 entries per list, both caps self-marking.

3. Lane was purely model-decided. A well-written feature pitch could talk
   itself into merge-lane. The model now RECOMMENDS; applyMechanicalDowngrades
   forces merge-lane -> needs-maintainer on any of: workflow edits, a new
   package.json dependency, a new src/core/ai/recipes/ provider file, new
   KNOWN_CONFIG_KEYS entries, >40 changed files, >400 net source lines outside
   test/, or a src/ change with no test file touched (#3665). The sticky
   comment reports them under "Mechanical downgrades applied".

4. The test file did not pin what it claimed. Added: no gh pr checkout /
   git fetch / refs/pull / pull/*/head in any spelling; exact permissions
   key->value map plus a single-permissions-block assertion so no job-level
   grant re-widens contents; the ${{ }}-in-run scanner now covers folded
   (`run: >`) and chomped blocks, with a guard-the-guard test; and mocked
   end-to-end runGate() runs for close-lane exit 1, marker hijack, sanitizer,
   truncation, refusal routing, NEUTRAL label clearing, label swap, and the
   spend guard.

5. Version-first title regex rejected the documented suffix form.
   `v0.31.1.1-fixwave fix: ...` now passes. VERSION_AT_END_RE no longer
   false-positives on `chore: bump zod (3.25.76)`: it fires only on a
   v-prefixed or 4-segment trailing version, i.e. this project's own shape.

6. Refusal fail-open. stop_reason=refusal exhausted retries into a green
   NEUTRAL — a deterministic way to dodge the red X. Refusal and
   schema-invalid output now route to needs-maintainer with an explicit
   note; only transport failure stays NEUTRAL. Refusal also short-circuits
   the retry loop, since retrying a deterministic refusal only burns spend.

ALSO

7. persist-credentials: false on the checkout step.
8. Dropped pull-requests:write. Everything the script calls is the issues
   API (comments, label create, label add/remove), so issues:write is the
   only grant that is actually needed.
9. Spend guard for the edited/synchronize amplification: the LLM call is
   skipped when sha256(title+body+head_sha) matches the hash recorded in the
   previous sticky comment's state block, and the stored lane's exit code is
   reused.
10. NEUTRAL runs now clear every gate:* label instead of leaving a stale
    verdict behind.

Verified: bun test test/pr-gate-workflow.test.ts 63 pass / 0 fail,
bun run typecheck clean, actionlint clean, check-privacy /
check-no-tracked-symlinks / check-progress-to-stdout /
check-bun-test-timeout clean, plus a live Anthropic smoke of the exact
request shape (HTTP 200, valid strict JSON, injection attempt rejected).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 10:46:09 +08:00
Garry TanandClaude Fable 5 ca1eed3e95 feat(ci): strict PR usefulness gate — merge-lane/close-lane verdict before any review effort (#3698) 
Runs on pull_request_target against master. PR code is never checked out
or executed: metadata + a 120KB-capped diff come from the GitHub API only;
the base-repo checkout provides the rubric script. All interpolations are
env-bound; actions SHA-pinned; permissions limited to contents:read +
pull-requests:write + issues:write; per-PR concurrency with cancel.

scripts/pr-gate.mjs (no deps, fetch only) classifies via the strict
usefulness rubric (claude-sonnet-5, strict JSON schema, thinking disabled;
sampling params omitted — rejected on this model), posts one sticky
comment (<!-- gbrain-pr-gate -->), applies exactly one gate:* label, and
exits 1 only on close-lane. Missing ANTHROPIC_API_KEY or API failure after
2 retries NEUTRAL-skips loudly (comment + warning, exit 0). Mechanical
(no-LLM) checks: version-first title rule and diff red flags (>40 files,
node_modules, symlinks, workflow edits force needs-maintainer, new
package.json deps, deleted tests).

test/pr-gate-workflow.test.ts pins the security invariants and unit-tests
the exported title rule + red-flag detector (import side-effect guarded).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 10:46:09 +08:00
Sina MatianGarry TanClaude Opus 5
 		6d1232d5a6 v0.42.72.1 docs(contributing): require a human-written intent paragraph + live screenshot on every issue and PR (#3745) 
Effective 2026-08-02. Every issue and PR must carry a paragraph the author
wrote themselves explaining why they are opening it, and a screenshot showing
gbrain actually in use in that situation. AI-generated or AI-polished intent
text is not accepted; AI assistance for the code is still fine. Missing either
one means closed without review, reopenable once added.

Stated in CONTRIBUTING.md and pre-filled in both issue templates plus a new
pull request template so the fields are in front of the author.

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-02 07:47:23 +08:00
Garry Tan f07e9dff11 fix(ci): release build gates on the artifact, not a serial re-run of the whole suite 
The first Release run (30698650484) failed before building anything: the
build job re-ran the entire unit suite serially on the release runner —
a different environment from the sharded Test workflow that had already
gated the exact SHA at merge — and died on ambient-env tests. The review
of #3573 predicted this ('a flaky test now blocks the release'). The
build job now compiles and smoke-tests the binary (--version must match
the VERSION file), which validates the actual artifact — something the
test suite never did. The Test workflow remains the code gate.
 2026-08-01 20:43:58 +08:00
Garry Tan 0244104b8d fix(ci): env-bind the release-notes version interpolation 
Review hardening from the #3573 gatekeeper pass: a ${{ }} inside run: is
shell injection by construction; bind through env instead. Not
attacker-reachable today (VERSION comes from master), correct anyway.
 2026-08-01 19:55:28 +08:00
Garry TanandTime Attakc 48618bd4bf feat(ci): publish GitHub releases so binary self-update can work (#3521) (#3573) 
Co-Authored-By: Time Attakc <89218912+time-attack@users.noreply.github.com>
 2026-08-01 19:55:28 +08:00
Garry TanandDiego a228776b8f docs(security): tighten responsible disclosure language (#3619) 
Co-Authored-By: Diego <diegodearagao@gmail.com>
 2026-08-01 10:06:17 +08:00
Time AttakcGarry TanClaude Opus 5
 		913d2d7f79 fix(test): give slow setup hooks a real timeout budget (#3566) 
bun ignores bunfig.toml's timeout key, and beforeAll/beforeEach hooks do
NOT inherit a test's third-arg timeout — a bare `bun test` gives every
hook the 5000ms default even when all tests in the file declare 30s+.
Measured on bun 1.3.14: a 6s hook dies at ~5001ms with the signature
`(unnamed) [5001ms] ... hook timed out` (the #3545 jsonb-parity CI
failure); both `beforeAll(fn, ms)` and the CLI `--timeout` flag are
enforced hook budgets (kills observed at exactly the configured ms).

Fixes:
- e2e.yml (jsonb-parity, tier1, tier2) and release.yml ran bare
  `bun test`; they now pass --timeout=60000 like every scripts/ runner.
- test/e2e/jsonb-roundtrip.test.ts (the #2339 double-encode guard, which
  only real Postgres can surface) additionally carries per-hook 60s
  budgets so a bare local run can't flake either — same pattern as its
  sibling op-checkpoint-jsonb-parity.test.ts.
- scripts/check-bun-test-timeout.sh: CI guard (run from test.yml's
  verify job) failing any future bare `bun test` in workflows/scripts.
- scripts/run-e2e.sh: correct the comment claiming --timeout is
  per-test-only (it covers hooks; the outer gtimeout exists for
  sync-blocking WASM hangs where no timer can fire).

Proof: with Postgres paused for 6s during setupDB's connect, the
unfixed file fails at 5001.81ms with the exact CI signature; the fixed
file passes the identical condition (5 pass, 6.57s). 396 slow
before-hooks across 362 test files lack per-hook budgets; all of them
run through --timeout-passing invocations after this change, enforced
by the new guard.

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-07-28 20:08:36 -07:00
Time AttakcGarry TanClaude Fable 5
 		800108e014 v0.42.65.0 chore(release): 93 verified fixes since v0.42.64.0 — changelog + version bump (#3346) 
* chore(ci): refresh GitHub Actions SHA pins (checkout v4, action-gh-release v2)

Pre-ship pin staleness check per docs/RELEASING.md: both floating major
tags moved upstream; pins updated to the current tag commits.

* v0.42.65.0 chore(release): 92 verified fixes since v0.42.64.0 — changelog + version bump

Aggregates everything merged to master since the v0.42.64.0 bump commit:
community fixes, credited takeovers, batch re-lands, CI hardening, and
maintainer-approved features. Net commit list excludes revert pairs.
No new schema migrations.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(deps): clear OSV-flagged transitive dependencies via override floors

Raise the existing security-floor overrides so the lockfile resolves
patched versions of three transitive packages flagged by the OSV scan
(@hono/node-server, fast-uri, body-parser). None are on gbrain's own
runtime path (@hono/node-server is only referenced by the MCP SDK's
optional hono transport, which gbrain does not load); the floors keep
the dependency scan green. MCP/OAuth unit tests pass against the
resolved versions.

* chore(release): fold #3110 into the v0.42.65.0 entry (93 net changes)

---------

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-23 16:45:29 -07:00
Time AttakcGarry TanClaude Fable 5
 		d69f211629 fix(ci): delta-assert reporter leak test + raise shard timeout to 22min (#3231) 
The signal-handler test asserted an absolute liveReporters===0 on a
module-global set, so any other test file in the shard holding a live
reporter flaked it — it red-flagged ~12 unrelated PR runs and one master
push in two days, purely as a function of shard composition. The delta
form pins the same claim (50 reporter lifecycles leak nothing).

The 15-minute shard timeout cancelled 13 fully-passing runs under
parallel PR load (PGLite WASM cold-starts stretch shards); the
test-status gate then reported the cancellations as failures.

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-22 13:41:32 -07:00
Time AttakcClaude Fable 5Sinabina
 		b075a9c8d4 test+ci: unbreak master — symlink-walker probe files + OSV caller permissions (#2926) 
* test: symlink-walker tests use a non-metafile probe (README now skipped by design, #2315)

The import walker deliberately skips README/metafiles since #2315 (closing
#345); the symlink-hardening tests used README.md as their probe file and
went red on the intersection. Probe with notes.md instead; test intent
(cycle hardening + strategy filter) unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: grant security-events write to the OSV caller job — reusable workflow requires it at startup

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-17 12:10:47 -07:00
Time AttakcSinabinaClaude Fable 5
 		9aaa3be05f ci(security): OSV dependency scan, release artifact attestations, Semgrep CE SAST (#2182 #2142 #2272) (#2917) 
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-17 11:05:13 -07:00
mzkarami 414940204a ci(release): run verify before build (#2243) 2026-07-16 20:47:35 -07:00
Garry TanandClaude Opus 4.8 814258dda6 v0.42.53.0 fix(sync,db): #2339 op_checkpoints jsonb double-encode + bug-class sweep + CI guard (#2375) 
* fix(sync): op_checkpoints pin write double-encodes jsonb — every sync aborts (#2339)

recordCompleted bound JSON.stringify(array) to a $3::jsonb param via postgres.js
.unsafe(), double-encoding it into a jsonb string scalar that violates the v119
op_checkpoints_completed_keys_array CHECK — aborting every multi-source sync on
real Postgres at the first checkpoint write. PGLite parses the string silently,
which is why unit tests stayed green and it shipped. Cast through $3::text::jsonb
so the text->jsonb cast parses a genuine array. Adds a DATABASE_URL-gated parity
test + a dedicated Postgres CI job so the guard can never silently skip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(db): sweep positional jsonb double-encode sites + AST CI guard (#2324)

Every executeRaw/.unsafe site that bound JSON.stringify(x) to a bare positional
jsonb cast double-encodes on real Postgres (same class as #2339). Sweep them all
to the text::jsonb form across query-cache, sources-ops, llm-base,
calibration-profile, impact-capture, subagent, receipt-write, traversal-cache,
symbol-resolver, and the agent/sources commands. Adds scripts/check-jsonb-params.mjs
(AST-lite scanner for the positional form the legacy template grep misses, incl.
generic-typed calls), wired into check-jsonb-pattern.sh, with a self-test. PGLite's
native db.query is not scanned — it parses text to jsonb natively, so the bug can't
occur there.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(search,eval): alias-hop injected results carry page_id (contradiction-probe crash)

applyAliasHop injected synthetic SearchResults without page_id (the `as
SearchResult` cast hid the missing field), so listActiveTakesForPages bound
undefined/NaN into ANY($1::int[]) and crashed the whole contradiction probe on
real Postgres. Stamp page_id=page.id at the injection site and add a finite-id
filter in generateIntraPagePairs as a defensive backstop (mirrors hybrid.ts:63).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(engines): positional jsonb binding rule (text::jsonb vs the double-encode trap)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* v0.42.53.0 fix(sync,db): #2339 op_checkpoints jsonb double-encode + bug-class sweep + CI guard

Bumps VERSION + package.json to 0.42.53.0, adds the CHANGELOG entry, and
regenerates llms-full.txt. Ships the #2339 sync-abort hotfix, the repo-wide
positional jsonb double-encode sweep, the alias-hop contradiction-probe crash
fix, and the new positional-form CI guard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: post-ship sync — jsonb invariant now covers the positional form + new guard

CLAUDE.md JSONB invariant + KEY_FILES (sql-query, check-jsonb-pattern, op-checkpoint)
now describe the #2339 positional double-encode class, the $N::text::jsonb fix, and
the new check-jsonb-params.mjs guard. Regenerates llms-full.txt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-24 06:05:16 -07:00
Garry TanandClaude Opus 4.8 9bf96db807 v0.42.51.0 fix(sync): contention-free clock + checkpoint integrity + honest sync freshness (#2255) 
* fix(sync): contention-free page-generation clock — sequence swap

The page-generation clock backed the query-cache Layer-1 bookmark via a
FOR EACH STATEMENT trigger running `UPDATE page_generation_clock SET
value=value+1 WHERE id=1`. That took a transaction-length RowExclusiveLock
on one tuple, so every concurrent page writer serialized on the prior
writer's COMMIT — sync ran at ~0.8 cores regardless of worker count.

Swap to a SEQUENCE bumped by nextval() (a microsecond LWLock, never a row
lock). The clock's only contract is monotonic advancement on any page
INSERT/UPDATE/DELETE; last_value is non-transactional, so rolled-back or
concurrent-uncommitted writers only OVER-invalidate the cache (lose a hit),
never serve stale.

- migration v118: CREATE SEQUENCE + load-bearing 2-arg setval (is_called=
  true, floor 1, seeded >= old clock and MAX(generation)) + repoint the
  trigger function body + DELETE query_cache so no old-clock bookmark
  survives the swap. v107 left immutable.
- query-cache-gate.ts: 3 readers -> SELECT last_value FROM page_generation_clock_seq.
- schema.sql + pglite-schema.ts (+ regenerated schema-embedded.ts) ship the
  sequence on fresh install; table + trigger names retained.
- tests: clockValue reads last_value; mechanism proof (trigger fn uses
  nextval not the row UPDATE); rollback-advances-clock safety pin; real
  PGLite sequence round-trip (is_called gotcha); shape test requires _seq.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sync): op_checkpoints array-shape guard — CHECK + repair + defensive loader

completed_keys is JSONB and the checkpoint loader runs
jsonb_array_elements_text over it. A non-array (scalar) value makes that
throw "cannot extract elements from a scalar", which takes down the whole
UNION load — including the valid op_checkpoint_paths child rows — and loses
all checkpoint progress for that key. No current writer produces a scalar,
but an older binary / external script / future bug could.

Make the corruption class structurally impossible and self-healing:

- migration v119: LOCK TABLE (so an out-of-band scalar can't land between
  repair and constrain; no-op on single-connection PGLite), repair any
  pre-existing scalar to '[]' (op_checkpoint_paths child rows are the
  append-only source of truth, so the reset loses nothing), then add the
  named CHECK (jsonb_typeof(completed_keys) = 'array') via a pg_constraint
  IF NOT EXISTS guard. A DB-enforced always-on guard — the correct pattern
  vs a migration verify-hook, which never runs on already-stamped brains.
- schema.sql + pglite-schema.ts (+ regenerated schema-embedded.ts) ship the
  same NAMED inline CHECK so fresh installs match migrated brains and v119
  skips the duplicate.
- op-checkpoint.ts loader: gate the legacy arm on jsonb_typeof = 'array' so
  a scalar parent is skipped (children still load) instead of throwing the
  whole union, and log a specific corruption warning when one is seen.
- tests: CHECK rejects a scalar (exactly one constraint, no blob+migration
  dupe); loader survives a scalar parent and returns the children; v119
  repair converts a scalar to '[]'.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(doctor): report actively-running sync via live lock, not stale freshness

A slow source that makes partial progress every cycle but never fully
completes used to read as permanently "stale" / "never synced" because
last_sync_at only advances on a full successful sync. The naive fix
(treat recent checkpoint banking as "in progress") is unsafe: a blocked
sync banks the good files then writes no anchor, so banking can't tell
in-progress from wedged.

Use the only honest signal: a LIVE, non-expired per-source sync lock
(inspectLock + syncLockId against gbrain_cycle_locks). Every non-skipLock
sync holds it and refreshes it; a blocked/failed sync's process has exited
(no lock row) and a wedged holder stops refreshing (TTL lapses), so either
correctly falls through to the stale path and is NEVER masked. An
actively-syncing source (including a never-synced source doing its first
sync) counts as synced_recently, preserving the pinned 3-bucket invariant.
The lock lookup reuses doctor's existing dynamic db-lock import and swallows
any throw (stub engine, pre-lock-table brain) to false, so it can only ADD
an in-progress verdict, never suppress a real stale one.

Tests (real PGLiteEngine + real lock rows): stale+no-lock -> fail;
stale+live-lock -> ok; never-synced+live-lock -> ok; never-synced+no-lock
-> fail; expired-TTL lock -> fail (wedged not masked); blocked source with
banked checkpoint rows but no lock -> still fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sync): honest --force-break-lock diagnostic when no lock is held

--force-break-lock used to emit the same terse "Lock ... is not held
(nothing to break)" line and exit 0 even when a sync was genuinely wedged,
sending the operator down a dead end — the wedge was not a held lock. Keep
rc=0 (breaking a non-existent lock is idempotently successful; flipping the
exit code would break automation), but under --force say plainly that
nothing was broken and point at the real next step (gbrain sync / gbrain
doctor) plus a `wedge_hint` field in --json output. The non-force path is
byte-for-byte unchanged.

runBreakLock is exported for the test. Tests: force+no-lock -> wedge_hint
JSON + human hint, rc 0; non-force+no-lock -> unchanged terse line, no hint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(doctor): surface the in-progress sync holder in the freshness message

Plan-completion follow-up to the BUG 4 live-lock signal: when a source is
actively syncing, name the holder (pid + host) in the check message instead
of silently folding it into synced_recently. The note is appended only when
something is in progress, so steady-state messages stay byte-for-byte
unchanged (the pinned exact-message + 3-bucket-invariant tests still pass).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sync): pre-landing review fixes — monotonic clock seed, scoped CHECK guard

Adversarial (codex) review of the implementation diff caught three:

- P1 (correctness): the fresh-schema setval was not monotonic. initSchema
  replays the schema blob, and the unconditional setval(MAX(generation))
  could move page_generation_clock_seq.last_value BACKWARD on an
  already-upgraded brain, letting a stored query_cache bookmark serve stale
  rows. Seed via GREATEST over the sequence's OWN last_value (+ old table
  value + MAX(generation)) in all 3 fresh schemas and migration v118, so a
  replay is idempotent — mirrors the old table's ON CONFLICT DO NOTHING.
  Pinned by a new monotonic regression test.
- P2: v119's CHECK-exists guard keyed on conname only (not globally unique).
  Scope it to conrelid = 'op_checkpoints'::regclass.
- P3: in-progress note ran into the prior sentence in fail/warn doctor
  messages; separate it with '. '.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: make Anthropic/ZE no-key tests hermetic against a dev config key

These "no key" tests cleared only ANTHROPIC_API_KEY / ZEROENTROPY_API_KEY
from the env, but hasAnthropicKey() and checkZeEmbeddingHealth() also read
the key from ~/.gbrain/config.json. On a dev machine whose real config holds
a key, the no-key assertions flipped and the tests failed locally (they
passed only in key-less CI). Add a shared with-env emptyHome() helper and
point GBRAIN_HOME at an empty dir in every no-key path so loadConfig finds
nothing — matching the already-hermetic anthropic-key / gateway-probe tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.44.1.0)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(key-files): sync doctor + op-checkpoint entries to v0.44.1.0 truth

checkSyncFreshness now reports an actively-running sync via the live
per-source lock (names holder pid+host, counts as synced_recently) instead
of flagging it stale; loadOpCheckpoint gates the legacy union arm on
jsonb_typeof = 'array' so a scalar parent can't take down the whole load,
and migration v119's CHECK constraint makes the corruption class
structurally impossible. Reference docs describe current behavior only —
both entries updated in place, no release-clause appends. Guard + llms
freshness test green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: re-version to v0.42.51.0 (natural next-off-master)

Maintainer override of the queue allocator's leap to 0.44.1.0 (it jumped past
in-flight sibling PR claims at 0.42.50/0.43.0/0.44.0). Take the natural next
slot in the 0.42.x line above the immediate sibling claim (0.42.50.0); a
merge re-bump resolves any collision if a cathedral PR lands first.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci(e2e): bound + retry the OpenClaw install so a transient npm hang can't burn the Tier 2 budget

The Tier 2 (LLM Skills) job failed at 30m16s — the `npm install -g
openclaw@2026.4.9` step hung on a transient npm/registry stall (orphan
`npm install openclaw` was still running at cancel time) and consumed the
entire 30m job budget that v0.42.50.0 (#2254) introduced. The install
normally finishes in under a minute (Tier 2 is ~4m end to end on master),
so this is flaky-install infra, not a test failure.

Wrap the install in `timeout 120` + a 3-attempt retry loop with an 8-minute
step backstop: a hung attempt is killed in 2 min and retried instead of
eating the whole job. Same bound-the-hang philosophy as #2254's job timeouts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-17 14:02:47 -07:00
Garry TanandClaude Opus 4.8 70d5f36db6 v0.42.50.0 ci: reliability hardening — cancel-superseded + per-job timeouts + actionlint + hermetic E2E env (#2254) 
* ci: cancel superseded runs + per-job timeouts on test.yml & e2e.yml

Ports the GH-Actions hygiene gbrain already uses in heavy-tests.yml to the two
hot-path workflows. concurrency cancels a superseded run (keyed on PR number for
pull_request events — fork-safe — with github.ref fallback for push/scheduled);
frees runners and stops a stale-SHA run reporting a flaky failure on an obsolete
commit. Per-job timeout-minutes (test matrix 15, verify 12, serial 15, slow-* 12,
e2e tier1 20 / tier2 30, trivial jobs 5-10) convert a wedged job from a 6-hour
zombie (GitHub's default) into a fast legible fail. fail-fast:false already set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: add actionlint workflow (rhysd/actionlint v1.7.11, SHA-pinned)

Lints workflow YAML on .github/workflows/** changes so a malformed workflow /
bad action ref / missing-permission bug is caught before it ships a broken
pipeline. gbrain edits these workflows often; cheap preventive guard. Mirrors
GStack's actionlint job, SHA-pinned to gbrain's convention.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci(e2e): scrub operator/agent env before E2E (hermetic runner)

A dev or Conductor shell exports CONDUCTOR_*/MCP_*/OPENCLAW_*/GBRAIN_* overrides
that silently change test behavior, making hermetic E2E non-hermetic and its
failures unreproducible across machines. run-e2e.sh drops those prefixes before
bun starts (denylist — PATH/HOME/TMPDIR/DATABASE_URL survive; GBRAIN_HOME kept
for the existing HOME isolation). Adapts GStack's buildHermeticEnv to gbrain's
shell runner. Verified: a 78-test e2e file passes with DATABASE_URL surviving +
a planted GBRAIN_BRAIN_ID scrubbed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* v0.42.50.0 ci: reliability hardening — cancel-superseded + per-job timeouts + actionlint + hermetic E2E env

Ports GStack's CI-reliability hygiene to gbrain's hot-path workflows: concurrency
cancel-in-progress (PR-number keyed), per-job timeout-minutes (no more 6-hour
zombie jobs), an actionlint workflow, and an operator-env scrub in run-e2e.sh.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-17 07:44:08 -07:00
Garry TanClaude Opus 4.8t
 		f79c1306a2 v0.41.32.0 fix(staleness): commit-relative sync staleness (supersedes #1623) (#1656) 
* fix(staleness): commit-relative sync staleness (HEAD-hash local, durable column remote)

Quiet, fully-caught-up repos no longer false-alarm as SEVERELY STALE in
gbrain doctor / sources status. Staleness now means "is there committed
content the sync hasn't ingested?" not raw wall-clock since the last sync.

- git-head.ts: requireCleanWorkingTree gains 'ignore-untracked' mode (git
  status --porcelain --untracked-files=no). Untracked dirs no longer defeat
  the freshness short-circuit — sync's incremental path keys off the commit
  diff and never imports untracked files, so doctor agrees with sync.
- source-health.ts: newestCommitMs (HEAD committer time) + pure
  lagFromContentMs comparator; computeAllSourceMetrics {probeContent} routes
  local→live commit-hash, remote→stored column. Dead isSourceStale removed.
- migration v108 sources.newest_content_at + fresh-schema blobs.
- sync.ts: writeSyncAnchor stamps newest_content_at atomically with
  last_commit/last_sync_at; buildSyncStatusReport (remote get_status_snapshot)
  reads the column — no git subprocess (v0.41.27.0 trust boundary intact).
- doctor.ts: checkSyncFreshness short-circuit ignores untracked; remote path
  reads the column; clock-skew check stays on raw wall-clock.

Local consumers probe live git (catch HEAD moving to an old-dated commit, which
a timestamp compare would miss); remote consumers read the durable column so a
remote-callable endpoint never shells out to a DB-supplied local_path.

Supersedes #1623 (re-implemented in base repo with the trust boundary preserved).

Co-Authored-By: t <t@t>

* chore(ci): offload tests to on-demand cloud runners from a local CLI

scripts/ship-remote-tests.sh pushes the branch, dispatches the test workflow,
and blocks on `gh run watch --exit-status` — a local caller (human or agent)
awaits the GitHub run exactly like a local `bun run test`, with a real pass/fail
exit code. Frees a load-saturated local machine (many Conductor agents running
their own bun-test suites at once → load avg 120 on 16 cores → PGLite OOM/crawl).

test.yml gains workflow_dispatch so the suite can be triggered from any branch.

* chore: bump version and changelog (v0.41.32.0)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: t <t@t>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-05-30 10:56:46 -07:00
Garry TanandClaude Opus 4.7 552ff4ed82 v0.41.11.1 ci: cut CI wallclock from 9min to 4.5min (#1457) 
* feat(eval-longmemeval): RunOpts.engine seam for shared benchmark brain

Adds optional `engine?: PGLiteEngine` field to RunOpts. When set,
runEvalLongMemEval uses the caller-provided engine and skips the
withBenchmarkBrain wrapper (no fresh PGLite create, no disconnect on
exit). When unset, the production CLI path is unchanged: withBenchmarkBrain
creates and disposes a fresh engine per invocation.

Designed for the test seam that's about to land: one beforeAll-created
brain shared across all 13 runEvalLongMemEval calls in
test/eval-longmemeval-e2e.slow.test.ts, amortizing the ~1-3s PGLite
cold-create cost. runOneQuestion already calls resetTables() as its first
line so per-test isolation is preserved across the shared engine.

Pure additive seam — every existing caller (CLI, current tests that
already create engines via withBenchmarkBrain implicitly) keeps its
current behavior because opts.engine defaults to undefined.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(test): split eval-longmemeval slow tests + share engine across e2e half

The 884-line test/eval-longmemeval.slow.test.ts was the heaviest single
file in CI at ~359s on the matrix. Split by runEvalLongMemEval usage:

- test/eval-longmemeval.slow.test.ts (trimmed): 8 pure describes, 15 tests.
  Harness lifecycle, resetTables, schema-migration robustness, warm-create
  speed gate, adapter haystackToPages, source-boost guard, loadResumeSet,
  buildByTypeSummary. Local wall: 1.985s, projected CI ~42s.

- test/eval-longmemeval-e2e.slow.test.ts (NEW): 8 e2e describes, 11 tests.
  Every describe that calls runEvalLongMemEval — 13 call sites total.
  Threads a single beforeAll-created PGLite via the v0.41.10 RunOpts.engine
  seam. Local wall: 9.33s (was 15.09s without sharing); projected CI ~196s
  (was ~317s).

- test/helpers/longmemeval-stub.ts (NEW): shared makeStubClient + StubCall.
  Matches the existing test/helpers/ convention (with-env.ts,
  reset-pglite.ts). Single source of truth across the two split files.

- scripts/test-weights.json: replaced 359087ms entry with TWO entries
  (42000ms pure, 196000ms e2e). Projected linearly from local wall-clock
  × 21 CI scaling factor. First post-merge CI run will refine via
  scripts/mine-shard-weights.ts.

Test count is preserved: 15 pure + 11 e2e = 26, matches original file.
No production code changes in this commit — only test reorganization +
opt-in to the RunOpts.engine seam from the previous commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(test): bump matrix 6→10, dedicate two slow files, cache bun-install

CI matrix wallclock: ~9 min → ~4.5 min. Three coordinated changes.

1. .github/workflows/test.yml matrix bumped from 6 → 10 shards. Per-shard
   total drops from 532s → 272s. Honest concurrency-budget call: total
   gated jobs go 13 → 18, so 2 concurrent PRs ≈ 36 queued, past the
   GH free-tier ~20 ceiling — single-PR runs unaffected, multi-PR days
   see queue pressure. Worth it for the 4-min CI saving.

2. Two slow files pulled out of the matrix and into their own dedicated
   jobs (sibling to verify, serial-tests):

   - slow-eval-longmemeval runs test/eval-longmemeval-e2e.slow.test.ts
     (~196s after the engine-sharing seam from the previous two commits).
   - slow-entity-resolve-perf runs test/entity-resolve-perf.slow.test.ts
     (~159s, single non-subdivisible perf test). The 60s default bun
     timeout is too tight for this file — bumped to 300000ms.

   scripts/test-shard.sh excludes both via -not -name clauses so the
   matrix sweep doesn't double-run them. Both new jobs wire into
   cache-write.needs and test-status.needs so CI gates on them.

3. actions/cache for ~/.bun/install/cache added to every job that runs
   bun install (test matrix, verify, serial-tests, slow-eval-longmemeval,
   slow-entity-resolve-perf). Keyed on bun.lock hash. Saves ~15s per job
   on cache hit; first-PR push pays full cost, subsequent runs hit cache.

Total CI wallclock now bounded by max(matrix ~4.5min, slow-eval ~3.3min,
slow-entity-resolve-perf ~2.6min) = ~4.5 min. The matrix is back to
being the floor; no single test file dominates a shard.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: v0.41.10.0 — CI wallclock 9min → 4.5min

VERSION + package.json + CHANGELOG entry for the three preceding commits:

  feat(eval-longmemeval): RunOpts.engine seam for shared benchmark brain
  refactor(test): split eval-longmemeval slow tests + share engine across e2e half
  ci(test): bump matrix 6→10, dedicate two slow files, cache bun-install

Net user-visible: CI 'Test' check finishes in ~4.5 min instead of ~9 min.
Net contributor-visible: new RunOpts.engine seam on runEvalLongMemEval for
benchmark suites that want to amortize PGLite cold-create across many calls.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(test): quarantine hybrid-meta + schema-pack-load-active to serial

The 6→10 matrix shard bump in this branch re-shuffled file distribution
across shard processes. Two pre-existing tests with hidden cross-file
state dependencies surfaced as failures in CI run #77779498812/13:

- test/hybrid-meta.test.ts shard 7: gateway state (configured by some
  other test in the same shard process) survived past the test's
  `delete process.env.OPENAI_API_KEY` call, so the early-return for
  expansion didn't fire and `expansion_applied` stayed true.
- test/schema-pack-load-active.test.ts shard 8: the schema-pack module's
  test-injected locator state was left behind by an earlier file, so
  `loadActivePack` with the default config didn't fall through to the
  bundled gbrain-base path.

Both files pass cleanly solo (verified). The pollution sources are
unidentified — bun's reporter only printed 14 of 71 file headers per
shard log, hiding the polluters. Rather than spelunk for the source,
rename both files to *.serial.test.ts. The serial pass runs them at
--max-concurrency=1 in a process that doesn't share state with the
parallel matrix shards.

Same-wave bookkeeping:
- scripts/check-test-isolation.allowlist: drop test/hybrid-meta.test.ts
  entry (file is now serial, no longer R1-eligible).
- scripts/test-weights.json: rename both weight entries to match the
  new filenames so future matrix LPT runs don't fall back to median.

Companion to a7d029d0/2e1c269e/5a749acb of this branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 15:46:25 -07:00
Garry TanandClaude Opus 4.7 3a2605e9a0 v0.41.6.0 feat(ci): CI test speedup — 23min → ~9min via matrix 4→6 + weight-aware sharding + auto SHA cache + parallel verify (#1444) 
* feat(ci): scripts/run-verify-parallel.sh — parallel verify dispatcher

Fans out the 21 pre-test grep guards via & + wait, captures per-check
exit codes in a tempdir, aggregates failures with named check + log
tail to stderr on miss. Wallclock 27s sequential → 13s parallel
locally (2x). Bigger CI win is shard 1 deload (workflow restructure
in a later commit).

Pinned by test/scripts/run-verify-parallel.test.ts (6 cases: CLI
contract + synthetic dispatcher failure-surfacing).

* feat(ci): weight-aware LPT bin-packer + auto SHA cache hash

scripts/sharding.ts (NEW) — pure TypeScript LPT bin-packer. Sort
weights desc, assign each file to the shard with current minimum total.
Worst-case makespan within 4/3 of optimal, O(n log n). Missing weights
fall back to corpus median (not 0). New test file → ships immediately
without regenerating weights. Pinned by test/scripts/sharding.test.ts
(23 cases).

scripts/mine-shard-weights.ts (NEW) — scrapes per-file timing from
gh run view --log via timestamp delta between ##[group]test/foo.test.ts:
headers within a shard. Three input modes: --run <ID>, --from-file
<PATH>, stdin. Stable JSON output (sorted keys). Initial weights mined
from run 26398061007. Pinned by test/scripts/mine-shard-weights.test.ts
(15 cases).

scripts/ci-cache-hash.sh (NEW) — deterministic 16-char sha256 over
git ls-files -s minus deny-list (CHANGELOG/TODOS/README/LICENSE/
docs/**/*.md). CLAUDE.md, AGENTS.md, skills/**/* deliberately
INCLUDED (8+ test files read them; deny-listing would create
false-pass holes). ~40ms on 1891 files. Pinned by
test/scripts/ci-cache-hash.test.ts (24 cases: 8 CRITICAL false-pass
guards + 7 SAFE deny-list invariants + 9 edge cases).

scripts/test-weights.json (NEW) — 712 weights. Total 3306s observed
runtime; median 30ms; max 6 min outlier.

* chore: bump version and changelog (v0.41.6.0)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-25 13:15:11 -07:00
Garry TanandClaude Opus 4.7 d28be5d091 v0.40.4.0 feat(search): selective graph signals + per-stage attribution + audit-writer unification (#1300) 
* v0.40.4.0 T1: shared audit-writer primitive

Extract createAuditWriter() helper. Five hand-rolled JSONL audit
modules (rerank-audit, shell-audit, supervisor-audit, audit-slug-
fallback, phantom-audit) duplicated the same ISO-week filename math,
best-effort write loop, and read-current-plus-previous-week loop.
T2 refactors all 5 onto this primitive.

Behavior preservation: filename format, JSONL line shape, mkdir
recursive, appendFileSync utf8, stderr-on-failure all byte-identical
to the existing modules so their tests pass unchanged.

resolveAuditDir() moves here from shell-audit.ts; shell-audit.ts
will re-export for back-compat (T2). Honors GBRAIN_AUDIT_DIR with
whitespace-trim, falls back to ~/.gbrain/audit/.

Test coverage: 22 cases covering ISO-week math + year-boundary edges
(2027-01-01 → 2026-W53), env override, mkdir-recursive, fail-open
stderr-warn shape, cross-week readback, corrupt-row skip, non-finite-
ts skip, round-trip with nested fields, computeFilename + resolveDir
accessors.

Plan ref: D5=B audit unification cathedral expansion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T2: refactor 5 audit modules onto shared writer

Replace the duplicated ISO-week filename math + best-effort write loop
+ read-current-plus-previous-week loop in:
  - src/core/rerank-audit.ts (rerank-failures-*.jsonl)
  - src/core/audit-slug-fallback.ts (slug-fallback-*.jsonl)
  - src/core/minions/handlers/shell-audit.ts (shell-jobs-*.jsonl)
  - src/core/minions/handlers/supervisor-audit.ts (supervisor-*.jsonl)
  - src/core/facts/phantom-audit.ts (phantoms-*.jsonl)

All five now delegate file I/O to createAuditWriter from T1. Public
API preserved bit-for-bit:
  - logRerankFailure, readRecentRerankFailures, computeRerankAuditFilename
  - logSlugFallback, readRecentSlugFallbacks, computeSlugFallbackAuditFilename
  - logShellSubmission, computeAuditFilename, resolveAuditDir
  - writeSupervisorEvent, readSupervisorEvents, computeSupervisorAuditFilename
    plus isCrashExit, summarizeCrashes, CrashSummary (domain-specific
    helpers stay in supervisor-audit.ts; only file I/O moves)
  - logPhantomEvent, readRecentPhantomEvents, computePhantomAuditFilename

Domain-specific behavior preserved:
  - audit-slug-fallback emits per-call stderr (D7 dual logging) in the
    caller; the shared writer is failure-only stderr
  - rerank-audit truncates error_summary to 200 chars before write
  - phantom-audit spreads optional fields conditionally (skip undefined)
  - supervisor-audit keeps single-file readback (no cross-week walk)
    to preserve pre-v0.40.4 doctor assertions

resolveAuditDir lives in src/core/audit/audit-writer.ts; shell-audit.ts
re-exports it so existing imports keep working (every other audit
module + gbrain-home-isolation.test.ts + minions.test.ts +
minions-shell.test.ts pull resolveAuditDir from shell-audit.ts).

Operator-visible drift: rerank-audit stderr line drops the
'rerank-failure audit' qualifier — was '[gbrain] rerank-failure audit
write failed (...)' now '[gbrain] write failed (...); search continues'.
Stderr is human-debugging, not machine-parsed; the file written gives
the qualifier away in `tail -f audit/*`.

Test coverage: 128/128 audit-touching tests pass unchanged.

Plan ref: D5=B audit unification cathedral expansion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T3: getAdjacencyBoosts engine method (PG+PGLite parity)

Add BrainEngine.getAdjacencyBoosts(pageIds) returning Map<page_id,
AdjacencyRow{hits, cross_source_hits}>. Returns ALL pages with
hits >= 1 (callers apply their own threshold).

Cross-source semantic (D15=A): cross_source_hits EXCLUDES the target
page's own source. A page in source A linked from 2 pages in source A
reports cross_source_hits = 0. Linked from 1 in source B + 1 in
source C reports 2.

Source-scope contract: pageIds MUST already be source-scoped by the
caller. Method does NOT filter by source_id. The in-set restriction
makes cross-source leakage impossible by construction. JSDoc spells
this out; same trust posture as cosineReScore's chunk_id handling.

COALESCE(p.source_id, 'default') on both target and from-page sides
for defense-in-depth even though pages.source_id is NOT NULL today.

JSDoc/SQL contract alignment (codex #2): HAVING >= 1 matches the
"returns ALL pages with hits >= 1" contract; threshold of 2 is the
caller's call in applyGraphSignals.

Known limitation (codex #15): cross_source_hits cannot distinguish
"genuinely linked from another team" from "mirrored imports from
another source." T-todo-4 captures the v0.41+ refinement.

SearchResult type extension (D4=A flat fields, D12=A attribution):
  - graph_adjacency_hits, graph_cross_source_hits,
    graph_session_demoted, graph_session_prefix
  - base_score, backlink_boost, salience_boost, recency_boost,
    exact_match_boost, graph_adjacency_boost, graph_cross_source_boost,
    session_demote_factor, reranker_delta
All optional; T4-T6 populate them.

Test coverage: 7/7 hermetic PGLite cases. Empty input, singleton,
same-source hub, cross-source attribution including the
"linked-only-from-other-source" case (widget in source b, linked
from alice+bob in source a → cross_source_hits=1), JSDoc HAVING>=1
contract. Postgres parity asserted by SQL-shape identity (will get a
mirror Postgres E2E in T10's eval gate work via DATABASE_URL when
set; PGLite hermetic case shipped now).

NULL source_id COALESCE branch noted as untestable in current PGLite
schema (pages.source_id is NOT NULL); kept as defense-in-depth.

Plan ref: T3 in v0.40.4.0 wave plan; D1=A, D3=A, D15=A.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T4+T11: applyGraphSignals 4th stage in runPostFusionStages

New file src/core/search/graph-signals.ts. Three signals:

  1. Adjacency-within-top-K (×1.05): hits >= 2 inbound from in-set.
  2. Cross-source adjacency (×1.10, stacks): cross_source_hits >= 2.
     Dormant on single-source brains.
  3. Session diversification (×0.95): if multiple top-K share a slug
     prefix, keep highest scoring, DEMOTE the rest. NOT amplify —
     codex caught the original framing was backwards (amplification
     of redundancy makes the cited "weak chunks compete for budget"
     problem worse, not better).

Conservative magnitudes (D14=B): 1.05/1.10/0.95. Score-distribution
probe (onScoreDistribution) collects min/p25/p50/p75/p95/max +
reorder_band_width to feed T-todo-2 magnitude calibration wave.

Slot: 4th stage inside runPostFusionStages (hybrid.ts:248), AFTER
backlink/salience/recency, pre-dedup. Inherits the v0.35.6.0
floor-ratio gate from computeFloorThreshold — this is the structural
protection that prevents a low-cosine hub from outranking a strong
non-hub (codex T2 / D1=A).

PostFusionOpts extends with graphSignalsEnabled, onGraphMeta,
onScoreDistribution. Caller (hybridSearch in subsequent T5 work)
resolves graph_signals from the mode bundle.

Source-scope contract preserved: getAdjacencyBoosts takes raw
page_ids, no source filter. Adjacency is in-set restricted so
cross-source leakage is impossible by construction (D3=A).

Fail-open: engine throw → JSONL audit row via shared createAuditWriter
(T1/T2 primitive, featureName='graph-signals-failures') + meta.errored
+ caller's results unchanged. Session diversification ALSO skips on
failure (predictable all-or-nothing posture).

Mutation note (codex #9): score mutated in place. base_score must be
stamped at runPostFusionStages entry BEFORE this stage so eval-capture
sees pre-boost score (T6 attribution wave).

Test coverage (24 cases, including T11 IRON RULE regression):
  - sessionPrefix multi/single/empty cases
  - computeScoreDistribution percentile math
  - Disabled + empty short-circuits
  - Adjacency hit, no-hit, cross-source stacking, cross-source alone
  - Session diversification 3-share + single-segment + singleton
  - Test seam injection (no engine call)
  - Fail-open: throw → audit row + meta.errored + unchanged
  - Empty Map → session still runs
  - Score-distribution always emits when enabled
  - Meta carries fire counts + duration_ms
  - Missing page_id silently skipped from dedup set
  - **T11 IRON RULE regression (3 cases):**
    * weak hub BELOW floor_threshold does NOT get boosted past
      above-floor non-hub (the bug class the floor gate exists for)
    * hub AT floor still gets boosted (gate is < not <=)
    * NaN score → NaN >= threshold is false → no boost

Plan ref: T4 + T11 in v0.40.4.0 wave plan; D1=A, D2=A, D11=B, D14=B,
D9=A, D5=B. Codex outside-voice #1 + #2 + #6 + #8 + #9 addressed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T5: graph_signals mode-bundle knob + KNOBS_HASH bump 3→4

ModeBundle gains graph_signals: boolean. Per-mode defaults:
  - conservative: false (cost-sensitive tier)
  - balanced:     true (the wave's primary surface for default-on)
  - tokenmax:     true (power-user tier, capstone fit)

SearchKeyOverrides + SearchPerCallOpts gain optional graph_signals
field. resolveSearchMode picks via the standard per-call → config
override → mode bundle chain.

loadOverridesFromConfig parses 'search.graph_signals' from the config
table ('1' or 'true' → true). SEARCH_MODE_CONFIG_KEYS adds the key
so `gbrain search modes --reset` clears it alongside other knobs.

KNOBS_HASH_VERSION bump 3→4 (append-only per CDX2-F13). New `gs=`
parts entry appended AFTER cross-modal + column + prov entries. A
graph-on cache write cannot be served to a graph-off lookup —
mid-deploy hit-rate dip clears within cache.ttl_seconds (3600s).

src/commands/search.ts KNOB_DESCRIPTIONS gains graph_signals entry
so `gbrain search modes` dashboard renders the new knob.

Test coverage:
  - test/search-mode.test.ts (+ 8 new cases): per-mode defaults
    canonical, config override both directions, per-call override
    wins, knobsHash distinct for on/off, config key registered,
    attributeKnob reports per-call + mode sources correctly.
  - test/search/knobs-hash-reranker.test.ts: version assertion
    bumped 3→4 with v0.40.4 rationale comment.
  - test/cross-modal-phase1.test.ts: version assertion bumped
    3→4 with v0.40.4 rationale comment.
  - Canonical-bundle assertions updated to include graph_signals
    in expected shape (3 cases).

50/50 search-mode tests pass. 45/45 cross-modal pass. 17/17
knobs-hash-reranker pass. 10/10 balanced-reranker pass.

Plan ref: T5 in v0.40.4.0 wave plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T6: per-stage attribution stamping in every boost

Every boost stage that mutates SearchResult.score now stamps a field
recording WHAT it multiplied:

  - applyBacklinkBoost  → backlink_boost (skipped when count == 0)
  - applySalienceBoost  → salience_boost (skipped when score == 0)
  - applyRecencyBoost   → recency_boost (skipped on evergreen prefix)
  - applyExactMatchBoost → exact_match_boost (skipped on no-match
    OR when intent's exactMatchBoost == 1.0 no-op)
  - runPostFusionStages → base_score stamped ONCE at entry, BEFORE
    any boost mutates r.score. Idempotent: caller-pre-stamped value
    preserved. Empty-results short-circuit unchanged.
  - applyReranker → reranker_delta = original_index - new_index
    (positive = rank improved; raw rerank score stays in rerank_score)
  - applyGraphSignals → graph_adjacency_boost, graph_cross_source_boost,
    session_demote_factor (T4 already stamped these)

Why: feeds the T7 `gbrain search --explain` formatter so it can
attribute the final score to its components. Without these stamps,
"why did this rank where it did?" is grep-and-guess.

SearchResult.reranker_delta doc updated to clarify it's a RANK delta
(positive = improved), not a score delta. The raw relevance score
stays in `rerank_score` (untyped, for back-compat with telemetry that
already reads it).

Test coverage: 16 new cases in test/search/attribution-stamping.test.ts.
Pins: every boost stamps when it fires AND skips stamping when it
doesn't (no false attribution on no-op stages). base_score idempotency
preserved. reranker_delta computed correctly across rank-improved +
rank-degraded cases.

All 178/178 search tests pass (no regressions).

Plan ref: T6 cathedral expansion in v0.40.4.0 wave plan; D12=A.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T7: gbrain search --explain per-stage attribution

New file src/core/search/explain-formatter.ts renders SearchResult[]
as a multi-line breakdown of how the final score was formed:

  1. people/alice (score=12.4)
     base=10.2 (rrf+cosine)
     + backlink ×1.08
     + salience ×1.05
     + adjacency ×1.05 (hits=3)
     + cross_source ×1.10 (other_sources=2)
     ↑ reranker rank +2
     = final 12.4

Reads the boost_* / base_score / *_hits fields populated by T4 + T6.
Empty path: "no boosts applied" when no stage stamped anything.
Session demote rendered with `-` prefix (not `+`) so the demotion
direction is visually distinct from boosts.

CliOptions gains `explain: boolean`; parseGlobalFlags recognizes
`--explain` anywhere in argv. cli.ts formatResult for `search` +
`query` cases reads CliOptions.explain via the module-level
singleton and routes to formatResultsExplain when set. Lazy import
keeps the hot path narrow for the common non-explain case.

Number formatting: 4-decimal precision, trailing zeros stripped
('1.0000' → '1', '0.1234' → '0.1234'). NaN preserved as 'NaN'.

Test coverage:
  - test/search/explain-formatter.test.ts: 19 cases pin output
    format. Each boost type renders correctly, every-stage stacking
    composes, reranker_delta=0 doesn't render, empty list short-
    circuits, rank numbering 1-based, number formatting edge cases.
  - test/cli-options.test.ts: 3 new cases for --explain parsing
    (basic, absent default, any-argv-position).

Existing CliOptions literals in test/cli-options.test.ts +
test/thin-client-upgrade-prompt.test.ts updated for new required
explain field.

JSON envelope unchanged — the same attribution fields surface in
existing --json output via JSON.stringify; no separate JSON formatter
needed.

Plan ref: T7 cathedral expansion in v0.40.4.0 wave plan; D12=A + D6=A.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T8: doctor check graph_signals_coverage

New checkGraphSignalsCoverage in src/commands/doctor.ts. Wired into
both runDoctor (local engine) and doctorReportRemote (HTTP MCP /
JSON path) so local AND remote-server brains both surface the metric.

Logic:
  1. Resolve active graph_signals setting: config override
     'search.graph_signals' wins, else mode bundle default
     ('search.mode' → conservative=false, balanced/tokenmax=true).
  2. When disabled → silent ok ("disabled — coverage not checked").
     Avoids polluting doctor output on installs that don't use the
     feature.
  3. When enabled, compute global inbound-link density:
     COUNT(DISTINCT to_page_id) / COUNT(*) across non-deleted pages.
  4. <10% → warn ("signal will rarely fire") with paste-ready
     `gbrain extract all` fix hint.
  5. >=30% → ok ("fire on most queries") with metric.
  6. 10-29% → ok ("fire occasionally") with metric.

Known limitation (codex outside-voice #14): global density is an
imperfect proxy for "top-K subgraphs have enough edges to fire."
T-todo-5 captures the v0.41+ refinement that measures actual fire
rate from search-stats after 30 days of data.

Best-effort: SQL errors → warn with the underlying message. Never
breaks doctor.

Test coverage (7 new cases in test/doctor.test.ts):
  - conservative mode → silent ok regardless of coverage
  - balanced default + 0 links → warn at 0% with fix hint
  - balanced default + 40% inbound → ok "fire on most queries"
  - balanced default + 20% inbound → ok "fire occasionally"
  - explicit search.graph_signals=false overrides mode default
  - empty brain → ok with explanation
  - check is wired into runDoctor (source-grep regression guard)

All 55/55 doctor.test.ts cases pass.

Plan ref: T8 in v0.40.4.0 wave plan; D6=A.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T9: gbrain search stats graph_signals section

runStatsSubcommand in src/commands/search.ts gains a graph_signals
section in both --json and human output:

  Graph signals:
    enabled:    true (mode default)
    failures:   3 fail-open event(s)
      ECONNREFUSED         2
      timeout              1

Data sources:
  - config: 'search.graph_signals' override → enabled + source=config,
    otherwise mode-bundle default → enabled + source=mode_default.
  - JSONL audit: readRecentGraphSignalsFailures(days) returns events;
    failures_count is len, failures_by_reason buckets by first word of
    error_summary (e.g. 'ECONNREFUSED', 'timeout').

JSON envelope (schema_version 2 unchanged; graph_signals is a new
sibling property of stats, so consumers reading the existing fields
keep working):

  {
    "schema_version": 2,
    ...stats...,
    "graph_signals": {
      "enabled": bool,
      "source": "config" | "mode_default",
      "failures_count": int,
      "failures_by_reason": { reason: count }
    },
    "_meta": { metric_glossary: { ..., graph_signals_enabled: ..., graph_signals_failures_count: ... } }
  }

Fire-rate metrics (adjacency_fires, cross_source_fires,
session_demotions) and score-distribution stats are NOT in this
section yet — they require telemetry-table writes from the
applyGraphSignals onMeta callback. Wired in v0.41+ via T-todo-2
calibration wave (the wave that needs them). For v0.40.4: status +
error count is the actionable surface for "is graph_signals on, and
is it failing?"

Human output: prints the section after the existing stats block.
Edge case: when total_calls is 0 BUT graph_signals is enabled OR
has historical failures, still prints the section so operators
don't lose the signal on a brain with no telemetry yet.

Test coverage (6 cases in test/search/search-stats-graph-signals.test.ts):
  - search.graph_signals=true → enabled true, source=config
  - mode=conservative → enabled false, source=mode_default
  - no config → enabled true (balanced default), source=mode_default
  - JSONL failures bucketed by first word of error_summary
  - empty audit → failures_count 0, empty failures_by_reason
  - human output includes "Graph signals:" header

Plan ref: T9 in v0.40.4.0 wave plan; D6=A.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T10: eval gates (longmemeval-mini A/B + paired bootstrap)

New test/e2e/graph-signals-eval.test.ts runs each longmemeval-mini
question twice (graph_signals off, graph_signals on) and asserts:

  Gate 1 (QUALITY) — paired bootstrap, 10,000 resamples:
    - If signals-on is significantly WORSE than off
      (delta < 0 AND p < 0.05) → fail.
    - Otherwise pass. p>=0.05 either direction OR delta >= 0 → ok.

  Gate 2a (CHANGE-MAGNITUDE): mean Jaccard@5 over result-set overlap
    must be >= 0.5. If results overlap less than half, the change is
    too large and needs human review before default-on.

  Gate 2b (CHANGE-MAGNITUDE): top-1 stability rate >= 0.7. If 30%+
    of top picks change, hard look required.

  Gate 3 (HARD ABSOLUTE FLOOR): recall@5 drop <= 5pt. Catastrophic
    regression catch (codex outside-voice #18 — addresses the "top-5
    must not drop at all" brittleness on tiny fixtures).

Bootstrap implementation:
  - Per-question observation is binary (recall@5 hit/miss).
  - Paired pairing on question_id between on/off branches.
  - Centered distribution under null (subtract observed mean) per
    standard paired-bootstrap-shift approach for binary outcomes.
  - Two-tailed p-value: |resampled delta| >= |observed delta|.
  - Deterministic seeded RNG so test runs are stable across CI.

pairedBootstrapPValue exported as a pure function with separate
tests for edge cases (empty input, all-equal, strong positive, strong
negative, determinism). Reusable from future calibration waves.

Hermetic: in-memory PGLite via createBenchmarkBrain + resetTables
between questions. No API keys needed (--no-embed import path
exercises keyword-only retrieval). Skips gracefully via describe.skip
when the fixture is missing.

Plan ref: T10 in v0.40.4.0 wave plan; D7=C absolute floor + D13=A
paired bootstrap; codex #4 + #18 stability-vs-quality distinction.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T12: VERSION + package.json + CHANGELOG + TODOS

VERSION: 0.37.11.0 → 0.40.4.0
package.json: 0.37.11.0 → 0.40.4.0
CHANGELOG.md: top entry for v0.40.4.0 in ELI10-lead voice per
  CLAUDE.md release rules. Lead is plain-English ("Your search now
  notices when a page is a hub for your query"); precise file paths
  / SQL semantics / numbers live in the "Itemized changes" section
  below. Includes the cathedral-expansion notes (D5=B audit
  unification, D12=A per-stage attribution, D13=A eval gates) and
  the "To take advantage of v0.40.4.0" verify-and-fix block.

TODOS.md: 5 new items captured under "v0.40.4 graph signals —
deferred follow-ups (v0.41+)":
  - T-todo-1: profile graph-signal SQL latency, merge if hot (D8=C)
  - T-todo-2: magnitude calibration wave from probe data (D14=B / D17)
  - T-todo-3: DB-backed audit table for cross-deploy observability (codex #15)
  - T-todo-4: sync-topology-aware cross-source signal (codex #11)
  - T-todo-5: replace doctor's global density with fire-rate (codex #14)

Verified the 3-line audit: VERSION + package.json + CHANGELOG topmost
all match 0.40.4.0. `bun install` ran (lockfile unchanged — root
package version isn't stored in bun.lock). `bun run build:llms`
refreshed llms.txt + llms-full.txt for the next commit.

Plan ref: T12 in v0.40.4.0 wave plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 TODO: document pre-existing shard-2 flake noticed during ship

3 isCacheSafe test failures in shard 2 reproduce on stashed clean
master. Confirmed pre-existing — not introduced by v0.40.4. Filed
under "Pre-existing flake on master (noticed during v0.40.4 ship)"
with reproduction commands + remediation options. Shipping v0.40.4
through it; future wave can fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 privacy scrub: replace wintermute → media in example slugs

CLAUDE.md line 550 bans the private OpenClaw fork name in public
artifacts. Example session prefix in sessionPrefix() docs + 3 test
fixtures swept to 'media/chat/...' instead. Pre-existing
scripts/check-privacy.sh in `bun run verify` caught it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 fix: wire graph_signals from mode bundle to runPostFusionStages

CRITICAL: pre-landing review (codex outside-voice via /ship Step 9)
caught that hybrid.ts's `postFusionOpts` literal at line 566 was
building PostFusionOpts WITHOUT threading `resolvedMode.graph_signals`
to `graphSignalsEnabled`. The gate at hybrid.ts:358 read the field
from a literal that never set it.

Result before this fix: the entire v0.40.4 graph-signals wave was
dead code in production. Mode bundles set
`balanced.graph_signals = true` and `tokenmax.graph_signals = true`,
but no production call site ever reached applyGraphSignals. The
KNOBS_HASH bump 3→4 correctly varied the cache key by the flag, so
contamination was prevented — but the feature itself never fired.

All shipped infrastructure (engine SQL, fail-open audit, attribution
stamps, --explain formatter, doctor coverage check, search-stats
section) was reachable only through the unit-test seam
(`opts.adjacencyFn`). The CHANGELOG-advertised behavior never
landed in user-visible search.

Fix: thread `graphSignalsEnabled: resolvedMode.graph_signals` into
the postFusionOpts literal (1 line). Inline comment names codex's
catch so future refactors see the regression class.

Tests: new test/search/graph-signals-wire-integration.test.ts pins
the wire end-to-end. Three cases:
  1. balanced mode → hybridSearch on a seeded brain with adjacency
     hub produces a result with base_score stamped (proves
     runPostFusionStages actually ran).
  2. search.graph_signals=false config override → no graph_* fields
     stamped (proves the gate honors the override path).
  3. Source-grep regression guard pinning the
     `graphSignalsEnabled: resolvedMode.graph_signals` literal in
     hybrid.ts so a future refactor can't silently disconnect.

All 57 existing v0.40.4 wave tests still pass. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 fix: pre-landing review AUTO-FIX findings (audit msg drift + deleted_at)

Two informational findings from /ship pre-landing review (Step 9):

1. Stderr message qualifier drift (rerank/slug-fallback/phantom audits)
   Pre-v0.40.4 messages included a per-feature qualifier:
     [gbrain] rerank-failure audit write failed (...)
     [gbrain] slug-fallback audit write failed (...)
     [gbrain] phantom audit write failed (...)
   The T2 refactor dropped the qualifier (plan promised "byte-identical"
   operator-visible behavior, but stderr lines did drift). Restored via
   new `errorMessagePrefix` option on `createAuditWriter` (optional, ''
   default). Three modules pass the per-feature qualifier; shell-audit
   and supervisor-audit unaffected (their pre-v0.40.4 messages didn't
   have a separate qualifier — label already carried the feature name).

2. Defense-in-depth `deleted_at IS NULL` on getAdjacencyBoosts
   SQL was previously protected by-construction (hybridSearch's
   visibility filter ensures input pageIds are live), but matches the
   v0.35.5.0 findOrphanPages pattern and closes the bug class if a
   future caller bypasses hybridSearch. Added to both Postgres and
   PGLite engines for parity. Three JOIN sites guarded (targets CTE,
   FROM-pages join). One inline comment per engine cites the codex
   review and the v0.35.5.0 precedent.

Plan ref: /ship pre-landing review v0.40.4.0 (codex finding C and F).

All 84 audit+graph-signals tests pass. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 fix: adversarial review HIGH findings (codex H1+H2 + Claude F1)

Three HIGH-severity issues from /ship adversarial pass:

H1 (Codex): Eval gate was a no-op.
  Test passed `graph_signals: graphSignalsOn` via `as any` cast, but
  SearchOpts had no field and hybridSearch's perCall didn't thread it.
  Both off/on branches resolved to the mode-bundle default — gate
  measured identical behavior, could pass while detecting nothing.

  Fix: add `graph_signals?: boolean` to SearchOpts (types.ts:794).
  Thread `opts.graph_signals` into perCall in both hybridSearch
  (hybrid.ts:425) AND hybridSearchCached (hybrid.ts:1027) so the
  cache-key resolver also sees the override. Drop the `as any` from
  the eval test — types are real now.

H2 (Codex): Session diversification fired on entity directories.
  sessionPrefix() used "any shared parent directory" as the session
  signal. Result: a search for "people in SF" returned `people/alice`
  + `people/bob` + `people/charlie` and the latter two got demoted
  to 0.95×. Every common entity-search query silently penalized
  legitimate same-type results. Default-on for balanced/tokenmax
  means production behavior was wrong.

  Fix: narrow sessionPrefix() to fire ONLY when the slug contains a
  session-like marker (`chat`/`session`/`sessions` segment OR a
  `YYYY-MM-DD` date segment). Entity directories (`people/`,
  `companies/`, `docs/`) return null → diversification skips.
  Returns NULL (not the slug itself) so the loop skips clean.
  Examples in JSDoc:
    your-agent/chat/2026-05-20-foo → 'your-agent/chat/2026-05-20-foo'
    daily/2026-05-20/journal-entry-1 → 'daily/2026-05-20'
    transcripts/chat/funding-discussion → 'transcripts/chat/funding-discussion'
    people/alice → null  ← codex H2 regression
    docs/quickstart → null

F1 (Claude adversarial subagent): case-sensitivity drift across 3 sites.
  loadOverridesFromConfig in mode.ts is case-insensitive +
  whitespace-trimmed for 'search.graph_signals' values. But
  doctor's checkGraphSignalsCoverage (doctor.ts:899) AND
  search-stats's readGraphSignalsStats (search.ts:288) used
  case-sensitive compare. User sets `search.graph_signals TRUE`:
  production enables the feature, but doctor + search-stats both
  silently report disabled. Operators lose the only observability
  surface for the new feature on values like 'True'/'TRUE'.

  Fix: trim + lowercase parity at both sites. Mirror the parser's
  semantic. Also case-normalized `search.mode` reads at both sites
  for the same divergence class.

Tests:
  - sessionPrefix block rewritten with 7 cases covering chat marker
    + date anchor + entity dirs (now-NULL) + degenerate (no /).
  - Added regression test pinning codex H2: people/alice +
    people/bob + people/charlie do NOT get diversified.
  - graph-signals-eval.test.ts drops `as any` — typed field works.
  - Existing tests using `chat/a`/`chat/b` updated to session-shaped
    `media/2026-05-20/chunk-a` so the date anchor actually fires.

111/111 graph-signals + doctor + search-stats tests pass. Typecheck clean.

Plan ref: /ship adversarial review v0.40.4.0 (codex H1, H2; Claude F1).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 TODOs: capture 11 LOW adversarial findings for v0.41+

Codex L1 (audit window underreport) + Claude F2/F3/F5-F8/F11/F12/F14/F16
from /ship adversarial review. None are load-bearing; all captured under
'v0.40.4 adversarial review LOW findings — captured for v0.41+'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: update project documentation for v0.40.4.0

- README: surface v0.40.4.0 graph signals + --explain in Hybrid search capability
- CLAUDE.md: annotate engine.ts getAdjacencyBoosts, new graph-signals.ts /
  explain-formatter.ts / audit/audit-writer.ts, plus hybrid.ts post-fusion
  4th stage, mode.ts graph_signals knob + KNOBS_HASH 3→4, cli-options.ts
  --explain flag, search stats + doctor coverage check
- llms-full.txt: regenerated from CLAUDE.md per the build:llms chaser rule

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(ci): pin bun-version to 1.3.13 across all workflows

setup-bun action with `bun-version: latest` calls the GitHub API
(https://api.github.com/repos/oven-sh/bun/git/refs/tags) to resolve
the tag. CI started failing today with HTTP 401 "Bad credentials"
even though the action receives a token (visible as `token: ***`
in the run log). Pinning the version eliminates the API call
entirely.

Affected workflows: test.yml, e2e.yml, release.yml, heavy-tests.yml
(5 invocations total). Pinned to 1.3.13 — matches package.json
engines (`bun >= 1.3.10`) and the version v0.40.4.0 was developed
against.

Bump cadence: when a new bun version is required, update this
pin in one PR. Trading "always-latest" for "always-deterministic"
is the right trade for a 5-shard CI matrix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-23 10:01:08 -07:00
Garry TanandClaude Opus 4.7 9a3ef3cda7 feat: pgGraph-inspired CI scaffolding wave (v0.37.4.0) (#1228) 
Schema-migration matrix + fuzz harness + RSS budget gate + read-latency
under sync + sync lock regression + tests/heavy convention + nightly CI
workflow + BFS frontier cap on traverseGraph.

CI infra (T1-T7):
- tests/heavy/ directory convention + scripts/run-heavy.sh + bun run test:heavy
- tests/heavy/pg_upgrade_matrix.sh: walk pre-v0.13 + pre-v0.18 brain shapes
  forward to head via bootstrap → SCHEMA_SQL → migrations → verifySchema
- test/fuzz/{pure,mixed,filesystem}-validators.test.ts: 1000-run fast-check
  property tests across 8 trust-boundary validators
- scripts/check-fuzz-purity.sh: bun-bundle + grep guard, wired into verify
- tests/heavy/measure_rss.sh: in-memory PGLite workload + peak RSS measurement
  via /proc/self/status (Linux) or process.memoryUsage().rss fallback (macOS,
  refuses to write baseline)
- tests/heavy/read_latency_under_sync.sh: phase A baseline + phase B under
  parallel writer load, reports p50/p95/p99 + delta_pct
- tests/heavy/sync_lock_regression.sh: N concurrent gbrain sync against one
  DB, asserts 1 winner + N-1 lock-busy + zero leaked gbrain_cycle_locks rows
- .github/workflows/heavy-tests.yml: cron '17 8 * * *' + heavy-tests label
  trigger + Postgres service + artifact upload on failure

Engine (T8):
- BrainEngine.traverseGraph opts gain frontierCap?: number + onTruncation?:
  (info: TruncationInfo) => void callback. Return shape preserved
  (Promise<GraphNode[]>) for MCP wire stability.
- Postgres CTE: parenthesized LIMIT N ORDER BY (slug, id) inside recursive term.
- PGLite: same SQL with positional params.
- Per-call callback closure — not engine-instance state — so concurrent
  traversals on the same engine don't cross-talk. 5 contracts pinned in
  test/regressions/v0_36_frontier_cap.test.ts.

Three plan-review passes ran before any code: CEO scope review (Approach C),
Eng dual-voice review (Claude subagent + Codex), and Codex 2nd-pass against
the revised plan. The 2nd pass caught issues the first two missed (Bun ESM
vs require.cache; engine-instance metadata stomping under concurrency;
fixture-size inconsistency). All addressed.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-20 20:25:41 -07:00
Garry TanandClaude Opus 4.7 baf1a47798 v0.35.0.0 feat: ZeroEntropy zembed-1 + zerank-2 reranker (#1008) 
* feat(ai): add ZeroEntropy recipe + reranker touchpoint type

Widens `TouchpointKind` with `'reranker'`, adds `RerankerTouchpoint`
interface, extends `Recipe.touchpoints` and `AIGatewayConfig` to carry
reranker model state. Registers `zeroentropyai` recipe (zembed-1
embeddings + zerank-{2,1,1-small} rerankers) in the recipe registry.

Recipe declares the 7 Matryoshka dims (2560/1280/640/320/160/80/40),
Voyage-style dense-payload hedge (chars_per_token=1, safety_factor=0.5),
and 5MB rerank payload cap. Pinned by test/ai/zeroentropy-recipe.test.ts
including F1 regression (implementation literal is 'openai-compatible')
and F2 regression (base_url_default ends with /v1, no doubling).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(ai/dims): thread input_type 4th-arg + ZE flexible-dim allowlist

`dimsProviderOptions` gains an optional `inputType?: 'query' | 'document'`
4th param so asymmetric providers (ZE zembed-1, Voyage v3+) can route
query-side vs document-side encoding. Per-model filtering inside the
openai-compatible branch keeps `input_type` from leaking to symmetric
providers (OpenAI text-3, DashScope, Zhipu) that would 400 on it.

Adds `ZEROENTROPY_VALID_DIMS` allowlist (2560/1280/640/320/160/80/40),
`supportsZeroEntropyDimension(modelId)`, and `isValidZeroEntropyDim(dims)`.
Throws `AIConfigError` with paste-ready fix hint when zembed-1 is
configured with an invalid dim (most common: defaulting to 1536 from
DEFAULT_EMBEDDING_DIMENSIONS).

The 4th-arg is optional; existing call sites (1 production + N tests
across Voyage/OpenAI/DashScope/Zhipu/MiniMax) compile unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(ai/gateway): zeroEntropyCompatFetch + embedQuery + gateway.rerank()

Two seams land together because they share the same recipe + auth path.

zeroEntropyCompatFetch handles ZE's non-OpenAI-compatible wire shape:
  - URL rewrite: SDK's `${base_url}/embeddings` -> `${base_url}/models/embed`
  - Body inject: `input_type` (default 'document'; 'query' when threaded
    via providerOptions) + explicit `encoding_format: 'float'`
  - Response rewrite: `{results: [{embedding}]}` -> `{data: [{embedding,
    index}]}` so the AI SDK's openai-compat schema validates
  - `usage.prompt_tokens` injected from `total_tokens` (Voyage hit the
    same SDK schema requirement at :655)
  - Layer 1 (Content-Length) + Layer 2 (per-embedding size) OOM caps
    via tagged `ZeroEntropyResponseTooLargeError` (kept separate from
    `VoyageResponseTooLargeError` because the Voyage cap tests do
    structural source-text greps pinning the Voyage name)
  - Wired in `instantiateEmbedding()` via the existing
    `recipe.id === 'voyage' ? voyageCompatFetch : ...` ternary pattern

embedQuery(text) routes `inputType: 'query'` through dimsProviderOptions
for the search hot path. Companion to embed(texts) which now takes an
optional 2nd-arg inputType (defaults to undefined -> 'document' for
asymmetric providers).

gateway.rerank() is the new native HTTP path (no AI-SDK reranking
abstraction). Resolves the configured reranker model via
`getRerankerModel()` (new accessor), parses + asserts the model is in
the recipe's touchpoint.reranker.models allowlist (CDX2-F11:
assertTouchpoint does not enforce allowlists for openai-compatible
recipes — rerank() does it directly). Posts to
`${recipe.base_url}/models/rerank` with bearer auth. Returns
`RerankResult[]` sorted by `relevanceScore`. Errors classify into
`RerankError.reason: 'auth' | 'rate_limit' | 'network' | 'timeout' |
'payload_too_large' | 'unknown'`. 5s default timeout. Pre-flight payload
guard rejects bodies over `recipe.max_payload_bytes` BEFORE any HTTP
call so applyReranker can fail-open without burning a round-trip.
`_rerankTransport` + `__setRerankTransportForTests` mirror the embed
test seam.

`AIGatewayConfig.reranker_model` + isAvailable('reranker') branch +
configureGateway / reconfigureGatewayWithEngine extensions thread the
reranker model through the same state path as embedding/expansion/chat.
`applyResolveAuth` + `defaultResolveAuth` widen the touchpoint param to
include `'reranker'`. `KnownTouchpointKey` + `getTouchpoint()` in
model-resolver widen to cover `'reranker'`.

Pinned by:
- test/ai/embedQuery.test.ts (8): returns single Float32Array, threads
  input_type='query' for ZE, drops field for OpenAI text-3,
  back-compat: legacy embed() callers without 4th arg keep their
  previous Voyage no-input_type shape
- test/ai/rerank.test.ts (21): URL (F2 regression — no /v1/v1/), body
  shape, bearer header, response parsing, error classification across
  6 HTTP shapes, payload pre-flight (no transport call), allowlist
  enforcement
- test/ai/zeroentropy-compat-fetch.test.ts (14): structural source
  assertions for the shim that mirror test/voyage-response-cap.test.ts —
  URL rewrite path, body injection, response rewrite, usage.prompt_tokens
  injection, OOM caps Layer 1 + Layer 2 + instanceof rethrow,
  instantiateEmbedding wiring branch

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(search): applyReranker + rerank-failure audit + hybrid wire-in

src/core/search/rerank.ts — the call-site abstraction. Slices the top
`opts.topNIn` deduped candidates, sends to gateway.rerank(), reorders by
relevanceScore desc, appends the un-reranked tail in its original RRF
order (recall protection). Fail-open on every RerankError.reason: logs
via `logRerankFailure` and returns the input array unchanged. Stamps
`rerank_score` onto reordered items. `topNOut: null` is the explicit
"don't truncate" signal — distinct from `undefined` (fall through to
mode bundle); pin in test (CDX2-F16).

src/core/rerank-audit.ts — failure-only JSONL audit at
`~/.gbrain/audit/rerank-failures-YYYY-Www.jsonl` (ISO-week rotation;
mirrors `src/core/audit-slug-fallback.ts`). Exports `logRerankFailure`
+ `readRecentRerankFailures(days)`. **No `logRerankSuccess`** — CDX2-F22
deliberately drops success-event logging: writing once per tokenmax
search is hot-path I/O churn AND success events leak query
volume + timing into a local audit. The doctor check reads
`search.reranker.enabled` first so "no events in window" gets
interpreted correctly (disabled -> healthy by definition; enabled ->
healthy because nothing failed). Query text is SHA-256-prefix-hashed
(8 hex chars) for privacy. Honors `GBRAIN_AUDIT_DIR`.

src/core/search/hybrid.ts — slots `applyReranker` between
`dedupResults()` and `enforceTokenBudget()` in the main RRF path.
Resolution: per-call `opts.reranker` overrides; otherwise pulled from
the resolved mode bundle (tokenmax -> enabled, others -> disabled in
commit 5). Cache rows store final reranked results; the bumped
knobsHash (commit 5) ensures rows can't leak across reranker configs.

src/core/types.ts — adds `SearchOpts.reranker` as a structural type so
callers can pass per-call overrides; runtime type lives in
src/core/search/rerank.ts (avoids circular import).

Tests:
- test/search/rerank.test.ts (14): reorder, tail preserve, fail-open on
  every error class, topNOut null vs number, score stamping, empty +
  enabled=false pass-through
- test/rerank-audit.test.ts (10): JSONL round-trip, error_summary
  truncated to 200, corrupt rows skipped, missing dir -> [], ISO-week
  rotation walks current + previous week, no logRerankSuccess export
  (CDX2-F22 contract)
- test/search/hybrid-reranker-integration.test.ts (6): reranker fires
  when enabled, doesn't when disabled, reorders correctly, preserves
  tail, stamps rerank_score, fail-opens on rerankerFn throw — uses
  PGLite + stubbed embed transport, no API keys

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(search/mode): reranker mode-bundle fields + KNOBS_HASH_VERSION v=2

Extends `ModeBundle` with five reranker fields: `reranker_enabled`,
`reranker_model`, `reranker_top_n_in`, `reranker_top_n_out`,
`reranker_timeout_ms`. Per-mode defaults:

  - conservative -> enabled=false (cost-sensitive)
  - balanced     -> enabled=false (opt-in via search.reranker.enabled)
  - tokenmax     -> enabled=true  (the high-cost-tolerant tier; ~$0.0003/query)

Defaults model to `zeroentropyai:zerank-2`, topNIn=30, topNOut=null
(no truncate by default; preserves tokenmax's searchLimit=50 end-to-end
per CDX2-F16), timeout_ms=5000.

`SearchKeyOverrides` + `SearchPerCallOpts` + `resolveSearchMode.pick`
all extend to thread the new fields through the resolution chain
(per-call -> per-key config -> mode bundle -> default).

`loadOverridesFromConfig` adds parsers for the five new
`search.reranker.*` config keys. `top_n_out` parsing distinguishes
three input shapes (CDX2-F15):
  key absent           -> undefined (fall through to mode bundle)
  'null'|'none'|empty  -> explicit null (no truncate)
  positive integer     -> that number

`SEARCH_MODE_CONFIG_KEYS` extends so `gbrain search modes --reset`
clears the reranker overrides too.

**KNOBS_HASH_VERSION bumps 1 -> 2** (CDX1-F14). Five new entries
appended to `parts[]` (append-only convention CDX2-F13; reordering
existing fields would silently rebuild every existing cache row).
Includes `reranker_timeout_ms` so a 5s -> 100ms change invalidates
stale rows (CDX2-F14: more fail-opens = different search behavior).

Mid-rolling-deploy note (CDX2-F12): v=1 and v=2 processes produce
distinct cacheRowIds for the same (source_id, query_text). Expect a
temporary hit-rate dip + cache-row doubling for hot queries. Clears
naturally within `cache.ttl_seconds` (default 3600s).

src/commands/search.ts extends `KNOB_DESCRIPTIONS` with five new
entries so `gbrain search modes` renders them. test/search-mode.test.ts
extends the three bundle fixtures and bumps the KNOBS_HASH_VERSION
expectation to 2.

Pinned by test/search/knobs-hash-reranker.test.ts (13): each of the 5
reranker fields independently flips the hash, top_n_out=null renders
stable, append-only convention enforced via source-position assertion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(doctor): probeRerankerConfig + reranker_health check

`gbrain models doctor` gains two new probes:

- `probeRerankerConfig` (zero-network) validates that the configured
  reranker model resolves through the recipe registry, that the recipe
  declares a `reranker` touchpoint, and that the model is in
  `touchpoint.models[]`. Direct allowlist check here — assertTouchpoint
  does not enforce allowlists for openai-compatible recipes (CDX2-F11).
  Surfaces paste-ready `gbrain config set search.reranker.model
  <zerank-2|zerank-1|zerank-1-small>` fix hint.

- `probeRerankerReachability` (1-token-equivalent) sends a minimal
  `{query: "probe", documents: ["probe"]}` rerank to verify auth + URL.
  Failures classify via `classifyError` into auth/rate_limit/network/
  unknown. Skipped silently when reranker is unconfigured.

Also extends `probeEmbeddingConfig` with a `providerId === 'zeroentropyai'`
branch that catches the silent-1536-default bug class for zembed-1
configurations (same posture as the existing Voyage branch).

`ProbeResult.touchpoint` widens to include `'reranker_config'`.

`gbrain doctor` adds `checkRerankerHealth` to both the abbreviated
(doctorReportRemote) and full (runDoctor) check sets. Logic:

  1) Read `search.reranker.enabled` first. Disabled + no failures =>
     'reranker disabled'. Enabled + no failures => healthy.
  2) Walk last 7 days of ~/.gbrain/audit/rerank-failures-*.jsonl.
  3) ANY auth failure warns (config-time problem the probe should have
     caught — surface it).
  4) ANY payload_too_large failure warns (workload mismatch).
  5) Transient (network/timeout/rate_limit) warns at >=5 in window.
     Below that they're noise; reranker fails open anyway.

CDX2-F21 blind-spot fix: reading enabled state first means "no events"
gets interpreted correctly — never confuses "never-used" with "success
logging broken" (the latter is impossible because there is no success
logging by design, CDX2-F22).

Engine-agnostic; file-based + one config-key read.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(e2e): ZeroEntropy live API round-trip + wire into Tier 2 CI

test/e2e/zeroentropy-live.test.ts exercises the full stack against the
real api.zeroentropy.dev: embed (default 2560-dim + flexible 1280),
embedQuery (asymmetric query side), batch embed (3 distinct vectors),
rerank (3 docs sorted by relevance score, photosynthesis-relevant docs
beat the irrelevant cat doc), rerank with topN truncation.

Gated on `ZEROENTROPY_API_KEY`: every test prints `[skip]` and returns
early without assertions when the env var is unset, so fork PRs and
contributor machines without a ZE account stay green.

CI wire-up: `.github/workflows/e2e.yml` Tier 2 step adds
`test/e2e/zeroentropy-live.test.ts` to its `bun test` invocation and
exposes `ZEROENTROPY_API_KEY: ${{ secrets.ZEROENTROPY_API_KEY }}` to
the runner. The secret is set on garrytan/gbrain at the repo scope
(separately from this commit — set via `gh secret set` so the value
never lands in source).

Tier 1 stays mechanical (no API keys); Tier 2 is the natural home for
provider-live tests because it's already the API-keyed lane.

Cost: each full run fires ~6 small HTTP calls totaling well under a
cent at the published $0.025/1M-token rate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.33.3.0 feat: ZeroEntropy zembed-1 + zerank-2 reranker

Release notes for the ZeroEntropy support wave: zembed-1 embeddings
(flexible-dim 2560/1280/640/320/160/80/40, asymmetric input_type) and
zerank-2 cross-encoder reranking land as a new openai-compatible recipe
alongside OpenAI/Voyage. Reranker defaults ON for tokenmax mode, OFF
for conservative/balanced (~$0.0003/query at tokenmax topNIn=30; rounding
error vs the tier's $700/mo Opus pairing per the CLAUDE.md cost matrix).

Search now ends with `RRF -> dedup -> reranker -> token-budget` when
reranker is enabled; fails open to RRF order on any error class
(audit-logged at ~/.gbrain/audit/rerank-failures-*.jsonl).

`KNOBS_HASH_VERSION` bumps 1 -> 2 to fold reranker config into the
query_cache row key. Rolling-deploy operators should expect a temporary
cache hit-rate dip + cache-row doubling for hot queries (clears
naturally within `cache.ttl_seconds`, default 3600s).

Files in this commit are pure docs / version bump:
- VERSION + package.json bump to 0.33.3.0
- CHANGELOG.md release-summary entry with "How to take advantage" block
- CLAUDE.md Key Files annotations for the new recipe + rerank.ts +
  rerank-audit.ts + gateway extensions
- docs/ai-providers/zeroentropy.md one-pager (setup, knob reference,
  failure observability, troubleshooting table)
- skills/migrations/v0.33.3.md (purely informational: no required user
  action; reranker is opt-in everywhere, ZE embedding is opt-in)
- llms-full.txt regenerated to match CLAUDE.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-15 06:48:36 -07:00
Garry TanandClaude Opus 4.7 943e7b9dec v0.31.4.1 chore: align VERSION + package.json with #795 + mandate 4-segment versions (#815) 
* v0.31.4.1 chore: align VERSION/package.json with #795 + mandate MAJOR.MINOR.PATCH.MICRO

PR #795 (takes v2) landed on master with `v0.31.4` in its commit subject but
never bumped VERSION, package.json, or CHANGELOG.md. Master shipped at 0.31.3.

This corrective release:
- Bumps VERSION + package.json to 0.31.4.1 (the dot-suffix follow-up channel
  documented in CLAUDE.md, so the patch number doesn't churn to 0.31.5)
- Adds the v0.31.4.1 CHANGELOG entry covering takes v2 (lessons from a 100K-take
  production extraction), the auth-on-Postgres regression fix, and the new
  `gbrain eval takes-quality` CLI surface
- Updates CLAUDE.md to mandate `MAJOR.MINOR.PATCH.MICRO` for every new release.
  Historical 3-segment versions in git log + migration filenames stay valid;
  do not rewrite. Going forward only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: regenerate llms-full.txt for v0.31.4.1 doc edits

The build-llms regen-drift guard caught that llms-full.txt was stale relative
to the CHANGELOG + CLAUDE.md edits in the prior commit. Per CLAUDE.md the
bundle is auto-derived: bump VERSION/CHANGELOG/CLAUDE.md, then run
`bun run build:llms`. Did the second part now.

llms.txt unchanged (it's just the curated index). Only llms-full.txt picks
up the v0.31.4.1 CHANGELOG entry and the new "Version format is mandatory"
section in CLAUDE.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ci): exclude *.serial.test.ts from test-shard.sh hash buckets

Root cause of test (2) failing on the v0.31.4.1 PR (and on master since
#795 landed): CI's scripts/test-shard.sh hashed every test file into 4
shards via FNV-1a, INCLUDING *.serial.test.ts files. Serial files share
file-wide state (top-level mock.module, module singletons) that's
supposed to be quarantined by the .serial.test.ts naming + local
run-serial-tests.sh running them at --max-concurrency=1.

In CI the quarantine didn't apply. eval-takes-quality-runner.serial.test.ts
(new in #795) hashes into shard 2, where it calls:

  mock.module('../src/core/ai/gateway.ts', () => ({
    chat: async (opts) => { ... },
    configureGateway: () => undefined,
  }));

That replaces every export of gateway.ts at module-load time for the
WHOLE shard process. voyage-multimodal.test.ts also lives in shard 2
(both files happen to hash there), and it imports `embedMultimodal` from
gateway.ts. After the serial file loads, `embedMultimodal` is undefined
inside the shard process, and all 18 of voyage-multimodal's
embedMultimodal tests fail. Tests still passed locally because
run-unit-shard.sh excludes .serial files from its parallel pass.

Fix:
  - scripts/test-shard.sh: add `-not -name '*.serial.test.ts'` to the
    find expression so serial files no longer compete for shard buckets.
    Add --dry-run-list flag to mirror run-unit-shard.sh's interface so
    the regression test can introspect without spawning bun test.
  - .github/workflows/test.yml: add a `bun run test:serial` step that
    runs on shard 1 (which already runs `bun run verify`). Uses the
    existing scripts/run-serial-tests.sh which invokes bun test at
    --max-concurrency=1, matching local behavior.
  - test/scripts/test-shard.slow.test.ts: 4 regression cases that pin
    the contract (no serial files in any shard, no e2e files in any
    shard, plain files partitioned without overlap). .slow.test.ts
    because it shells out 4× with pure-bash FNV-1a hashing (~14s
    wallclock); excluded from the local fast loop, runs in CI via the
    same hash bucketing as other slow tests.
  - CLAUDE.md: update the CI vs local divergence section so this
    intentional asymmetry is documented going forward.

Build-llms drift in test (1) was fixed in the prior commit (c99a4af1).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: regenerate llms-full.txt for the CI-fix CLAUDE.md edits

The prior commit updated the "CI vs local: intentionally divergent file sets"
section in CLAUDE.md, which drifted llms-full.txt. Per CLAUDE.md the bundle
is auto-derived: edit CLAUDE.md, then run `bun run build:llms`. Did the
second part now.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-10 11:28:01 -07:00
Garry TanandClaude Opus 4.7 d97f159793 v0.26.4 test: parallel unit-test loop (12x speedup, failure-first logging) (#605) 
* test: parallel unit-test wrapper + failure-first logging (commit 1/8)

Lay foundation for v0.26.4 parallel test loop:

- scripts/run-unit-parallel.sh: spawns N shards (default min(8, cpu_count))
  via run-unit-shard.sh, captures per-shard logs, post-shard single-writer
  failure-log aggregation at .context/test-failures.log, 10s heartbeat to
  stderr, per-shard 600s timeout (gtimeout/timeout/bg-pid fallback chain),
  loud final banner with absolute path + tail-30 of failures, summary file
  for at-a-glance status. Single writer eliminates concurrent-write hazards
  on the failure log.
- scripts/run-serial-tests.sh: discovers *.serial.test.ts files (concurrency-
  unsafe by design), runs them with --max-concurrency=1. Invoked after the
  parallel pass.
- scripts/run-unit-shard.sh: now accepts --max-concurrency=N (forwarded to
  bun test); --dry-run-list moved into argv parsing alongside; excludes
  *.serial.test.ts in addition to *.slow.test.ts.
- bunfig.toml: trim stale comment about typecheck-chained timeout.
- .gitignore: add .context/ (Conductor workspace artifacts directory; the
  failure log + summary + per-shard logs all live here).

No package.json changes yet (commit 2). No test reorganization yet
(commits 4-7).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: split package.json scripts; bun run test = parallel fast loop (commit 2/8)

Per Codex Tension #4 (verify scope), distinguish three tiers cleanly:

- `bun run test` = fast loop, file-level parallel fan-out via the new wrapper
  (scripts/run-unit-parallel.sh). No pre-checks, no typecheck, no wasm
  compile in the hot path. ~15s of pre-test gates removed.
- `bun run verify` = CI's authoritative gate set: check:jsonb +
  check:progress + check:wasm + typecheck. Matches what
  .github/workflows/test.yml runs on shard 1, no scope drift. The 4
  checks not in CI (privacy, no-legacy-getconnection, trailing-newline,
  exports-count) move to `bun run check:all` for opt-in local use.
- `bun run test:full` = verify + parallel + slow + smart e2e (runs e2e
  only if DATABASE_URL is set; else loud skip notice to stderr per Open
  Item #7). The local equivalent of "everything CI runs."

Adds `bun run test:serial` for the *.serial.test.ts subset (concurrency-
unsafe files run with --max-concurrency=1).

Bumps VERSION + package.json to 0.26.4. Both move together per the CI
version-gate contract in CLAUDE.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: fix-wave for parallel wrapper + tighten privacy gate (commit 3/5)

Wave: makes the new wrapper actually green and tightens the CI gate it
exposed.

Wrapper bug fixes (scripts/run-unit-parallel.sh):
- grep_count helper: avoids the `grep -c | echo 0` double-output bug
  where 0 matches yields a 2-line "0\n0" string and breaks arithmetic.
- bun_summary_count helper: parses Bun's actual end-of-shard summary
  format (`N pass` / `N fail` / `N skip`), not the per-test markers
  (which are `✓` / `(fail)`, never `(pass)` / `(skip)`).
- Heartbeat now reads `^\s+✓` (Bun's per-test pass marker) for live
  progress mid-run; final summary still uses the summary-line counts
  for accuracy.

Privacy gate tightening:
- Move scripts/check-privacy.sh into `bun run verify` (was previously
  only in the now-removed `bun run test` chain). Without this, after
  commit 2 the privacy check ran in nothing automatic.
- .github/workflows/test.yml now calls `bun run verify` instead of
  inlining the gate list. Single source of truth for "what's the ship
  gate." This is what verify == CI was supposed to mean per Codex T#4.
- Pre-existing `Wintermute` references in src/core/mounts-cache.ts:6
  and :324 caught by the now-running gate; replaced with `your OpenClaw`
  per CLAUDE.md privacy rule (verify gate now passes on master HEAD).
- test/privacy-script-wired.test.ts updated: regression guard now
  asserts verify includes check:privacy AND that test.yml runs
  `bun run verify`, replacing the obsolete "test script includes
  check-privacy.sh" assertion.

Quarantine 2 cross-file-contention flakes:
- test/brain-registry.test.ts: 28 tests pass alone (41ms); 1 test
  ("empty/null/undefined id routes to host") fails when run alongside
  other files in the same shard. Renamed → *.serial.test.ts so it
  runs in scripts/run-serial-tests.sh's serial pass after the parallel
  pass completes.
- test/reconcile-links.test.ts: 6 tests pass alone (1s); a beforeEach
  hook times out (~896s) under cross-file contention. Same treatment.

Both flakes are bun-process-level shared-state leaks (PGLite singletons
or top-level imports). Fixing them properly is the v0.27.0+ intra-file
parallelism project (TODO P0 — see commit 5).

Measurement after this commit:
  bun run test = 94s (was 18 min sequential)
  3639 pass, 0 fail, 0 skip across 8 parallel shards + 34 serial tests
  Failure-log + heartbeat + summary all working

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: regression tests for parallel wrapper + serial-test contracts (commit 4/5)

Three regression suites pin the v0.26.4 contracts. Without these,
future refactors of the wrapper or shard scripts could silently
regress the work in commits 1-3.

test/scripts/run-unit-shard.test.ts (4 cases — gap b):
- Asserts the unit-shard `--dry-run-list` output excludes every
  *.slow.test.ts and *.serial.test.ts file, plus the test/e2e/ subtree.
- Catches a future `find` expression that drops one of the `-not -name`
  clauses and silently un-quarantines slow/serial files into the
  parallel pass.

test/scripts/serial-files.test.ts (3 cases — gap e):
- Every checked-in *.serial.test.ts (via `git ls-files`) is listed by
  scripts/run-serial-tests.sh's `--dry-run-list`.
- The script's source contains `bun test --max-concurrency=1` (the
  serial-pass guarantee that quarantined files don't run intra-file
  concurrent and reintroduce the contention they were quarantined for).
- Disjoint set: a file is never in both the unit-shard list AND the
  serial list — pins the carve-out contract.

test/scripts/run-unit-parallel.test.ts (6 cases — gaps a + d):
- Exit-code propagation (a): wrapper exits non-zero when ANY shard
  has a failing test; exits zero when all pass. The hardest contract
  to silently break in a fan-out wrapper (`for ... &; wait` returns
  the LAST child's status, not any failure's).
- Failure-log contract (d): on failure, .context/test-failures.log
  exists, is non-empty, contains the `--- shard N:` prefix and the
  failing test's describe text. Stderr banner contains the absolute
  log path. On success, the log is cleared (no stale content).
- Summary file format: `shard N/M: pass=X fail=Y skip=Z rc=W` per
  shard, machine-parseable for future tooling.

The wrapper test runs against a 4-file tempdir (3 pass + 1 fail) so
it executes in ~500ms; spawning the wrapper against the real test
suite would take ~90s and isn't worth the cost in a regression suite.

All 13 cases pass on first run.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(v0.26.4): testing tier docs + CHANGELOG + intra-file P0 TODO (commit 5/5)

Closes the v0.26.4 ship.

CLAUDE.md Testing section rewritten:
- New tier table: test (fast loop, 85s) / verify (CI gates, 12s) /
  test:full (everything local) / test:slow / test:serial / test:e2e /
  check:all. Each row names its scope, wallclock, and when to use.
- Intentional CI vs local divergence section: CI matrix (test-shard.sh,
  hash-bucketed, includes slow) vs local fast loop (run-unit-shard.sh,
  round-robin, excludes slow + serial). Codex correctly flagged that a
  parity test would always fail by design — this is the documentation
  that explains why.
- Failure-first logging contract: .context/test-failures.log format,
  stderr banner, summary file, wedge handling.
- File taxonomy: *.test.ts / *.slow.test.ts / *.serial.test.ts /
  test/e2e/. Names the two currently-quarantined files and points at the
  intra-file P0 TODO for the proper fix.

CHANGELOG.md `## [0.26.4]` entry per voice rules:
- Two-line headline: "bun run test finishes in 85 seconds. Was 18
  minutes." + failure-log directive.
- Lead paragraph names what shipped and why.
- Numbers-that-matter table: BEFORE / AFTER / Δ for wallclock, pre-test
  gates, failure visibility, shards, pipe-survival.
- "What this means for you" closing tied to the inner-loop user.
- "To take advantage of v0.26.4" block per the v0.13+ self-repair
  template (gbrain upgrade + contributor steps).
- Itemized changes by area (new scripts, script extensions, package.json
  tier split, CI tightening, failure-first logging, quarantine, regression
  tests, bunfig).
- "What did NOT ship" section names the intra-file project + E2E
  template-DB project as P0/P1 follow-ups with concrete acceptance
  criteria.
- Process section names the codex review + scope-correction loop
  honestly: "snapped back to ship today once empirical measurement showed
  Bun's --max-concurrency does nothing on tests not marked
  test.concurrent()."
- For-contributors note on portability + single-writer + fallback paths.

TODOS.md adds two P-rated entries:
- P0: intra-file parallelism via --concurrent flag. Sweep ~58 PGLite
  sites + ~40 env mutations + 2 mock.module sites. Target: bun run test
  < 30s. ~1-2 weeks. Detailed acceptance criteria. References Codex
  findings and plan-file rationale.
- P1: E2E parallelism via Postgres template databases. CREATE DATABASE
  TEMPLATE gbrain_template per test file. ~1-2 days.

llms.txt + llms-full.txt regenerated via `bun run build:llms` to absorb
the CLAUDE.md changes (per CLAUDE.md's "After any release ship that
touches the Key Files annotations in CLAUDE.md, run bun run build:llms"
rule). The build-llms regression test was firing in shard 7 of the
parallel pass — caught the drift, regeneration cleared it. Final
measurement after fix: 94s wallclock, 3652 pass, 0 fail across 8
parallel shards + 34 serial tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-03 20:16:15 -07:00
Garry TanClaude Opus 4.7Garry Tan
 		4fc1246606 v0.24.0: production-hardening pass on the skillify loop (#387) 
* feat: v0.19.0 — skillify loop + AGENTS.md compat + brain-first convention

This is the v0.19.0 release. The branch ships four new CLI commands, a
refactor to check-resolvable, and an expansion of the brain-first
convention for sub-agent tool discovery. The original commit message
described only the convention expansion, undercounting the scope by ~5x;
this amend captures the full release.

NEW COMMANDS

- gbrain skillify scaffold <name>     — 4 stub files + idempotent resolver row
- gbrain skillify check [path]        — 10-item post-task audit (promoted)
- gbrain skillpack list / install     — curated 25-skill bundle, atomic install
- gbrain skillpack diff <name>        — per-file diff preview
- gbrain routing-eval                 — dedicated CI verb for Check 5 fixtures

CHECK-RESOLVABLE REFACTOR

- Accepts AGENTS.md as a resolver file alongside RESOLVER.md, at either
  the skills directory or one level up (workspace root layout).
- Auto-derives the skill manifest by walking skills/*/SKILL.md when
  manifest.json is missing.
- Splits ResolvableReport into errors[] + warnings[] so advisory checks
  (filing audit, routing gaps, DRY violations) don't break CI by default.
- New --strict opt-in flag promotes warnings to exit 1.

BRAIN-FIRST CONVENTION

- skills/conventions/brain-first.md expanded from 5-step lookup guide to
  full sub-agent reference: tool inventory, lookup chain, score thresholds,
  authority hierarchy, sync rules, entity page conventions, sub-agent
  propagation rule.

PRODUCTION-READINESS HARDENING (this branch's review pass)

- routing-eval --llm: emits stderr placeholder notice + runs structural
  layer only. README, CHANGELOG, CLI help all rewritten consistently.
  Was a silent no-op against documented contract.
- skillpack installer: receipt comment in fence (cumulative-slugs="...")
  preserves single-skill-install accumulation while letting install --all
  prune removed bundle skills cleanly. Unknown rows preserved + stderr
  warning for the operating agent. Pre-v0.19 fences upgrade silently.
- skillify scaffold: resolver-row regex broadened to detect backticked,
  quoted, and bare path forms. No duplicate row on --force after the
  user normalizes formatting.
- scripts/check-privacy.sh: now wired into package.json test chain so
  the wintermute-ban rule is actually enforced. New regression test.
- E2E Tier 2 (LLM skills) promoted from schedule-only to required per-PR
  CI. Local Tier 1 + Tier 2 verified clean.
- Stale v0.17/v0.18 version labels rewritten across new files.

TESTS

- test/routing-eval-cli.test.ts: 4 cases covering --llm warn semantics
- test/privacy-script-wired.test.ts: regression guard for CI wiring
- test/skillpack-install.test.ts: 4 new cases for receipt + cumulative
  + unknown-row preserve+warn + pre-v0.19 upgrade path
- test/skillify-scaffold.test.ts: 4 new cases for broadened regex

VERIFICATION

- bun test: 2237 pass / 18 known PGLite-contention flakes (CI green;
  documented as P3 dev-experience in TODOS.md)
- bun run typecheck: clean
- bun run test:e2e: 18/19 files green (1 pre-existing flake on master,
  not caused by this branch — verified via git stash)
- llms.txt + llms-full.txt regenerated to match README + CHANGELOG

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: scrub banned fork name from public artifacts

The privacy guard wired into the test chain in this branch caught 5
pre-existing references to the banned OpenClaw fork name in CHANGELOG.md
(2x), skills/migrations/v0.19.0.md (1x), src/cli.ts (1x), and
src/commands/sync.ts (1x). All originated in master's v0.19.0 release
notes and migration doc when the privacy script existed but wasn't
wired into CI yet.

Replacements per CLAUDE.md privacy mapping:
- Origin-story copy (CHANGELOG layer narratives, code comments naming
  the production deployment that drove the feature) → "Garry's OpenClaw"
- Reader-facing migration step → "your OpenClaw"

No code semantics changed. Comments + headings only.

Verification: scripts/check-privacy.sh exits 0, full CI guard chain
green (privacy + jsonb + progress + wasm + typecheck).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump VERSION to 0.24.0 + new CHANGELOG entry

Bump branch version above master's v0.21.0 per CLAUDE.md
"CHANGELOG + VERSION are branch-scoped" rule. The new v0.24.0 entry at
the top of CHANGELOG covers what THIS branch adds vs master:

- routing-eval --llm honesty pass (4-surface contract drift fix)
- skillpack installer cumulative-receipt + unknown-row preserve+warn
  (the Codex-caught regression that would have shipped in master if
  the original v0.19.0 had landed without this branch's review pass)
- skillify scaffold resolver-row regex broadening (backtick + quoted
  + bare forms; idempotency contract preserved under hand-editing)
- 5 banned-name leaks scrubbed from public artifacts
- check-privacy.sh wired into CI test chain + regression guard test
- 7 stale v0.17/v0.18 version labels rewritten across 5 files
- Tier 2 (LLM-skills E2E) promoted from schedule-only to required per-PR

VERSION 0.21.0 → 0.24.0
package.json version field synced.
llms.txt + llms-full.txt regenerated (no content drift; sizes match).

Test suite: 62/62 green across the 5 test files this branch added or
extended (routing-eval-cli, privacy-script-wired, skillpack-install,
skillify-scaffold, build-llms).

CI guards: privacy + jsonb + progress + wasm + typecheck all clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: update project documentation for v0.24.0

Auto-discovered drift via /document-release after the v0.24.0 hardening
pass landed. All factual corrections clearly warranted by the diff.

CLAUDE.md:
- Skillpack installer: documented the cumulative-slugs receipt comment,
  install --all prune semantics, unknown-row preserve+warn behavior,
  and pre-v0.24 silent upgrade. Was previously vague about
  "tracks a skill manifest so install --update diffs cleanly" without
  explaining what the receipt is or why it matters.
- routing-eval: replaced the false claim that --llm "opts into a Haiku
  tie-break layer for CI." Now correctly describes the placeholder
  semantic landed in v0.24.0 (stderr notice + structural-only run).

README.md:
- Skillpack section: added one paragraph on the receipt comment + the
  user-visible stderr message for hand-added rows. Connects the safe
  rerun promise to the v0.24.0 implementation that actually enforces it.

CONTRIBUTING.md:
- Running tests section: now recommends `bun run test` (full CI guard
  chain + typecheck + tests) before pushing. Names each guard so new
  contributors understand what catches what. The privacy guard (newly
  wired in v0.24.0) is one of these — without `bun run test` you'd skip
  it locally and find out from CI.

llms-full.txt: regenerated to reflect CLAUDE.md changes.

Verification: full guard chain green locally (privacy + jsonb + progress
+ wasm + typecheck).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Garry Tan <garry@ycombinator.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-30 23:17:54 -07:00
Garry TanClaude Opus 4.7Wintermute
 		08746b06d2 v0.22.9 feat: structured error code summary for sync --skip-failed (#501) 
* feat: structured error code summary for sync --skip-failed (#500)

When sync encounters per-file failures, the blocked/skip-failed messages
now include a breakdown by error code (SLUG_MISMATCH, YAML_PARSE, etc.)
instead of just a raw count. This makes it immediately obvious *why*
files failed without requiring manual investigation.

Changes:
- Add classifyErrorCode() — maps error messages to ParseValidationCode
- Add summarizeFailuresByCode() — groups failures into sorted code summary
- SyncFailure now carries a 'code' field (backfilled on acknowledge)
- acknowledgeSyncFailures() returns AcknowledgeResult {count, summary}
- sync blocked + skip-failed messages show code breakdown
- doctor sync_failures check shows code breakdown for both unacked and historical
- 12 new tests for classifyErrorCode, summarizeFailuresByCode, and structured returns

Before:
  Sync blocked: 2688 file(s) failed to parse.

After:
  Sync blocked: 2688 file(s) failed to parse:
    SLUG_MISMATCH: 2685
    YAML_DUPLICATE_KEY: 3

Closes #500

* fix: eng-review fixes for sync error-code classification

- Reorder classifyErrorCode() so DB-layer errors (DB_DUPLICATE_KEY,
  STATEMENT_TIMEOUT) check BEFORE YAML patterns. Postgres "duplicate key
  value violates unique constraint" no longer mislabels as YAML_DUPLICATE_KEY.
- Rewrite MISSING_OPEN/MISSING_CLOSE/EMPTY_FRONTMATTER/NULL_BYTES/NESTED_QUOTES
  regexes to match the canonical messages emitted by collectValidationErrors()
  in src/core/markdown.ts. Previous patterns (e.g. /missing.*open/i) never
  fired because the upstream throw site emits prose ("File is empty...",
  "No closing --- delimiter found"), not the code name.
- Extract formatCodeBreakdown() helper that accepts either raw failures or
  pre-summarized {code, count}[] input. Replaces 3 duplicate inline builders
  in src/commands/sync.ts.
- 15 new tests (37/37 pass on test/sync-failures.test.ts):
  - DB vs YAML duplicate-key disambiguation (3 cases)
  - Canonical-message coverage for the 5 frontmatter codes (7 cases)
  - acknowledgeSyncFailures() legacy-entry backfill branch (2 cases)
  - formatCodeBreakdown() dual-input shape (3 cases)
- TODOS.md: file 3 follow-ups (P2 plumb structured ParseValidationCode;
  P0-at-ship CHANGELOG migration note for AcknowledgeResult; P3 concurrent-
  safe ack of sync-failures.jsonl).

Eng-review plan: ~/.claude/plans/then-codex-synchronous-toucan.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.22.9)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: 16-core runner + 4-way matrix shard for test job

The unit test suite ran 22m17s on ubuntu-latest (2-core/7GB) because:
- 187 test files run with bun test parallelism bounded by core count
- 23 of those files spin up a fresh PGLiteEngine + initSchema in beforeEach,
  paying ~22s WASM cold-start per test on the small runner

This commit fixes the runner side:
- runs-on: ubuntu-latest-16-cores (16 vCPU / 64 GB RAM)
- strategy.matrix.shard splits 4 parallel jobs, each running ~40 of 158 unit
  test files. Single-file wall-time floor is ~3 min after the test refactor,
  so 4 shards × 16 cores hits the floor quickly without wasting cores past it.
- pre-test gates (typecheck, check-jsonb, check-progress, check-wasm) only run
  on shard 1 — they're not test files and don't benefit from sharding.

scripts/test-shard.sh partitions test files by stable FNV-1a hash mod N. Same
file always lands in the same shard, so retries are reproducible. Pure shell,
portable to bash 3.2 (macOS) and bash 5.x (CI). Excludes test/e2e/ which runs
via bun run test:e2e separately and needs DATABASE_URL.

Also: ignore .claude/ harness state files (scheduled_tasks.lock etc) instead
of just .claude/skills/.

Cost: ~$0.19/run vs $0 (public repo, default runner is free). At 50 PRs/month
that's ~$10/month for ~5x faster CI.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: refactor top-3 PGLite-heavy files to share one engine per file

Three test files were spinning up a fresh PGLiteEngine + connect + initSchema
in beforeEach. PGLite WASM cold-start is ~22s on the small CI runner; doing
this per test multiplied wall-time across the suite. The 3 files alone
accounted for ~6.5 min of the 22m CI run (177s + 132s + 87s).

Refactor: move PGLite setup to beforeAll (one engine per file), wipe data
in beforeEach via the new test/helpers/reset-pglite.ts helper.

The reset helper:
- TRUNCATEs every public table CASCADE, including sources (so tests that
  register their own sources don't leak rows into the next test).
- Re-seeds the default source row that pages.source_id's DEFAULT FKs against.
  Without this, the next page insert would fail FK validation.
- Preserves schema_version so migration helpers don't think the brain is on v0.

Files refactored:
- test/extract-incremental.test.ts (8 tests, was 177s on CI)
- test/brain-writer.test.ts (16 tests; only the scanBrainSources block uses
  PGLite, was 132s on CI)
- test/sync.test.ts (37 tests; only the performSync dry-run block uses PGLite,
  was 87s on CI)

All 61 tests still pass locally. The remaining 20 PGLite-heavy files use the
same beforeEach anti-pattern; this commit only refactors the proven worst
offenders. Sweep the rest in a follow-up if CI numbers indicate it's worth it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: fall back to ubuntu-latest for matrix shard

The ubuntu-latest-16-cores label requires a provisioned larger-runner pool in
repo/org settings. Without that setup, jobs queue indefinitely waiting for a
runner that doesn't exist (verified: 4 shards stuck in 'queued' status with
empty runner_name for 5+ min).

Drop back to the default 2-core ubuntu-latest. The 4-way matrix shard still
delivers ~5-6x speedup via parallelism alone — 4 jobs running in parallel,
each handling ~40 of 158 unit test files. Cost stays $0 (default runner is
free for public repos).

If we ever provision a larger-runner pool, flip this label back to
ubuntu-latest-16-cores. The matrix + sharder will use the bigger boxes
unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Wintermute <wintermute@garrytan.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 10:12:10 -07:00
Garry TanandClaude Opus 4.7 8b3c24c891 v0.20.0 feat: extract BrainBench to sibling gbrain-evals repo (#195) 
* fix(link-extraction): v0.10.5 drive works_at + advises accuracy on rich prose

Extends inferLinkType patterns to cover rich-prose phrasings that miss with
v0.10.4 regexes. Targets the residuals called out in TODOS.md: works_at at
58% type accuracy, advises at 41%.

WORKS_AT_RE additions:
- Rank-prefixed: "senior engineer at", "staff engineer at", "principal/lead"
- Discipline-prefixed: "backend/frontend/full-stack/ML/data/security engineer at"
- Possessive time: "his/her/their/my time at"
- Leadership beyond "leads engineering": "heads up X at", "manages engineering at",
  "runs product at", "leads the [team] at"
- Role nouns: "role at", "position at", "tenure as", "stint as"
- Promotion patterns: "promoted to staff/senior/principal at"

ADVISES_RE additions:
- Advisory capacity: "in an advisory capacity", "advisory engagement/partnership/contract"
- "as an advisor": "joined as an advisor", "serves as technical advisor"
- Prefixed advisor nouns: "strategic/technical/security/product/industry advisor to|at"
- Consulting: "consults for", "consulting role at|with"

New EMPLOYEE_ROLE_RE page-level prior: fires when the page describes the subject
as an employee (senior/staff/principal engineer, director, VP, CTO/CEO/CFO) at
some company. Biases outbound company refs toward works_at when per-edge context
is possessive or narrative without an explicit work verb. Scoped to person -> company
links only. Precedence: investor > advisor > employee (investors often hold board
seats which would otherwise mis-classify as advise/works_at).

ADVISOR_ROLE_RE broadened from "full-time/professional/advises multiple" to catch
any page that self-identifies the subject as an advisor ("is an advisor",
"serves as advisor", possessive "her advisory work/role/engagement").

Tests: 65 pass (16 new v0.10.5 coverage tests + 4 regression guards against
v0.10.4 tightenings). Templated benchmark still 88.9% type_accuracy (10/10 on
works_at and advises). Rich-prose measurement requires the multi-axis report
upgrade (next commit) to validate retroactively.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(eval): type-accuracy runner on rich-prose corpus + wire into all.ts

New Category 2 in BrainBench: per-link-type accuracy measured directly on the
240-page rich-prose world-v1 corpus. Distinct from Cat 1's retrieval metrics,
this measures whether inferLinkType() correctly classifies extracted edges
when the prose varies (the 58% works_at and 41% advises residuals that v0.10.5
regexes targeted).

How it works:
  1. Loads all pages from eval/data/world-v1/
  2. Derives GOLD expected edges from each page's _facts metadata
     (founders → founded, investors → invested_in, advisors → advises,
      employees → works_at, attendees → attended, primary_affiliation +
      role drives person-page outbound type)
  3. Runs extractPageLinks() on each page → INFERRED edges
  4. Per (from, to) pair, compares inferred type vs gold type
  5. Emits per-link-type table: correct / mistyped / missed / spurious +
     type accuracy + recall + precision + strict F1 (triple match)
  6. Full confusion matrix rows=gold, cols=inferred

v0.10.5 validation on 240-page corpus (up from pre-v0.10.5 baselines):
  - works_at:    58%  → 100.0%   (+42 pts) — 10/10 correct, 0 mistyped
  - advises:     41%  → 88.2%    (+47 pts) — 15/17 correct
  - attended:    —    → 100.0%   131/134 recall
  - founded:    100%  → 100.0%   40/40
  - invested_in: 89%  → 92.0%    69/75
  - Overall:    88.5% → 95.7%    type accuracy (conditional on edge found)

Strict F1 overall: 53.7%. Lower because the _facts-based gold set only
captures core relationships; rich prose extracts many peripheral mentions
(190 spurious "mentions" edges) that aren't bugs but are correctly-typed
prose references without a _facts counterpart. Spurious counts are signal
for future type-precision tuning, not failure.

Wired into eval/runner/all.ts as Cat 2 so every full benchmark run includes
the rich-prose type accuracy table alongside retrieval metrics.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(eval): Phase 2 adapter interface + EXT-1 ripgrep+BM25 baseline

Phase 2 credibility unlock: BrainBench now compares gbrain to external
baselines on the same corpus and queries. Transforms the benchmark from
internal ablation ("gbrain-graph beats gbrain-grep") to category comparison
("gbrain-graph beats classic BM25 by 32 pts P@5"). This is the #1 fix
from the 4-review arc — addresses Codex's core critique that v1's
before/after was self-referential.

Added:
  eval/runner/types.ts                      — Adapter interface (v1.1 spec)
  eval/runner/adapters/ripgrep-bm25.ts      — EXT-1 classic IR baseline
  eval/runner/adapters/ripgrep-bm25.test.ts — 11 unit tests, all pass
  eval/runner/multi-adapter.ts              — side-by-side scorer

Adapter interface (eng pass 2 spec):
  - Thin 3-method Strategy: init(rawPages, config), query(q, state), snapshot(state)
  - BrainState is opaque to runner (never inspected)
  - Raw pages passed in-memory; gold/ never crosses adapter boundary
    (structural ingestion-boundary enforcement)
  - PoisonDisposition enum reserved for future poison-resistance scoring

EXT-1 ripgrep+BM25:
  - Classic Lucene-variant IDF + k1/b tuned at standard 1.5/0.75
  - Title tokens double-weighted for entity-page slug-match bias
  - Stopword filter, alphanumeric tokenization, stable lexicographic tie-break
  - Pure in-memory inverted index — no external deps, ~100 LOC core

First side-by-side results on 240-page rich-prose corpus, 145 relational queries:

| Adapter       | P@5    | R@5    | Correct top-5 |
|---------------|--------|--------|---------------|
| gbrain-after  | 49.1%  | 97.9%  | 248/261       |
| ripgrep-bm25  | 17.1%  | 62.4%  | 124/261       |
| Delta         | +32.0  | +35.5  | +124          |

gbrain-after is the hybrid graph+grep config from PR #188. Ripgrep+BM25 is
a genuinely strong classic-IR baseline (BM25 is what Lucene/Elasticsearch
ship). gbrain's ~+32-point lead on relational queries reflects real work
by the knowledge graph layer: typed links + traversePaths surface the
correct answers in top-K that BM25 only pulls in via partial-text overlap.

Next in Phase 2: EXT-2 vector-only RAG + EXT-3 hybrid-without-graph
adapters. Both plug into the same Adapter interface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(eval): Phase 2 EXT-2 vector-only RAG adapter

Second external baseline for BrainBench. Pure cosine-similarity ranking
using the SAME text-embedding-3-large model gbrain uses internally —
apples-to-apples on the embedding layer so any gbrain lead reflects the
graph + hybrid fusion, not a better embedder.

Files:
  eval/runner/adapters/vector-only.ts      ~130 LOC
  eval/runner/adapters/vector-only.test.ts 6 unit tests (cosine math)

Design:
  - One vector per page (title + compiled_truth + timeline, capped 8K chars).
  - No chunking (intentional; chunked vector RAG would be EXT-2b later).
  - No keyword fallback (that's EXT-3 hybrid-without-graph).
  - Embeddings in batches of 50 via existing src/core/embedding.ts (retry+backoff).
  - Cost on 240 pages: ~$0.02/run.

Three-adapter side-by-side on 240-page rich-prose corpus, 145 relational queries:

| Adapter       | P@5    | R@5    | Correct top-5 |
|---------------|--------|--------|---------------|
| gbrain-after  | 49.1%  | 97.9%  | 248/261       |
| ripgrep-bm25  | 17.1%  | 62.4%  | 124/261       |
| vector-only   | 10.8%  | 40.7%  |  78/261       |

Interesting finding: vector-only scores WORSE than BM25 on relational queries
like "Who invested in X?" — exact entity match matters more than semantic
similarity for these templates. BM25 nails the entity-name term; vector-only
returns topically-similar-but-not-mentioning pages. This is the known failure
mode of pure-vector RAG on precise relational/identity queries. Real-world
vector RAG systems always add keyword fallback; EXT-3 (hybrid-without-graph)
will be that fairer comparator.

gbrain's lead widens in vector-only comparison: +38.4 pts P@5, +57.2 pts R@5.
The graph layer is doing the heavy lifting for relational traversal; pure
vector RAG can't express "traverse 'attended' edges from this meeting page."

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(eval): Phase 2 EXT-3 hybrid-without-graph adapter — graph isolated

Third and closest-to-gbrain external baseline. Runs gbrain's full hybrid
search (vector + keyword + RRF fusion + dedup) WITHOUT the knowledge-graph
layer. Same engine, same embedder, same chunking, same hybrid fusion —
only traversePaths + typed-link extraction turned off.

This is the decisive comparator for "does the knowledge graph do useful
work?" Same everything-else, only graph differs. Any lead gbrain-after has
over EXT-3 is 100% attributable to the graph layer.

Files:
  eval/runner/adapters/hybrid-nograph.ts   — ~110 LOC

Implementation:
  - New PGLiteEngine per run; auto_link set to 'false' (belt).
  - importFromContent() used instead of bare putPage() so chunks +
    embeddings get populated (hybridSearch needs them).
  - NO runExtract() call — typed links/timeline stay empty (suspenders).
  - hybridSearch(engine, q.text) answers every query. Aggregate chunks
    to page-level by best chunk score.

FOUR-adapter side-by-side on 240-page rich-prose corpus, 145 relational queries:

| Adapter         | P@5    | R@5    | Correct/Gold |
|-----------------|--------|--------|--------------|
| gbrain-after    | 49.1%  | 97.9%  | 248/261      |
| hybrid-nograph  | 17.8%  | 65.1%  | 129/261      |
| ripgrep-bm25    | 17.1%  | 62.4%  | 124/261      |
| vector-only     | 10.8%  | 40.7%  |  78/261      |

The headline delta nobody can hand-wave away:
  gbrain-after → hybrid-nograph  = +31.4 P@5, +32.9 R@5
  hybrid-nograph → ripgrep-bm25  = +0.7 P@5,  +2.7 R@5

Hybrid search (vector+keyword+RRF) over pure BM25 gains ~1 point. The
knowledge graph layer over hybrid gains ~31 points. The graph is doing
the work; adding it to a retrieval stack is what actually moves the needle
on relational queries. The vector/keyword/BM25 debate is a footnote.

Timing: hybrid-nograph init is ~2 min (embeds 240 pages once); query loop
is fast. gbrain-after is ~1.5s total because traversePaths doesn't need
embeddings. Runs at ~$0.02 Opus-equivalent in embedding cost.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(eval): Phase 2 query validator + Tier 5 Fuzzy + Tier 5.5 synthetic + N=5 tolerance bands

Closes multiple Phase 2 items in one commit since they form a cohesive
package: query schema enforcement + new query tiers + per-query-set
statistical rigor.

Added:
  eval/runner/queries/validator.ts               — hand-rolled Query schema validator
  eval/runner/queries/validator.test.ts          — 24 unit tests, all pass
  eval/runner/queries/tier5-fuzzy.ts             — 30 hand-authored Tier 5 Fuzzy/Vibe queries
  eval/runner/queries/tier5_5-synthetic.ts       — 50 SYNTHETIC-labeled outsider-style queries (author: "synthetic-outsider-v1")
  eval/runner/queries/index.ts                   — aggregator + validateAll()

Modified:
  eval/runner/multi-adapter.ts                   — N=5 runs per adapter (BRAINBENCH_N override), page-order shuffle, mean±stddev reporting

Query validator (hand-rolled, no zod dep to match gbrain codebase style):
  - Temporal verb regex enforces as_of_date (per eng pass 2 spec):
    /\\b(is|was|were|current|now|at the time|during|as of|when did)\\b/i
  - Validates tier enum, expected_output_type enum, gold shape per type
  - gold.relevant must be non-empty slug[] for cited-source-pages queries
  - abstention requires gold.expected_abstention === true
  - externally-authored tier requires author field
  - batch validation catches duplicate IDs

Tier 5 Fuzzy/Vibe (30 queries, hand-authored):
  - Vague recall: "Someone who was a senior engineer at a biotech company..."
  - Trait-based: "The engineer who pushed back on microservices"
  - Cultural/epithet: "Who is known as a 'systems builder' in security?"
  - Abstention bait: "Which Layer 1 project did the crypto guy leave?" (prose
    mentions but never names; good systems abstain)
  - Addresses Codex's circularity critique — vague queries where graph-heavy
    systems shouldn't inherently win.

Tier 5.5 Synthetic Outsider (50 queries, AI-authored placeholder):
  - Clearly labeled author: "synthetic-outsider-v1"
  - Phrasing variety not in the 4 template families:
    * fragment style ("crypto founder Goldman Sachs background")
    * polite/natural ("Can you pull up what we have on...")
    * comparison ("What is the difference between X and Y?")
    * follow-up ("And who else advises Orbit Labs?")
    * typos/misspellings ("adam lopez bioinformatcis")
    * similarity ("Find me someone like Alice Davis...")
    * imperative ("Pull up Alice Davis")
  - Real Tier 5.5 from outside researchers supersedes synthetic via
    PRs to eval/external-authors/ (docs ship in follow-up commit).

N=5 tolerance bands:
  - Default N=5, override via BRAINBENCH_N env var (e.g. BRAINBENCH_N=1 for dev loops)
  - Per-run seeded Fisher-Yates shuffle of page ingest order (LCG seed = run_idx+1)
  - Surfaces order-dependent adapter bugs (tie-break-by-first-seen etc.)
  - Reports mean ± sample-stddev per metric
  - "stddev = 0" is honest signal that the adapter is deterministic, not a bug.
    LLM-judge metrics (future) will naturally produce non-zero stddev.

Validation: all 80 Tier 5 + 5.5 queries pass validateAll(). 24 validator
unit tests pass.

Next commit: world.html contributor explorer (Phase 3).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(eval): Phase 3 world.html explorer + eval:* CLI surface

Contributor DX magical moment. Static HTML explorer renders the full
canonical world (240 entities) as an explorable tree, opens in any browser,
zero install. Every string HTML-entity-encoded (XSS-safe — direct vuln
class per eng pass 2, confidence 9/10).

Added:
  eval/generators/world-html.ts         — renderer (~240 LOC; single-file
                                          HTML with inline CSS + minimal JS)
  eval/generators/world-html.test.ts    — 16 tests (XSS + rendering correctness)
  eval/cli/world-view.ts                — render + open in default browser
  eval/cli/query-validate.ts            — CLI wrapper for queries/validator
  eval/cli/query-new.ts                 — scaffold a query template

Modified:
  package.json                          — 7 new eval:* scripts
  .gitignore                            — ignore generated world.html

package.json scripts shipped:
  bun run test:eval                 all eval unit tests (57 pass)
  bun run eval:run                  full 4-adapter N=5 side-by-side
  bun run eval:run:dev              N=1 fast dev iteration
  bun run eval:world:view           render world.html + open in browser
  bun run eval:world:render         render only (CI-friendly, --no-open)
  bun run eval:query:validate       validate built-in T5+T5.5 (or a file path)
  bun run eval:query:new            scaffold a new Query JSON template
  bun run eval:type-accuracy        per-link-type accuracy report

XSS safety:
  escapeHtml() encodes the 5 critical chars (& < > " '). Tested directly
  with representative Opus-generated attacks:
    <img src=x onerror=alert('xss')>  → &lt;img src=x onerror=alert(&#39;xss&#39;)&gt;
    <script>fetch('/steal')</script>  → &lt;script&gt;fetch(&#39;/steal&#39;)&lt;/script&gt;
  Ledger metadata (generated_at, model) also escaped — covers the less
  obvious attack surface where Opus could emit tag-like content into the
  metadata file.

world.html structure:
  - Left rail: entities grouped by type with counts (companies, people,
    meetings, concepts), alphabetical within type
  - Right pane: per-entity cards with title + slug + compiled_truth +
    timeline + canonical _facts as collapsed JSON
  - URL fragment deep-links (#people/alice-chen)
  - Sticky rail on desktop; responsive stack on mobile
  - Vanilla JS for active-link highlighting on scroll (no framework)

Generated file: ~1MB for 240 entities (full prose). Gitignored; rebuild
with `bun run eval:world:view`. Regeneration is ~50ms.

Contributor TTHW (Tier 5.5 query authoring):
  1. bun run eval:world:view                         # see entities
  2. bun run eval:query:new --tier externally-authored --author "@me"
  3. edit template with real slug + query text
  4. bun run eval:query:validate path/to/file.json
  5. submit PR

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(eval): Phase 3 contributor docs + CI workflow for eval/ tests

Ships the contributor-onboarding surface promised in the plan. With this
commit, external researchers have a self-serve path from clone to PR in
under 5 minutes.

Added:
  eval/README.md                                — 5-minute quickstart,
                                                  directory map, methodology
                                                  one-pager, adapter scorecard
  eval/CONTRIBUTING.md                          — three contributor paths:
                                                    1. Write Tier 5.5 queries
                                                    2. Submit an external adapter
                                                    3. Reproduce a scorecard
  eval/RUNBOOK.md                               — operational troubleshooting:
                                                  generation failures, runner
                                                  failures, query validation,
                                                  world.html rendering, CI
  eval/CREDITS.md                               — contributor attribution
                                                  (synthetic-outsider-v1 labeled
                                                  as placeholder; real submissions
                                                  land here)
  .github/PULL_REQUEST_TEMPLATE/tier5-queries.md — structured PR template
                                                  for Tier 5.5 submissions
  .github/workflows/eval-tests.yml              — CI: validates queries,
                                                  runs all eval unit tests,
                                                  renders world.html on every PR
                                                  touching eval/** or
                                                  src/core/link-extraction.ts

CI scope (intentionally narrow):
  - Triggers on paths: eval/**, src/core/link-extraction.ts, src/core/search/**
  - Runs: bun run eval:query:validate (80 queries), test:eval (57 tests),
          eval:world:render (smoke-test the HTML renderer)
  - Pinned actions by commit SHA (matches existing .github/workflows/test.yml)
  - Zero API calls — all Opus/OpenAI paths stubbed or skipped in unit tests
  - Fast: ~30s total wall clock

Contributor TTHW (clone → first merged PR):
  - Path 1 (Tier 5.5 queries): ~5 min
  - Path 2 (external adapter): ~30 min for a simple adapter
  - Path 3 (reproduce scorecard): ~15 min wall clock (N=5 run)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(eval): teardown PGLite engines so bun run eval:run exits 0

The multi-adapter runner left PGLite engines alive after each run.
GbrainAfterAdapter and HybridNoGraphAdapter both instantiate a
PGLiteEngine in init() but never disconnect it; Bun's shutdown path
exits with code 99 when embedded-Postgres workers outlive main().

Added optional `teardown?(state)` to the Adapter interface, implemented
it on both engine-backed adapters, and call it from scoreOneRun after
the N=5 loop. ripgrep-bm25 and vector-only hold no DB resources and
don't need a teardown.

Verified: gbrain-after, hybrid-nograph, ripgrep-bm25, vector-only all
exit 0 at N=1. Full test:eval passes (57 tests). No metric change.

* docs(bench): 2026-04-19 multi-adapter scorecard

Reproducibility run of the 4-adapter side-by-side at commit b81373d
(branch garrytan/gbrain-evals). N=5, 240-page corpus, 145 relational
queries from world-v1.

Headline: gbrain-after 49.1% P@5 / 97.9% R@5. hybrid-nograph 17.8% /
65.1%. ripgrep-bm25 17.1% / 62.4%. vector-only 10.8% / 40.7%. All
adapters deterministic (stddev = 0 across the 5 runs per adapter).

Matches the scorecard in eval/README.md byte-for-byte for the three
deterministic adapters; hybrid-nograph matches within tolerance bands.

* docs(bench): 2026-04-19 gbrain v0.11.1 vs v0.12.1 regression comparison

Runs the same eval harness against two gbrain src/ trees on the same
240-page corpus and 145 queries. Patches the v0.11 copy's gbrain-after
adapter to use getLinks/getBacklinks (v0.11 has no traversePaths)
with identical direction+linkType semantics.

gbrain-after P@5 22.1% -> 49.1% (+27 pts); R@5 54.6% -> 97.9% (+43
pts); correct-in-top-5 99 -> 248 (+149). hybrid-nograph flat at 17.8%
/ 65.1% on both (v0.12 didn't touch hybridSearch / chunking).

Driver is extraction quality, not graph presence: v0.12 emits 499
typed links (v0.11: 136, x3.7) and 2,208 timeline entries (v0.11: 27,
x82) on the same 240 pages. Sharpens the April-18 "graph layer does
the work" claim -- on v0.11 that architecture only beat hybrid-nograph
by 4.3 points; the 31-point lead in the multi-adapter scorecard comes
from graph + high-quality extract in combination.

* feat(eval): BrainBench v1 portable JSON schemas + gold templates

Adds the v1→v2 contract boundary for BrainBench. 6 JSON schemas at
eval/schemas/ pin the shape of every artifact a stack must emit to be
scorable: corpus-manifest, public-probe (PublicQuery with gold stripped),
tool-schema (12 read + 3 dry_run tools, 32K tool-output cap), transcript,
scorecard (N ∈ {1, 5, 10}), evidence-contract (structured judge input).

8 gold file templates at eval/data/gold/ scaffold the sealed qrels,
contradictions, poison items, and citation labels. Empty-but-valid
skeletons; Day 3b fills them with real content once the amara-life-v1
corpus generates.

48 tests validate schema syntax, $schema/$id/title/type headers,
round-trip stability, and cross-schema coherence (new Page types in
manifest enum, tool counts, token cap, N enum).

When v2 ports to Python + Inspect AI + Docker, these schemas are the
boundary. Same fixtures, same tool contracts, zero rework.

* feat(eval): amara-life-v1 skeleton + Page.type enum for email/slack/cal/note

Deterministic procedural generator for the twin-amara-lite fictional-life
corpus (BrainBench v1 Cat 5/8/9/11 target). 15 contacts picked from
world-v1, 50 emails + 300 Slack messages across 4 channels + 20 calendar
events + 8 meeting transcripts + 40 first-person notes. Mulberry32 PRNG
gives byte-identical output under reseed.

Plants 10 contradictions + 5 stale facts + 5 poison items + 3 implicit
preferences at deterministic positions. Fixture_ids are unique across the
corpus so gold/contradictions.json + gold/poison.json + gold/implicit-
preferences.json can cross-reference by stable ID.

PageType extended in both src/core/types.ts and eval/runner/types.ts to
include email | slack | calendar-event | note (+ meeting on the production
side). src/core/markdown.ts inferType() heuristics updated for the new
one-slash slug prefixes (emails/em-NNNN, slack/sl-NNNN, cal/evt-NNNN,
notes/YYYY-MM-DD-topic, meeting/mtg-NNNN).

17 tests cover counts (50/300/20/8/40), perturbation counts (exact
10/5/5/3), seed determinism + divergence, slug regex conformance (matches
eval/runner/queries/validator.ts:131 one-slash rule), unique fixture_ids,
amara-in-every-email invariant, calendar dtstart < dtend, and Amara-is-
attendee on every meeting.

* feat(eval): amara-life-gen.ts with structured cache key + $20 cost gate

Opus prose expansion of the amara-life-v1 skeleton. Per-item structured
cache key = sha256({schema_version, template_id, template_hash, model_id,
model_params, seed, item_spec_hash}). Prompt-template tweak changes
template_hash; only those items regenerate. Schema bump changes
schema_version; everything invalidates cleanly. Interrupted runs resume
from the last cached item; zero re-spend.

Cost-gated at $20 hard-stop with Anthropic input/output pricing tracking.
Dry-run mode (--dry-run) executes the full pipeline with stub bodies for
smoke-testing the I/O layout without LLM spend. --max N caps items per
type for debugging. --force ignores cache.

Writes per-format outputs under eval/data/amara-life-v1/:
  inbox/emails.jsonl (one email per line with body_text appended)
  slack/messages.jsonl (one message per line with text appended)
  calendar.ics (RFC-5545 VEVENT format, templated — no LLM)
  meetings/<id>.md (transcript with YAML frontmatter)
  notes/<YYYY-MM-DD-topic>.md (first-person journal)
  docs/*.md (6 reference docs, templated — no LLM)
  corpus-manifest.json (per eval/schemas/corpus-manifest.schema.json,
    including per-item content_sha256 and generator_cache_key)

Perturbation hints (contradiction, stale-fact, poison, implicit-
preference) flow through the prompt so Opus weaves the specific claim
into each item's body. Poison items are hand-crafted to include
paraphrased prompt-injection attempts (not literal 'IGNORE ALL
PREVIOUS' — defense is the structured-evidence judge contract at
Day 5, not regex redaction).

New package.json scripts:
  eval:generate-amara-life       # real run (~$12 Opus estimated)
  eval:generate-amara-life:dry   # smoke test, zero spend

test:eval extended to include test/eval/. 10 cache-key tests cover
determinism, invalidation across every field of the key, canonical JSON
stability under object-key reorder, and per-skeleton-item spec-hash
uniqueness (50 distinct hashes for 50 distinct emails).

* chore: bump version and changelog (v0.15.0)

Resets package.json from stale 0.13.1 to 0.15.0 (matches VERSION).
v0.14.0 shipped with the stale package.json version; this sync catches
that up and moves to v0.15.0 in one step.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update CLAUDE.md + README + eval/README for v0.15.0 BrainBench

CLAUDE.md: adds a full BrainBench section to the Key Files list — 14 new
entries covering eval/README.md, multi-adapter.ts, types.ts (with new
PublicPage/PublicQuery), adapters/, queries/, type-accuracy.ts,
adversarial.ts, all.ts, world.ts/gen.ts, world-html.ts, amara-life.ts,
amara-life-gen.ts, schemas/, data/world-v1/, data/gold/,
data/amara-life-v1/, docs/benchmarks/, and test/eval/. Adds 3 new
test/eval/ lines to the unit-tests catalog.

eval/README.md: file tree updated to reflect v0.15 additions —
data/amara-life-v1/, data/gold/, schemas/, generators/amara-life.ts +
amara-life-gen.ts, runner/all.ts + adversarial.ts.

README.md: updates hero benchmark numbers (L7 intro + L353 mid-page)
from v0.10.5 PR #188 numbers (R@5 83→95, P@5 39→45) to current v0.12.1
4-adapter numbers (P@5 49.1% · R@5 97.9% · +31.4 pts vs hybrid-nograph).
Adds the v0.11→v0.12 regression comparison as the secondary reference.
Deeper-section tables (L422+) labeled "BrainBench v1 (PR #188)" are
preserved as historical data.

CHANGELOG is untouched — /ship already wrote the v0.15.0 entry.
TODOS.md is untouched — Cat 5/6/8/9/11 remain open (only foundations
shipped in v0.15.0; Cat runners ship in v1 Complete follow-ups).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(eval): Day 4 — pdf-parse + flight-recorder + tool-bridge (dry_run + expand:false)

Three infrastructure modules for BrainBench v1 Complete Cats 5/8/9/11.

**eval/runner/loaders/pdf.ts** — Thin pdf-parse wrapper. Lazy import keeps
pdf-parse out of the module-load path (avoids library debug-mode side
effects). Size cap (50MB default), encryption detection, structured error
classes (PdfEncryptedError, PdfTooLargeError, PdfParseError). Only Cat 11
multimodal will import this; production bundle never sees pdf-parse.

**eval/runner/tool-bridge.ts** — Maps 12 read-only operations from
src/core/operations.ts to Anthropic tool definitions + adds 3 dry_run write
tools. Three structural invariants enforced:

  1. No hidden LLM calls. `operations.query` defaults expand=true which
     routes through expansion.ts → Haiku. Bridge strips `expand` from the
     query tool's input schema AND executor hard-sets expand:false. Zero
     nested Haiku calls in any agent trace.

  2. Mutating ops throw ForbiddenOpError. put_page, add_link, delete_page,
     etc. are rejected by name. Agents record intent via dry_run_put_page /
     dry_run_add_link / dry_run_add_timeline_entry which persist to the
     flight-recorder without mutating the engine. This is how Cat 8's
     back_link_compliance + citation_format metrics measure anything with
     a read-only tool surface.

  3. Poison tagged by the bridge, not the judge. Every tool result is
     scanned for slugs matching gold/poison.json fixtures. Matched
     fixture_ids flow into tool_call_summary.saw_poison_items for the
     structured-evidence judge contract. Judge never reads raw tool
     output — Section-3 defense against paraphrased prompt injections
     (poison payloads never reach the judge model at all).

32K-token cap (~128K chars) with "…[truncated]" suffix.

**eval/runner/recorder.ts** — Per-run flight-recorder bundle emitter. Full
6-artifact bundle (transcript.md, brain-export.json, entity-graph.json,
citations.json, scorecard.json, judge-notes.md) when the adapter provides
an AdapterExport; 3-artifact fallback (transcript + scorecard +
judge-notes) otherwise. Atomic writes via tmp+rename. Collision-safe:
duplicate directory names get incremental -2, -3 suffix. `safeStringify`
handles circular references without throwing and JSON-serializes
Float32Array embeddings.

**package.json:** adds pdf-parse@2.4.5 as a devDependency. Scoped to eval/
use only; production gbrain binary unaffected.

**Tests:** 63 new — 30 tool-bridge, 21 recorder, 12 pdf-loader. All pass.
Fake engine uses a Proxy with `__default__` fallback so poison-matching
tests don't have to mock the exact engine method name that each operation
calls (some route via searchKeyword, others via getPage — proxy handles
both uniformly).

Total eval suite now: 132 pass, 0 fail, 923 expect() calls.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(eval): Day 5 — agent adapter + judge with structured evidence contract

Two modules that together wire Cat 8 / Cat 9 / Cat 5 end-to-end scoring.

**eval/runner/judge.ts** — Haiku 4.5 via tool-use `score_answer`. Input is
the structured JudgeEvidence contract (fix #16 from the plan's codex
review): probe + final_answer_text + evidence_refs + tool_call_summary +
ground_truth_pages + rubric. Raw tool output NEVER reaches the judge —
that's the Section-3 defense against paraphrased prompt-injection payloads
in gold/poison.json.

Retry policy: one retry on malformed tool_use response. If the second
attempt is still malformed, score the probe as `judge_failed` (all scores
0, verdict=fail) so the run still completes.

Aggregation: weighted mean across rubric criteria. Canonical thresholds
(pass ≥3.5, partial 2.5-3.5, fail <2.5) — judge can propose a verdict but
the computed verdict from the weighted mean is what the scorecard records.
This prevents the model from inflating or deflating its own verdict.

Score values are clamped to 0-5 on parse even if the model returns out of
range. `assertNoRawToolOutput(evidence)` is a regression guard that
returns the list of forbidden fields (tool_result, raw_transcript, etc.)
if any leak into the evidence contract.

**eval/runner/adapters/claude-sonnet-with-tools.ts** — The agent adapter.
Implements `Adapter` interface minimally: `init()` spins up PGLite and
seeds it, `query()` throws because the adapter is Cat 8/9-only and emits
a final-answer text, not a RankedDoc[]. Retrieval scorecard stays at 4
adapters.

`runAgentLoop(probeId, text, state, config)` drives the multi-turn loop:
Sonnet → tool_use → tool-bridge.executeTool → tool_result → back to
Sonnet. Turn cap 10. max_tokens 1024. System prompt (brain-first iron
law, citation format, amara context) is cached via cache_control.
Exponential backoff on rate-limit errors (1s, 2s, 4s).

Emits a `Transcript` per eval/schemas/transcript.schema.json — consumed
directly by recorder.ts for the flight-recorder bundle.

`brain_first_ordering` classifies Cat 8's flagship metric: did the agent
call search/get_page BEFORE producing the final answer? The `no_brain_calls`
case (agent answers from general knowledge without ever hitting the brain)
is the compliance failure to surface.

ForbiddenOpError + UnknownToolError from the bridge are caught in the
agent loop and surfaced as tool_result with is_error=true — keeps the
loop going and preserves full audit trail for the judge.

**Tests (35 new):** judge (23) — happy path, retry, fallback, evidence
contract sanitization, rendered prompt does not contain raw tool_result
text, verdict thresholds, score clamping, weighted mean with mixed
weights, parseToolUse rejects malformed input. agent-adapter (12) —
Adapter.query() throws, init() seeds PGLite, end-to-end tool loop with
stubbed Sonnet, turn cap exhaustion, mutating-op rejection surfaces as
tool_result error, extractSlugs regex.

All 12 agent tests take ~23s because PGLite runs 13 schema migrations per
test; the alternative of shared-engine-across-tests was rejected so each
test is isolated.

Total eval suite now: 167 pass, 0 fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(eval): Day 6 — adversarial-injections + Cat 6 prose-scale + Cat 11 multi-modal

Three modules that together cover BrainBench v1 Cat 6 (prose-scale
extraction fidelity) and Cat 11 (multi-modal ingest fidelity).

**eval/runner/adversarial-injections.ts** — 6 deterministic content
transforms shared by Cat 10 (adversarial.ts, 22 hand-crafted cases) and
Cat 6 (prose-scale variants). Each injection produces a modified content
string + a structured GoldDelta describing what the extractor MUST and
MUST NOT produce. Kinds:
  - code_fence_leak — fake [X](people/fake) inside ``` fence, must NOT extract
  - inline_code_slug — `people/fake` in backticks, must NOT extract
  - substring_collision — "SamAI" near real `people/sam`, exactly one link
  - ambiguous_role — "works with" vs "works at", downgrade type to mentions
  - prose_only_mention — strip markdown link syntax, bare name → mentions only
  - multi_entity_sentence — pack 4+ entities into one clause, extract all

Mulberry32 PRNG keeps variant generation deterministic under fixed seed.
Codex flagged the original plan's wording ("extract injection engine from
adversarial.ts") as overstated — adversarial.ts is a static case list,
not a reusable engine. This module is NEW code.

**eval/runner/cat6-prose-scale.ts** — Runner. Loads world-v1, applies all
6 injection kinds to sampled base pages (default 50 variants per kind ×
6 kinds = 300 variants), runs extractPageLinks on each, compares to gold
delta. Emits per-kind + overall metrics (precision, recall, F1,
code_fence_leak_rate, substring_fp_rate, pages_with_links_coverage,
mean_links_per_page). **v1 verdict is always "baseline_only"** — no
gating threshold per codex fix #9 (current extractor residuals make
>0.80 unreachable; v1 records a baseline, regression guard triggers on
drop below it).

**eval/runner/cat11-multimodal.ts** — PDF + HTML + audio runners.
Fixtures load from eval/data/multimodal/<modality>/fixtures.json
manifests; each modality skips gracefully when manifest missing or
(audio) when neither GROQ_API_KEY nor OPENAI_API_KEY is set. Metrics:
  - PDF: char-level similarity via Levenshtein + optional entity_recall
  - HTML: word-recall over normalized tokens (multiset semantics)
  - Audio: WER (word error rate) via Levenshtein on word sequences
Fixtures are NOT committed; a future eval:fetch-multimodal script will
download them hash-verified from public sources (arXiv CC-licensed
papers, Wikipedia CC-BY-SA, Common Voice CC0).

Injectable audio transcriber (`opts.transcribe`) means tests don't need
GROQ/OpenAI keys — stubbed transcriptions exercise the WER math path
directly.

**Tests (60 new):** adversarial-injections (19) — per-kind assertions +
dispatcher coverage + slug regex conformance; cat6 (12) — variant
determinism, scoreVariant shape, aggregate per-kind + overall metrics,
corpus resolver slug rules; cat11 (29) — charSimilarity / wordRecall /
wer math, htmlToText strips scripts + decodes entities, HTML modality
with real fixtures, audio modality gracefully skips without key + uses
stub transcriber correctly.

All 60 tests pass in 48ms + 41ms.
Total eval suite now: 227 pass, 0 fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(eval): Day 7 — Cat 5 provenance runner + structured classify_claim judge

**eval/runner/cat5-provenance.ts** — BrainBench Cat 5 scoring. Samples
claims from gbrain brain-export and classifies each against its source
material via a dedicated Haiku judge (classify_claim tool with a
three-label enum: supported | unsupported | over-generalized).

Separate from judge.ts by design: Cat 5 is a single three-way
classification per claim, not a weighted rubric. Rather than overload
judge.ts with a mode switch, Cat 5 has its own tool definition
(CLASSIFY_CLAIM_TOOL) and prompt. The retry-once pattern, $20 cost gate
semantics, and structured parsing are mirrored from judge.ts so failures
look the same across Cats.

Metric: `citation_accuracy` = fraction where predicted label equals
gold expected_label. Threshold (informational): >0.90 per design-doc
METRICS.md. v1 ships with `enableThreshold: false` so the verdict is
always baseline_only — we don't have hand-authored gold claims yet, and
codex flagged that threshold gating should wait until the amara-life-v1
corpus + gold file authoring lands in Day 3b.

runCat5 uses a bounded-concurrency worker pool (default 4) to respect
Haiku rate limits across 100+ claim batches. Evidence pages are looked
up by slug from a caller-provided pagesBySlug map — missing pages don't
crash, they just pass an empty source list to the judge (correct
behavior for genuinely unsupported claims).

**Tests (23):** classifyClaim happy/retry/fallback paths with stubbed
Haiku, aggregate accuracy math, threshold gating (pass/fail vs
baseline_only), runCat5 concurrency + missing-page handling,
renderClaimPrompt embeds claim + sources correctly, parseClassification
rejects invalid enum values + plain-text responses.

Total eval suite now: 250 pass, 0 fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(eval): Day 8 — Cat 8 skill compliance + Cat 9 end-to-end workflows

**eval/runner/cat8-skill-compliance.ts** — Deterministic, judge-free Cat 8
scoring. Replays inbound signals through the agent adapter (Day 5) and
extracts four iron-law metrics directly from the tool-bridge state:

  - brain_first_compliance: agent called search/get_page BEFORE producing
    its final answer. Non-compliance = hallucinating from general knowledge.
  - back_link_compliance: every dry_run_put_page intent has at least one
    markdown [Name](slug) back-link in its compiled_truth.
  - citation_format: timeline entries use canonical `- **YYYY-MM-DD** |
    Source — Summary`; long final answers cite at least one slug.
  - tier_escalation: simple probes use light tooling (≥1 brain call);
    complex probes require ≥2 brain calls or a dry_run write when
    expects_dry_run_write is set.

No judge call required — everything is computable from
`tool_bridge_state.made_dry_run_writes` + `count_by_tool` + final_answer
regex. Fast, deterministic, reproducible.

Bounded concurrency (p-limit style) worker pool at default 4 to keep
Sonnet rate limits comfortable across 100-probe batches.

**eval/runner/cat9-workflows.ts** — Rubric-graded Cat 9. 5 canonical
workflows (meeting_ingestion, email_to_brain, daily_task_prep, briefing,
sync) × ~10 scenarios each. Each scenario runs through the agent adapter,
then judge.ts scores the answer against a per-scenario rubric.

`buildEvidence(scenario, agentResult, pagesBySlug)` composes the
JudgeEvidence contract: resolves ground_truth_slugs to full
GroundTruthPage[] from a slug-map, pulls tool_call_summary directly from
tool_bridge_state (no raw tool_result content — Section-3 defense),
attaches rubric from the scenario.

Per-workflow rollup: each workflow gets its own pass_rate so the verdict
can fail one workflow without failing the whole Cat. Overall verdict
requires every populated workflow's pass_rate ≥ threshold (default 0.80)
when enableThreshold=true.

Both Cats default to verdict=baseline_only in v1 per codex fix #9: real
thresholds return after 10-probe Haiku-vs-hand-score calibration (κ > 0.7)
runs against the Day 3b amara-life-v1 corpus.

**Tests (23):** Cat 8 per-metric scorer unit tests covering every branch
(brain_first ordering, back-link compliance on mixed writes, long vs
short answer citation requirement, tier escalation for simple/complex/
writey probes, finalAnswerCiteCount dedups across syntaxes). Cat 9
buildEvidence contract shape — evidence_refs flow from agent, missing
slugs skip gracefully, no raw_transcript/tool_result leakage to judge.
Cat 9 runCat9 integration with stubbed agent + mixed-verdict judge
produces fractional pass rates correctly.

Total eval suite now: 273 pass, 0 fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(eval): Day 9 — sealed qrels via PublicPage + PublicQuery at adapter boundary

Codex fixes #1, #2, #3 from the plan's outside-voice review. Enforcement
shifts from SOFT-VIA-TYPE-COMMENT to SOFT-VIA-SANITIZED-OBJECT. Hard
enforcement via process isolation waits for BrainBench v2 Docker sandbox.

**eval/runner/types.ts** additions:
  - `PublicPage = Pick<Page, 'slug' | 'type' | 'title' | 'compiled_truth' |
    'timeline'>` — the exact 5 fields adapters should see. No _facts.
    No frontmatter (a known hiding spot for accidental gold leaks).
  - `sanitizePage(p: Page): PublicPage` — returns a NEW object with the 5
    fields only. Cannot be bypassed by `(page as any)._facts` because the
    field does not exist on the sanitized object.
  - `PublicQuery = Omit<Query, 'gold'>` — strips the gold field.
  - `sanitizeQuery(q: Query): PublicQuery` — enumerates public fields
    explicitly (not spread+delete) so no prototype weirdness leaves gold
    reachable.

**eval/runner/multi-adapter.ts** — scoreOneRun now calls sanitizePage /
sanitizeQuery before passing to adapter.init / adapter.query. The scorer
retains the full Query shape (including gold.relevant) for precision /
recall computation. Adapter signatures unchanged — the sealing is at the
OBJECT level, not the type level. This keeps existing adapters
(ripgrep-bm25, vector-only, hybrid-nograph, gbrain-after) binary-compatible.
Verified: no existing adapter reads q.gold or page._facts, so the change
is safe without further adapter updates.

**test/eval/sealed-qrels.test.ts** (17 tests):
  - sanitizePage strips _facts + frontmatter + arbitrary hidden keys
  - Output has exactly the 5 public keys (deep introspection)
  - Proxy tripwire simulates a malicious adapter: any access to _facts or
    gold throws `sealed-qrels violation`
  - sanitizeQuery retains optional fields (as_of_date, tags, author,
    acceptable_variants, known_failure_modes) but omits undefined ones
  - Honest documentation of the seal's limits: filesystem bypass and
    Proxy attacks would still work in v1; Docker isolation (v2) is the
    real enforcement

Every existing eval test still passes (273 before + 17 sealed-qrels = 290).

Total eval suite now: 290 pass, 0 fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(eval): Day 10 — all.ts rewrite + llm-budget + BrainBench N tiers

Final wiring of BrainBench v1 Complete. all.ts now orchestrates the full
Cat catalog (1-12) via a mix of subprocess dispatch (Cats 1, 2, 3, 4, 6,
7, 10, 11, 12 — standalone runners with CLI entry points) and
programmatic invocation (Cats 5, 8, 9 — require runtime inputs that
can't come via CLI flags). Subprocess Cats run concurrently under a
p-limit(2) bound to cap peak memory around ~800MB (two PGLite instances
at ~400MB each).

Cats 5/8/9 show as "programmatic" in the report with a one-line
reference to their `runCatN({...})` harness API. They're deliberately
skipped from the master runner because their inputs (claim catalog,
probe catalog, scenario catalog, pre-seeded agent state, evidence
pagesBySlug) are task-specific and assembled at the caller.

**eval/runner/all.ts** — rewritten:
  - CATEGORIES is a tagged union of SubprocessCategory | ProgrammaticCategory
  - runCatSubprocess spawns Bun with pipe'd stdout/stderr, 10-min timeout
    per Cat (124 exit + SIGTERM on timeout; no hung subprocesses)
  - runConcurrently is a bounded worker pool preserving input order
  - buildReport emits the full markdown with per-Cat elapsed times,
    migration-noise filter, and a separate programmatic-only section
  - Honors BRAINBENCH_N (1/5/10 for smoke/iteration/published),
    BRAINBENCH_CONCURRENCY (default 2),
    BRAINBENCH_LLM_CONCURRENCY (default 4, consumed by llm-budget)

**eval/runner/llm-budget.ts** — shared LLM rate-limit semaphore. A full
N=10 published scorecard makes ~900 Anthropic calls (150 Cat 8/9 probes
× N=10 + 100 Cat 5 claims × N=10). Without coordination, concurrent
adapters trigger 429s on per-minute limits.

  - LlmBudget class: acquireSlot/releaseSlot + withLlmSlot(fn) wrapper
    that releases on success AND throw (try/finally)
  - getDefaultLlmBudget() singleton reads BRAINBENCH_LLM_CONCURRENCY,
    falls back to 4 on missing/garbage values
  - capacity enforced ≥1 (rejects 0/negative)
  - Double-release is a no-op (guards against upstream double-call bugs)
  - Active + waiting counts exposed for observability / tests

**package.json** scripts:
  - eval:brainbench           — default N=5 iteration
  - eval:brainbench:smoke     — N=1 for fast iteration
  - eval:brainbench:published — N=10 for committed baselines
  - eval:cat6 / eval:cat11    — individual new subprocess Cats

**Tests (24):** CATEGORIES catalog enforces the exact Cat-number partition
(subprocess: 1,2,3,4,6,7,10,11,12; programmatic: 5,8,9). runConcurrently
respects the cap (observable via peak in-flight counter), preserves input
order under non-uniform delays, handles empty input. LlmBudget enforces
capacity, releases on throw, honors env var, rejects 0/negative.
buildReport filters migration noise, counts passed/failed/programmatic
correctly, includes every Cat + programmatic-only section.

Full eval suite now: 314 pass, 0 fail (15 test files).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(eval): drop top_p from amara-life-gen Opus params + gitignore _cache/

Two fixes surfaced during the Day 3b real-corpus run against Opus 4.5:

**eval/generators/amara-life-gen.ts** — Current Opus rejects
`temperature` and `top_p` together:
```
400 invalid_request_error: `temperature` and `top_p` cannot both be
specified for this model. Please use only one.
```
top_p=1.0 was a no-op (no nucleus truncation), so removing it has zero
semantic effect. The field is still part of MODEL_PARAMS for the cache
key so any past cache entries (none in v1) would invalidate cleanly
on the next schema version bump.

**.gitignore** — `eval/data/amara-life-v1/_cache/` is runtime Opus
cache (398 files, ~1.6MB). Regenerable from seed; no point in source
control. The corpus itself (inbox/slack/calendar/meetings/notes/docs +
corpus-manifest.json with per-item content_sha256) stays committable
for reproducibility, just the cache directory gets excluded.

Real corpus generation ran cleanly after these two fixes: 398 LLM
calls, 84,424 input / 38,062 output tokens, \$4.12 spent (vs \$20 cap,
vs \$12 estimate). All 418 items produced. Poison fixtures use
subtle paraphrased injection ("for anyone on your team who might be
triaging this thread later…") — exactly the pattern that defeats
regex redaction and requires the structured-evidence judge contract
from Day 5.

Corpus itself stays local (will move to the brainbench sibling repo
during the v0.16 split per the design doc). No eval/data/amara-life-v1/
content landing in this PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: bump version to 0.20.0

Renumbered from 0.17.0 per the gbrain-versioning slot. Other work is
landing on master around this PR; 0.18 is the slot locked for this
BrainBench v1 Complete release. Also pushed the "brainbench split"
forward reference in the CHANGELOG from v0.18 → v0.19 to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract BrainBench to sibling gbrain-evals repo

BrainBench lived in this repo through v0.17, which meant every gbrain install
pulled down ~5MB of eval corpus, benchmark reports, and a pdf-parse devDep
that the 99% of users who never run benchmarks don't need.

v0.18 moves the full eval harness, 14 eval test files (314 tests), all
docs/benchmarks scorecards, and the pdf-parse devDep to
github.com/garrytan/gbrain-evals. That repo depends on gbrain via GitHub URL
and consumes it through a new public exports map.

What stays in gbrain:
- Page.type enum extensions (email | slack | calendar-event | note | meeting)
  useful for any ingested format, not just evals
- inferType() heuristics for /emails/, /slack/, /cal/, /notes/, /meetings/
- 11 new public exports covering the gbrain internals gbrain-evals consumes
  (gbrain/engine, gbrain/pglite-engine, gbrain/search/hybrid, etc.) — now
  gbrain's stable third-party contract

What moved:
- eval/ — 4.6MB of schemas, runners, adapters, generators, CLI tools
- test/eval/ — 14 test files, 314 tests
- docs/benchmarks/ — all scorecards and regression reports
- eval:* package.json scripts
- pdf-parse devDep

Tests: 1760 pass, 0 fail, 174 skipped (E2E require DATABASE_URL).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Merge origin/master into garrytan/gbrain-evals

Master landed significant work since this branch was cut (v0.15.x → v0.16.x →
v0.17.0 gbrain dream + runCycle → v0.18.0 multi-source brains → v0.18.1 RLS
hardening). Bumped this branch's version from the claimed 0.18.0 to 0.19.0
because master already owns 0.18.x.

Conflicts resolved:
- VERSION: 0.19.0 (was 0.18.0 on HEAD vs 0.18.1 on master)
- package.json: 0.19.0, kept all 11 eval-facing exports, merged master's
  typescript devDep + postinstall script + test script (typecheck added)
- src/core/types.ts: union of both PageType additions. Master had added
  `meeting | note`; this branch added `email | slack | calendar-event`
  for inbox/chat/calendar ingest. Final enum carries all five.
- CHANGELOG.md: renumbered the BrainBench-extraction entry to 0.19.0 and
  placed it above master's 0.18.1 RLS entry. Tweaked copy ("In v0.17 it
  lived inside this repo" → "Previously it lived inside this repo") to
  stop implying a specific version that never shipped.
- CLAUDE.md: adjusted "BrainBench in a sibling repo" heading from
  (v0.18+) → (v0.19+).
- docs/benchmarks/2026-04-18-minions-vs-openclaw-production.md:
  resolved modify-vs-delete conflict in favor of delete (the extraction).
- scripts/llms-config.ts: dropped the docs/benchmarks/ entry (directory
  no longer exists here; lives in gbrain-evals).
- llms.txt / llms-full.txt: regenerated after the config change.
- bun.lock: accepted master's (master already dropped pdf-parse as a
  drive-by; aligned with our removal).

Tests: 2094 pass, 236 skip, 18 fail. Spot-checked failures — build-llms,
dream, orphans tests all pass in isolation. Failures reproduce only under
full-suite parallel load and are pre-existing master flakiness (matches the
graph-quality flake noted in the earlier summary). Not merge-introduced.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump to v0.20.0

Master is now at v0.18.2 (migration hardening + RLS + multi-source brains).
BrainBench extraction ships as v0.20.0 to leave v0.19 free for any in-flight
work on other branches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: remove eval-tests workflow (moved to gbrain-evals)

The Eval tests workflow ran `bun run eval:query:validate`, `test:eval`, and
`eval:world:render` — all three scripts moved to the gbrain-evals repo when
BrainBench was extracted in v0.20.0. The workflow has been failing on master
since the split because the scripts no longer exist here.

Eval CI now runs from gbrain-evals's own workflows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(tests): bump PGLite hook timeouts to 60s for parallel-load stability

Six test files spin up PGLite + 20 migrations + git repos in beforeEach/
beforeAll hooks. Under 136-way parallel test file execution, bun's default
5s hook timeout wasn't enough, producing 18 flaky failures that only
reproduced under full-suite parallel load (all 6 files passed in isolation).

Root cause: PGLite.create() + initSchema() takes ~3-5s under idle load, but
under 136 concurrent WASM instantiations the OS thrashes and hooks stall
well past 5s. The bunfig.toml `timeout = 60_000` applies to TESTS, not HOOKS
— bun requires per-hook timeouts as the third beforeEach/beforeAll argument.

Files touched (hook timeouts added, no test logic changed):
- test/dream.test.ts           — 5 describe blocks × before/afterEach
- test/orphans.test.ts         — 1 beforeEach + afterEach
- test/core/cycle.test.ts      — shared beforeAll + afterAll
- test/brain-allowlist.test.ts — beforeAll + afterAll
- test/extract-db.test.ts      — beforeAll + afterAll
- test/multi-source-integration.test.ts — beforeAll + afterAll

Results: 2317 pass / 0 fail (was 2253 pass / 18 fail).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: coverage for inferType() BrainBench corpus dirs

Closes the 1 gap surfaced by Step 7 coverage audit. 9 table-driven
assertions covering the new Page.type branches:
  emails/*.md, email/*.md       -> 'email'
  slack/*.md                    -> 'slack'
  cal/*.md, calendar/*.md       -> 'calendar-event'
  notes/*.md, note/*.md         -> 'note'
  meetings/*.md, meeting/*.md   -> 'meeting'

The fixtures use realistic paths from the amara-life-v1 corpus in the
sibling gbrain-evals repo (em-0001, sl-0037, evt-0042, mtg-0003) so the
test doubles as a contract check between the two repos.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(TODOS): mark BrainBench Cats 5/6/8/9/11 + v0.10.5 inferLinkType as completed

All five BrainBench categories shipped in v0.20.0 (to the gbrain-evals
sibling repo). v0.10.5 inferLinkType regex expansion shipped in-tree.

Remaining P1 BrainBench work: Cat 1+2 at full scale (2-3K pages) —
currently 240 pages in world-v1 corpus.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: sync CLAUDE.md + polish CHANGELOG voice for v0.20.0

CLAUDE.md: add v0.19 commands to key-files list (skillify, skillpack,
routing-eval, filing-audit, skill-manifest, resolver-filenames);
add 8 new test files + openclaw-reference-compat E2E to test index;
repoint the release-summary template's benchmark source from
`docs/benchmarks/[latest].md` to `gbrain-evals/docs/benchmarks/` since
those files now live in the sibling repo.

CHANGELOG voice polish for v0.20.0: replace em dashes with periods,
parens, or ellipses per project style guide. No content changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: regenerate llms-full.txt after CLAUDE.md + CHANGELOG edits (fixes CI)

The v0.20.0 doc-sync commit (9e567bb) added 7 new v0.19 modules to the
CLAUDE.md Key Files index and polished CHANGELOG voice. Both are
includeInFull: true inputs to llms-full.txt but the generator wasn't
re-run, so the drift-detection guard (test/build-llms.test.ts) failed CI.

One-line fix: regenerate. No content changes beyond what the two source
docs already carry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-24 00:08:54 -07:00
Garry TanClaude Opus 4.7WintermuteWintermute
 		96178d726e fix(subagent): v0.16.3 — bind Anthropic SDK correctly + enable tsc in CI (#318) 
* fix(subagent): bind Anthropic SDK messages.create() correctly

The makeSubagentHandler was casting `new Anthropic()` directly to
MessagesClient, but MessagesClient.create() maps to sdk.messages.create(),
not sdk.create(). Every subagent job immediately died with:

  client.create is not a function

Fix: wrap the SDK instance so .create() delegates to .messages.create()
with proper `this` binding via .bind(sdk.messages).

Discovered on first production run of gbrain agent against Supabase.

Co-Authored-By: Wintermute <wintermute@openclaw.ai>

* chore(ci): add typescript typecheck to test pipeline + clean up baseline errors

Root cause infra gap that let the v0.16.0 subagent bug ship: CI ran
only `bun test`, which transpiles types without checking them. Type
errors only surfaced at runtime, in production.

Changes:
- Add `typescript` devDep and a `typecheck` npm script (`tsc --noEmit`).
- Chain `bun run typecheck` into `bun run test` so developers get the
  same pipeline locally that CI runs.
- Flip `.github/workflows/test.yml` to invoke `bun run test` (the npm
  script, including typecheck) instead of `bun test` (runner only).
- Clean up 100+ pre-existing type errors across 30+ files so the first
  run of `tsc --noEmit` is green. Root causes were:
  - `databaseUrl` → `database_url` rename drift in test fixtures (9 files)
  - `PageType` union missing `'meeting'` / `'note'` entries that are
    already used in both src and tests (link-extraction.ts comments
    acknowledged the gap)
  - `GBrainConfig.storage` field never declared despite being read in
    files.ts and operations.ts
  - `ErrorCode` union missing `'permission_denied'`
  - `OrchestratorOpts` shape changed; test callers not updated
  - Dead-code comparisons in migration orchestrators against narrowed
    status types
  - postgres.js `Row`-callback type drift on several `.map()` calls
  - Buffer-as-BodyInit assignment in supabase.ts (real but non-fatal
    runtime bug; Uint8Array slice works and is type-correct)
  - Various `as X` single-step casts that now need `as unknown as X`
    per TS's stricter structural-conversion rules
- Bump `beforeAll` hook timeout to 30s on four PGLite-heavy tests that
  were flaky under parallel test execution: wait-for-completion,
  extract-fs, e2e/search-quality, e2e/graph-quality. All pass in
  isolation; timeouts only happened when dozens of PGLite instances
  init'd simultaneously.

The new CI pipeline now fails on any type error across src/ or test/,
giving us the compile-time regression guard the subagent fix depends on.

* fix(subagent): bind Anthropic SDK messages.create() correctly

Shipped bug: v0.16.0 cast `new Anthropic()` to `MessagesClient`, but
`.create()` lives at `sdk.messages.create`, not on the top-level client.
Every subagent job in production died on first LLM call with
`client.create is not a function`. Discovered on the first `gbrain agent
run` against Supabase.

Fix: assign `sdk.messages` directly to the `MessagesClient` slot.
`sdk.messages` IS the object with a callable `.create()`; the original
bug was picking the wrong entry point on the SDK. No helper, no
wrapper, no `.bind()` — JS method-call semantics preserve `this` at
the call site because `subagent.ts:336` invokes `client.create(...)`
with `client === sdk.messages`.

The one-line assignment also typechecks cleanly against the existing
`MessagesClient` interface (SDK's first `create` overload:
`(MessageCreateParamsNonStreaming, Core.RequestOptions?) =>
APIPromise<Message>` is assignable structurally). This gives us
compile-time regression protection: anyone reverting to
`new Anthropic()` would fail tsc because `Anthropic` has no top-level
`.create`. (The companion chore commit puts `tsc --noEmit` in CI so
this guard is enforced.)

Also adds a `makeAnthropic?: () => Anthropic` dep-injection seam so
the factory default construction branch is testable without real API
calls. Regression test drives one handler turn through a fake SDK,
asserting `sdk.messages.create` is actually called. If someone later
reverts to `new Anthropic()`, both guards fire: tsc fails AND the test
fails.

Co-Authored-By: Wintermute <wintermute@garrytan.com>

* chore(tests): add bunfig.toml + 60s hook timeouts to stabilize PGLite-heavy suites

After turning on tsc in CI (previous commit), running the full `bun run test`
suite in one shot triggered flaky `beforeEach/afterEach hook timed out`
failures on 8+ test files. Every failure traced to PGLite WASM init
contention when many test files spin up fresh PGLite instances in parallel;
each one alone passes in isolation.

- `bunfig.toml` sets the global test hook timeout to 60s (default is 5s),
  covering every test file without per-file edits.
- Individual `beforeAll(fn, 60_000)` / `beforeEach(fn, 15_000)` calls on
  the 8 tests that flaked most stay in place as explicit safety nets so
  a future bunfig config change doesn't silently re-introduce the flake.

Result: 1997 pass, 0 fail on `bun run test` (117 tests added since the
prior baseline by picking up typecheck-gated passes). No infrastructure
flake tolerated in CI.

* chore: bump version and changelog (v0.16.3)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Wintermute <wintermute@garrytan.com>
Co-authored-by: Wintermute <wintermute@openclaw.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-22 01:34:22 -07:00
Garry TanandClaude Opus 4.6 eb218a96ad security: pin GitHub Actions, add gitleaks CI, harden permissions (v0.4.2) (#23) 
* security: pin GitHub Actions to commit SHAs, add gitleaks CI

- Pin all 5 actions (checkout, setup-bun, upload-artifact, download-artifact,
  action-gh-release) to commit SHAs across 3 workflow files
- Add permissions: contents: read to test.yml and e2e.yml
- Add gitleaks secret scanning job to test.yml
- Pin openclaw install to v2026.4.9 in e2e.yml

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* security: add .gitleaks.toml config

Allowlists test fixtures, example env files, and skill documentation
to prevent false positives from the gitleaks CI step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add GitHub Actions SHA maintenance rule to CLAUDE.md

Instructs /ship and /review to check for stale SHA pins and update
them, keeping action versions fresh without manual effort.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add S3 Sig V4 TODO from CSO audit

Deferred from security audit. S3 storage backend accepts credentials
but sends unsigned requests. Implement when S3 becomes a real
deployment path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.4.2)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-10 05:26:09 -10:00
Garry TanandClaude Opus 4.6 2555de269a chore: add GitHub issue templates 
Bug report template (includes gbrain doctor --json field) and
feature request template.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-09 08:22:53 -10:00
Garry TanandClaude Opus 4.6 a86f995883 feat: GBrain v0.3.0 — contract-first architecture + ClawHub plugin (#7) 
* feat: contract-first operations.ts with OperationError, dry_run, importFromContent

30 shared operations as single source of truth for CLI and MCP.
- OperationError with typed error codes (page_not_found, invalid_params, etc.)
- dry_run support on all mutating operations
- importFromContent split from importFile with transaction wrapping
- Idempotency hash now includes ALL fields (title, type, frontmatter, tags)
- Config env var fallback: GBRAIN_DATABASE_URL > DATABASE_URL > config file

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: rewrite MCP server + CLI + tools-json from operations

server.ts: 233 -> ~80 lines. Tool definitions and dispatch generated from operations[].
cli.ts: shared operations auto-registered, CLI-only commands kept as manual dispatch.
tools-json: generated FROM operations[], eliminating the third contract surface.
Parity test verifies structural contract between operations, CLI, and MCP.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: delete 12 command files migrated to operations.ts

Handler logic for get, put, delete, list, search, query, health, stats,
tags, link, timeline, and version now lives in operations.ts.
Kept: init, upgrade, import, export, files, embed, sync, serve, call, config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: init --non-interactive, upgrade verification, schema migration

- gbrain init --non-interactive --url <url> for plugin mode (no TTY required)
- Post-upgrade version verification in gbrain upgrade
- Drop storage_url from files table (storage_path is the only identifier)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: tool-agnostic skills + new setup skill

All 7 skills rewritten with intent-based language instead of CLI commands.
Works with both CLI and MCP plugin contexts.
New setup skill replaces install: auto-provision Supabase via CLI,
AGENTS.md injection, target TTHW < 2 min.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: ClawHub bundle plugin, CI workflows, v0.3.0

- openclaw.plugin.json with configSchema, MCP server config, skill listing
- GitHub Actions: test on push/PR, multi-platform release (macOS arm64 + Linux x64)
- Version bump 0.3.0, CHANGELOG, README ClawHub section, CLAUDE.md updated

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: idempotency hash mismatch + MCP dry_run passthrough

importFromContent now passes its all-fields hash through putPage via
content_hash on PageInput, so the stored hash matches the computed hash.
Previously the skip-if-unchanged check never fired because the hash
formulas differed.

MCP server now passes dry_run from tool params to OperationContext.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.3.0.0)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: schema loader handles PL/pgSQL $$ blocks

Delete the semicolon-based SQL splitter in db.ts which broke on
PL/pgSQL trigger functions containing semicolons inside $$ delimiter
blocks. Use single conn.unsafe(schemaSql) call instead — the postgres
driver handles multi-statement SQL natively. schema.sql already uses
IF NOT EXISTS / CREATE OR REPLACE for idempotency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: E2E test infrastructure + realistic brain fixtures

Add test infrastructure for running E2E tests against real
Postgres+pgvector. Includes:
- test/e2e/helpers.ts: DB lifecycle, fixture import, timing, diagnostics
- 13 fixture files as a miniature realistic brain (people, companies,
  deals, meetings, concepts, projects, sources) following the
  compiled truth + timeline format from GBRAIN_RECOMMENDED_SCHEMA.md
- docker-compose.test.yml: local pgvector convenience (port 5433)
- .env.testing.example: template for test credentials
- package.json: add test:e2e script

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: E2E test suites + CI workflow

Tier 1 (mechanical.test.ts): 14 test suites covering all operations
against real Postgres — page CRUD, search with quality scoring, links,
tags, timeline, versions, admin, chunks, resolution, ingest log, raw
data, files, idempotency stress, setup journey (full CLI flow), init
edge cases, schema idempotency, schema diff guard, performance baselines.

Tier 1 (mcp.test.ts): MCP protocol test — spawns server, sends JSON-RPC,
verifies tools/list matches operations count.

Tier 2 (skills.test.ts): OpenClaw skill tests — ingest, query, health.
Skips gracefully when dependencies missing.

CI (.github/workflows/e2e.yml): Tier 1 on every PR (pgvector service),
Tier 2 nightly/manual with API key secrets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: E2E test fixes + traverseGraph jsonb cast

- Fix traverseGraph query: cast json_agg to jsonb_agg so SELECT DISTINCT works
- Fix put_page tests to use importFromContent with noEmbed (no OpenAI key in Tier 1)
- Fix get_health assertion (page_count not total_pages)
- Fix raw_data test to handle JSONB string/object return
- Simplify MCP test to verify tool generation directly
- Add timeouts to CLI subprocess tests
- Use port 5434 for docker-compose (5433 often in use)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update all project docs for E2E test suite

- CLAUDE.md: updated test count (9 unit + 3 E2E), added E2E test
  instructions, fixed skill count to 8
- CONTRIBUTING.md: updated project structure with test/e2e/, added E2E
  test instructions, rewrote "Adding a new command" to reflect
  contract-first architecture (add to operations.ts, done)
- README.md: fixed table count (10 not 9), added recommended schema doc
  to Docs section, added E2E instructions to Contributing section
- CHANGELOG.md: added E2E test suite, docker-compose, schema loader fix,
  and traverseGraph jsonb fix to v0.3.0 entry

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 23:26:11 -10:00