Compare commits

..
Author SHA1 Message Date
Garry TanandClaude Opus 5 dc0504785a fix(check-update): resolve the latest version from VERSION, not the empty releases API (#486)
The repo publishes zero GitHub releases, so releases/latest is a permanent
404 and fetchLatestRelease() could never succeed — the entire upgrade
notification subsystem was a silent no-op, and refreshUpdateCache() cached
a fabricated up_to_date marker on every failure.

- Resolve the latest version from raw.githubusercontent.com/.../master/VERSION
  (same trusted host fetchChangelog already uses). Bounded, shape-gated parse;
  handles legacy 3-segment and -suffix channel forms.
- Discriminate network_error from no_releases in the --json error field and
  human output.
- Never write up_to_date on a failed check: preserve the last-known-good
  marker (mtime bump keeps the TTL throttle) or write nothing.
- Rejected the issue's proposed npm fallback: the gbrain npm package is an
  unrelated GPU library (#505) and would produce false upgrade prompts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 13:02:33 -07:00
79 changed files with 246 additions and 1901 deletions
+1 -1
View File
@@ -396,7 +396,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this
- `src/commands/lint.ts` — Page quality linter (catches LLM artifacts, placeholder dates)
- `src/commands/report.ts` — Structured report saver (audit trail for maintenance/enrichment)
- `src/core/destructive-guard.ts` — three-layer protection against accidental data loss. `assessDestructiveImpact(engine, sourceId)` counts pages/chunks/embeddings/files for a source. `checkDestructiveConfirmation(impact, opts)` is the fail-closed gate (`--confirm-destructive` required when data is present; `--yes` alone is rejected). `softDeleteSource` / `restoreSource` / `listArchivedSources` / `purgeExpiredSources` drive the source-level archive lifecycle via `sources.archived BOOLEAN`, `archived_at TIMESTAMPTZ`, `archive_expires_at TIMESTAMPTZ`. Page-level analog: `BrainEngine.softDeletePage` / `restorePage` / `purgeDeletedPages` plus `pages.deleted_at TIMESTAMPTZ` and a partial purge index. The MCP `delete_page` op rewires to `softDeletePage`; ops `restore_page` (`scope: write`) and `purge_deleted_pages` (`scope: admin`, `localOnly: true`) round out the surface. Search visibility (`buildVisibilityClause` in `src/core/search/sql-ranking.ts`) hides soft-deleted pages and archived sources from `searchKeyword` / `searchKeywordChunks` / `searchVector` in both engines. The autopilot cycle's `purge` phase calls `purgeExpiredSources` + `engine.purgeDeletedPages(72)` so the 72h TTL is real.
- `src/commands/pages.ts``gbrain purge-deleted [--older-than HOURS|Nd] [--dry-run] [--json]` operator escape hatch. Mirror of `gbrain sources purge` for the page-level lifecycle. Hard-deletes pages whose `deleted_at` is older than the cutoff; cascades to content_chunks/page_links/chunk_relations.
- `src/commands/pages.ts``gbrain pages purge-deleted [--older-than HOURS|Nd] [--dry-run] [--json]` operator escape hatch. Mirror of `gbrain sources purge` for the page-level lifecycle. Hard-deletes pages whose `deleted_at` is older than the cutoff; cascades to content_chunks/page_links/chunk_relations.
- `src/core/op-checkpoint.ts` — DB-backed checkpoint primitive for long-running ops. Migration v67 introduces `op_checkpoints (op TEXT, fingerprint TEXT, completed_keys JSONB, updated_at TIMESTAMPTZ, PK(op, fingerprint))`. Per-op fingerprint helpers (`embedFingerprint`, `extractFingerprint`, `reindexFingerprint`, `integrityFingerprint`, `purgeFingerprint`) compute `sha8(canonical-JSON(relevant-params))` so re-running with the same params resumes from `completed_keys` and re-running with different params (e.g. `--limit 100` vs `--limit 200`) starts fresh. Cross-worker safe on Postgres (DB row, no file-lock race); PGLite degrades gracefully. Replaces per-op file-backed JSON checkpoints scattered across `import.ts`, `embed.ts`, `reindex.ts`. The 7-day TTL GC runs in the cycle's `purge` phase. All writes (`recordCompleted`, `clearOpCheckpoint`) route through `engine.executeRawDirect` + `withRetry(BULK_RETRY_OPTS)` so they survive Supavisor pool exhaustion, and `recordCompleted` returns `boolean` (banked vs failed-after-retries) — the 9 non-sync consumers keep its REPLACE-into-`completed_keys` semantics. Resumable sync uses the additive `appendCompleted(key, deltaKeys)` / `appendCompletedOnce` (the latter no-retry for the SIGTERM path) which INSERT a delta into the `op_checkpoint_paths` child table (migration v115: `(op, fingerprint, path)` PK, FK to `op_checkpoints` ON DELETE CASCADE) via a single writable-CTE `unnest($3::text[])` write — O(delta), killing the old O(N²) full-set rewrite. `loadOpCheckpoint` returns the `UNION ALL` of legacy `completed_keys` + child-table paths (deduped in JS), so an in-flight upgrade loses nothing. The legacy arm is gated on `jsonb_typeof(completed_keys) = 'array'` so a non-array (scalar) parent row can't make `jsonb_array_elements_text` throw "cannot extract elements from a scalar" and take down the whole union (which would discard the valid child rows and lose all banked progress for the key); a third union arm flags the corruption so the loader logs it once and keeps the child rows. Migration v119 adds the `op_checkpoints_completed_keys_array` CHECK (`jsonb_typeof(completed_keys) = 'array'`) — a DB-enforced, always-on guard that makes the scalar-corruption class structurally impossible going forward; the migration repairs any pre-existing scalar to `'[]'` under `LOCK TABLE ... IN SHARE ROW EXCLUSIVE MODE` and `src/core/schema-embedded.ts` + `src/core/pglite-schema.ts` ship the same CHECK on fresh installs (a loader hit now implies schema drift, a disabled constraint, or an out-of-band writer). `recordCompleted` binds its array through `$3::text::jsonb` (NOT a bare `$3::jsonb`) so postgres.js `.unsafe()` doesn't double-encode `JSON.stringify(sorted)` into the scalar string that CHECK rejects — the #2339 bug that aborted every multi-source sync at the first pin write (PGLite parsed it silently, so it shipped). A DATABASE_URL-gated `test/e2e/op-checkpoint-jsonb-parity.test.ts` (its own CI job) asserts the array shape on real Postgres. `syncFingerprint({sourceId, lastCommit})` keys the sync rows. Pinned by `test/op-checkpoint.test.ts` (incl. delta-append, union read, cascade clear, durable-write boolean, and the scalar-parent guard). `import-checkpoint.ts` was NOT migrated to this primitive — both checkpoint systems coexist without conflict; migrating requires async-propagating 4 sync call sites in `src/commands/import.ts` and rewriting 18 tests, deferred.
- `src/core/brain-score-recommendations.ts` — pure data layer consumed by both `gbrain doctor --remediation-plan` / `--remediate` and `gbrain features`. `computeRecommendations(checks, opts)` returns `Remediation[]` with stable `id`, content-hash `idempotency_key`, `severity`, `est_seconds`, `est_usd_cost`, `depends_on` (references stable ids, not check names — so plan order is reproducible). `classifyChecks(report)` triages every doctor check three-state into `remediable | human_only | blocked` (`human_only` covers RLS warnings and other human-judgment gates; `blocked` covers dependency chains where a parent check failed). `maxReachableScore(checks)` computes the ceiling for empty/under-configured brains (no entity pages → graph_coverage caps at 70; no embedding key → embedding_coverage caps at 60). Cost estimates pull from `anthropic-pricing.ts` (synthesize/patterns/consolidate) and `embedding-pricing.ts` (embed jobs). Pinned by `test/brain-score-recommendations.test.ts` (~27 cases incl. determinism, content-hash idempotency, DB-backed checkpoint provenance, three-state triage).
- `src/commands/doctor.ts` extension — `--remediation-plan [--json] [--target-score N]` prints what would run (stable `id`, `idempotency_key`, `severity`, `est_seconds`, `est_usd_cost`, `depends_on`); `--remediate [--yes] [--target-score N] [--max-usd N]` submits each plan step as a Minion job in dependency order, re-checking score between steps. `--target-score N` defaults to 90; refuses to start when target exceeds `maxReachableScore()` and lists what's missing. `--max-usd N` is the cron-safety guard — submission refuses when the plan's `est_total_usd_cost` exceeds the cap. JSON envelope adds a `Check.remediation` field (additive, schema_version unchanged). Pinned by tests in `test/doctor.test.ts`.
+1 -1
View File
@@ -229,7 +229,7 @@ add `GBRAIN_AUDIT_FULL=1` (v0.43+ TODO; not yet wired).
- Per-source pack-upgrade (the handler accepts `sourceId` but
`findPackSuccessors` doesn't yet pass it through)
- Cross-brain federated mounts that disagree on canonical packs
- Automatic rollback (today: manual SQL or `gbrain restore`)
- Automatic rollback (today: manual SQL or `gbrain pages restore`)
- LLM-assisted mapping_rules codegen from production data (`gbrain
schema detect-mappings`; deferred to v0.43+)
+1 -1
View File
@@ -214,7 +214,7 @@ gbrain schema downgrade
1. `git revert <merge-commit>` — restores the code.
2. `gbrain schema downgrade --to gbrain-base` — restores config.
3. (Optional) `gbrain purge-deleted --older-than 0h` — drops
3. (Optional) `gbrain pages purge-deleted --older-than 0h` — drops
v0.39-typed pages that no longer have a matching type in the active
pack.
+10 -10
View File
@@ -19,13 +19,11 @@ entire DB from scratch.
This means:
- **Disaster recovery is a short, boring sequence.** If your DB volume
corrupts, if Postgres eats itself, if PGLite's WASM lock wedges — you
don't need a backup. You wipe the derived tables (on PGLite,
`gbrain reinit-pglite` wipes the whole embedded DB), re-import from
your brain repo with `gbrain sync`, and `gbrain extract all`
regenerates the derived state. See "Disaster recovery" below for the
exact commands.
- **Disaster recovery is one command.** If your DB volume corrupts, if
Postgres eats itself, if PGLite's WASM lock wedges — you don't need
a backup. You wipe the DB, re-import from your brain repo, and the
derived state regenerates. v0.32.3 ships `gbrain rebuild
--confirm-destructive` as the documented one-liner.
- **Multi-machine sync is git.** Your brain is a repo. Push from one
machine, pull from another, and the second machine's DB rebuilds on
its next sync. No "back up the database" step.
@@ -148,9 +146,11 @@ The promise the rule makes:
# Snapshot what's there
gbrain stats > /tmp/before.txt
# Wipe and rebuild — delete the derived tables (pages + content_chunks
# survive the CASCADE-safe design), then re-derive from the repo.
# On PGLite, `gbrain reinit-pglite` wipes the whole embedded DB instead.
# Wipe and rebuild
gbrain rebuild --confirm-destructive # v0.32.3 — deletes derived tables
# (pages + content_chunks survive
# the CASCADE-safe design)
# OR manually for v0.32.2:
psql -c 'DELETE FROM facts; DELETE FROM takes; DELETE FROM links; DELETE FROM timeline_entries;'
gbrain sync
gbrain extract all
+2 -2
View File
@@ -108,8 +108,8 @@ Every primitive ships with a documented rollback:
| Operation | Rollback |
|-----------|----------|
| Retype | `frontmatter.legacy_type = <original>` preserved on every page (D8). One SQL UPDATE restores types: `UPDATE pages SET type = frontmatter->>'legacy_type' WHERE frontmatter ? 'legacy_type'`. |
| Page-to-link | Source page soft-deleted with 72h TTL. `gbrain restore <slug>` within 72h. Link row stays harmless if source restored. |
| Page-to-alias | Source page soft-deleted with 72h TTL. `gbrain restore <slug>` within 72h. Alias row stays harmless (or `DELETE FROM slug_aliases WHERE alias_slug = <slug>` to clean up). |
| Page-to-link | Source page soft-deleted with 72h TTL. `gbrain pages restore <slug>` within 72h. Link row stays harmless if source restored. |
| Page-to-alias | Source page soft-deleted with 72h TTL. `gbrain pages restore <slug>` within 72h. Alias row stays harmless (or `DELETE FROM slug_aliases WHERE alias_slug = <slug>` to clean up). |
| Active-pack flip | `gbrain schema use gbrain-base` reverses the flip. |
## What if my brain doesn't fit?
+1 -1
View File
@@ -183,6 +183,6 @@ This also means the best AI agent setups will be open source by default. Closed,
Software distribution reimagined: the package is a markdown file, the runtime is a sufficiently smart model, the package manager is your AI agent, and the app store is a git repo.
`gbrain skillpack scaffold voice-agent`
`gbrain install voice-agent`
That's it.
+1 -1
View File
@@ -69,7 +69,7 @@ update_brain_page(slug, new_info, source):
page = gbrain get {slug}
// TIMELINE: always APPEND (never edit existing entries)
gbrain timeline-add {slug} {
gbrain add_timeline_entry {slug} {
date: today,
summary: new_info.summary,
detail: new_info.detail,
+9 -9
View File
@@ -46,10 +46,10 @@ on user_shares_media(url_or_file):
# Step 4: Extract and cross-reference entities
for person in transcript.mentioned_people:
gbrain link <slug> <person_slug>
gbrain link <person_slug> <slug>
gbrain timeline-add <person_slug> {date} \
"Discussed in {video_title}: {what_was_said}" \
gbrain add_link <slug> <person_slug>
gbrain add_link <person_slug> <slug>
gbrain add_timeline_entry <person_slug> \
--entry "Discussed in {video_title}: {what_was_said}" \
--source "YouTube: {url}"
# PATTERN 2: Social Media Bundles
@@ -80,8 +80,8 @@ on user_shares_media(url_or_file):
# Extract entities and cross-reference
for entity in bundle.mentioned_entities:
gbrain link <slug> <entity_slug>
gbrain link <entity_slug> <slug>
gbrain add_link <slug> <entity_slug>
gbrain add_link <entity_slug> <slug>
# PATTERN 3: PDFs and Documents
elif media.type == "pdf" or media.type == "document":
@@ -109,8 +109,8 @@ on user_shares_media(url_or_file):
"""
for entity in document.mentioned_entities:
gbrain link <slug> <entity_slug>
gbrain link <entity_slug> <slug>
gbrain add_link <slug> <entity_slug>
gbrain add_link <entity_slug> <slug>
# Always sync after ingestion
gbrain sync
@@ -127,7 +127,7 @@ on user_shares_media(url_or_file):
## How to Verify
1. Ingest a YouTube video. Run `gbrain get media/youtube/{slug}`. Confirm the page has: the agent's analysis (not just a summary), key quotes with speaker attribution, and the full diarized transcript.
2. Run `gbrain call get_links '{"slug": "media/youtube/{slug}"}'`. Confirm back-links exist to brain pages for every person and company mentioned in the video.
2. Run `gbrain get_links media/youtube/{slug}`. Confirm back-links exist to brain pages for every person and company mentioned in the video.
3. Pick a person mentioned in the video. Run `gbrain get <person_slug>`. Confirm their timeline has a new entry referencing the video with specific context.
4. Ingest a tweet. Confirm the brain page includes the thread context, linked article summaries, and entity cross-references -- not just the tweet text.
5. Run `gbrain search "{topic_from_video}"`. Confirm the media page appears in search results (verifies the content is indexed and searchable).
+9 -9
View File
@@ -49,23 +49,23 @@ on enrich(entity, trigger):
data["contacts"] = google_contacts(entity.email) # Contact data
# Step 5: Store raw data (auditable, re-processable)
gbrain call put_raw_data \
'{"slug": "<entity_slug>", "data": {"sources": {"crustdata": {"fetched_at": "...", "data": {...}}, ...}}}'
gbrain put_raw_data <entity_slug> \
--data '{"sources": {"crustdata": {"fetched_at": "...", "data": {...}}, ...}}'
# Overwrite on re-enrichment, don't append
# Step 6: Write to brain page
if path == "CREATE":
gbrain put <entity_slug> --content "<compiled_truth_from_all_sources>"
gbrain timeline-add <entity_slug> {date} "Page created via enrichment"
gbrain add_timeline_entry <entity_slug> --entry "Page created via enrichment"
elif path == "UPDATE":
# Append timeline, update compiled truth ONLY if materially new
gbrain timeline-add <entity_slug> {date} "Enriched: {new_signal}"
gbrain add_timeline_entry <entity_slug> --entry "Enriched: {new_signal}"
# Flag contradictions -- don't silently resolve them
# Step 7: Cross-reference the graph
gbrain link <person_slug> <company_slug> # person -> company
gbrain link <company_slug> <person_slug> # company -> person
gbrain link <person_slug> <deal_slug> # person -> deal
gbrain add_link <person_slug> <company_slug> # person -> company
gbrain add_link <company_slug> <person_slug> # company -> person
gbrain add_link <person_slug> <deal_slug> # person -> deal
# Every entity page links to every other entity page that references it
# People page sections (not a LinkedIn profile -- a living portrait):
@@ -94,8 +94,8 @@ on enrich(entity, trigger):
## How to Verify
1. Enrich a Tier 1 person. Run `gbrain get <slug>` and confirm the page has Executive Summary, State, What They Believe, Contact, and Timeline sections populated from multiple sources.
2. Run `gbrain call get_raw_data '{"slug": "<slug>"}'`. Confirm raw API responses are stored with `sources.{provider}.fetched_at` timestamps.
3. Run `gbrain call get_links '{"slug": "<slug>"}'`. Confirm cross-reference links exist to the person's company page, deal pages, and related entities.
2. Run `gbrain get_raw_data <slug>`. Confirm raw API responses are stored with `sources.{provider}.fetched_at` timestamps.
3. Run `gbrain get_links <slug>`. Confirm cross-reference links exist to the person's company page, deal pages, and related entities.
4. Check a page that was enriched AND has a user-written Assessment. Confirm the Assessment section was preserved, not overwritten by API data.
5. Try to re-enrich the same person. Confirm the system checks the `fetched_at` timestamp and skips if less than a week old.
+5 -5
View File
@@ -53,7 +53,7 @@ on upcoming_meeting(meeting):
"last_interaction": page.timeline[0], # most recent
"open_threads": page.open_threads,
"relationship_temperature": page.relationship,
"relevant_deals": gbrain call get_links '{"slug": "<attendee_slug>"}',
"relevant_deals": gbrain get_links <attendee_slug>,
}
else:
briefing[attendee] = "No brain page -- consider enriching"
@@ -67,14 +67,14 @@ on inbox_cleared():
for email in processed_emails:
if email.contained_new_information:
# Update the sender's brain page with new signal
gbrain timeline-add <sender_slug> {date} \
"Email re: {subject}. Key info: {extracted_signal}" \
gbrain add_timeline_entry <sender_slug> \
--entry "Email re: {subject}. Key info: {extracted_signal}" \
--source "email from {sender} re {subject}, {date}"
# Update any mentioned entity pages too
for entity in email.mentioned_entities:
gbrain timeline-add <entity_slug> {date} \
"{what_was_said_about_them}" \
gbrain add_timeline_entry <entity_slug> \
--entry "{what_was_said_about_them}" \
--source "email from {sender}, {date}"
# WORKFLOW 4: Scheduling Nudges
+7 -7
View File
@@ -32,15 +32,15 @@ on new_meeting_transcript(meeting):
# Step 3: Propagate to ALL entity pages (MANDATORY -- most agents skip this)
for person in meeting.attendees + meeting.mentioned_people:
gbrain timeline-add <person_slug> {date} \
"Met in '{meeting.title}' on {date}. Key points: ..." \
gbrain add_timeline_entry <person_slug> \
--entry "Met in '{meeting.title}' on {date}. Key points: ..." \
--source "Meeting notes '{meeting.title}', {date}"
# Update their State section if new information surfaced
# Update company pages for each person's company if relevant
for company in meeting.mentioned_companies:
gbrain timeline-add <company_slug> {date} \
"Discussed in '{meeting.title}': {what_was_said}" \
gbrain add_timeline_entry <company_slug> \
--entry "Discussed in '{meeting.title}': {what_was_said}" \
--source "Meeting notes '{meeting.title}', {date}"
# Step 4: Extract action items
@@ -49,8 +49,8 @@ on new_meeting_transcript(meeting):
# Step 5: Back-link everything (bidirectional graph)
for entity in all_entities_mentioned:
gbrain link <slug> <entity_slug> # meeting -> entity
gbrain link <entity_slug> <slug> # entity -> meeting
gbrain add_link <slug> <entity_slug> # meeting -> entity
gbrain add_link <entity_slug> <slug> # entity -> meeting
# Step 6: Sync so new pages are immediately searchable
gbrain sync
@@ -73,7 +73,7 @@ on new_meeting_transcript(meeting):
1. After ingesting a meeting, run `gbrain get meetings/{date}-{slug}`. Confirm the page has the agent's analysis above the bar and the full diarized transcript below it.
2. For each attendee, run `gbrain get <attendee_slug>`. Check that their timeline has a new entry referencing the meeting with specific insights (not just "attended meeting").
3. Pick a company mentioned in the meeting. Run `gbrain get <company_slug>`. Confirm a timeline entry exists referencing what was discussed about the company.
4. Run `gbrain call get_links '{"slug": "meetings/{date}-{slug}"}'`. Verify back-links exist to all attendee and entity pages.
4. Run `gbrain get_links meetings/{date}-{slug}`. Verify back-links exist to all attendee and entity pages.
5. Run `gbrain search "{meeting_topic}"`. Confirm the meeting page appears in search results (verifies sync ran).
---
+3 -3
View File
@@ -91,7 +91,7 @@ first):
6. The seeded `default` source.
So inside `~/.gstack/plans/` on a brain that pinned `gstack` to
`~/.gstack` via `.gbrain-source`, `gbrain put` implicitly writes to
`~/.gstack` via `.gbrain-source`, `gbrain put-page` implicitly writes to
the `gstack` source. Outside any registered directory with no env/dotfile
set, it writes to the default.
@@ -188,10 +188,10 @@ citations keep working.
```bash
# Pass --source explicitly
gbrain put topics/ai ... --source wiki
gbrain put-page topics/ai ... --source wiki
# Or rely on the dotfile / env / CWD match
cd ~/.gstack && gbrain put plans/multi-repo ...
cd ~/.gstack && gbrain put-page plans/multi-repo ...
# → source auto-resolves to gstack
```
+8 -8
View File
@@ -20,8 +20,8 @@ on every_inbound_message(message):
for entity in entities:
existing = gbrain search "{entity.name}"
if existing:
gbrain timeline-add <entity_slug> {date} \
"{what_was_said}" \
gbrain add_timeline_entry <entity_slug> \
--entry "{what_was_said}" \
--source "User, direct message, {timestamp}"
# else: flag for enrichment if important enough
@@ -64,13 +64,13 @@ on nightly_schedule("02:00"):
# The brain COMPOUNDS overnight.
# 5a: Entity sweep -- find unlinked mentions
pages = gbrain list
pages = gbrain list_pages
for page in pages:
mentions = extract_entity_mentions(page.content)
existing_links = gbrain call get_links '{"slug": "<page.slug>"}'
existing_links = gbrain get_links <page.slug>
for mention in mentions:
if mention not in existing_links:
gbrain link <page.slug> <mention_slug> # fix broken graph
gbrain add_link <page.slug> <mention_slug> # fix broken graph
# 5b: Citation audit -- find facts without sources
for page in pages:
@@ -80,7 +80,7 @@ on nightly_schedule("02:00"):
# 5c: Memory consolidation -- update compiled truth from timeline
for page in stale_pages(older_than="7d"):
timeline = gbrain timeline <page.slug>
timeline = gbrain get_timeline <page.slug>
if timeline.has_new_entries_since_last_consolidation:
# Re-synthesize compiled truth from accumulated timeline
updated_truth = consolidate(page.compiled_truth, timeline.new_entries)
@@ -110,11 +110,11 @@ on nightly_schedule("02:00"):
## How to Verify
1. Send a message mentioning a person with a brain page. Confirm the agent detects the entity and adds a timeline entry to their page (`gbrain timeline <slug>`).
1. Send a message mentioning a person with a brain page. Confirm the agent detects the entity and adds a timeline entry to their page (`gbrain get_timeline <slug>`).
2. Ask the agent about someone in the brain. Confirm it runs `gbrain search` or `gbrain get` BEFORE reaching for external APIs (check the tool call order).
3. Write a new page with `gbrain put`, then immediately run `gbrain search` for it. Confirm it appears in results (verifies sync ran).
4. Run `gbrain doctor`. Confirm it returns a health report with database status, page count, and any flagged issues.
5. After a dream cycle runs, check a page that had unlinked entity mentions. Confirm new links were added (`gbrain call get_links '{"slug": "<slug>"}'`).
5. After a dream cycle runs, check a page that had unlinked entity mentions. Confirm new links were added (`gbrain get_links <slug>`).
---
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md).*
+3 -3
View File
@@ -47,8 +47,8 @@ on user_message(message):
# Step 3: Cross-link to everything that shaped the thinking
for entity in idea.influences:
gbrain link originals/{slug} <entity_slug>
gbrain link <entity_slug> originals/{slug}
gbrain add_link originals/{slug} <entity_slug>
gbrain add_link <entity_slug> originals/{slug}
# Step 4: Sync
gbrain sync
@@ -79,7 +79,7 @@ on user_message(message):
1. Generate an original idea in conversation (e.g., "I call this the 'ambition debt' problem -- every year you delay going big, the compound interest works against you"). Confirm a new page appears at `brain/originals/ambition-debt` with `gbrain get originals/ambition-debt`.
2. Check that the page uses the user's exact phrasing for the title and slug -- not a sanitized version.
3. Run `gbrain call get_links '{"slug": "originals/ambition-debt"}'`. Confirm cross-links exist to related people, meetings, or other originals.
3. Run `gbrain get_links originals/ambition-debt`. Confirm cross-links exist to related people, meetings, or other originals.
4. Express a take on someone else's idea (e.g., "I think Thiel's contrarian question is wrong because..."). Confirm it goes to `originals/` (synthesis is original), not `concepts/`.
5. Run `gbrain search "ambition debt"`. Confirm the originals page appears in search results and is discoverable.
+1 -1
View File
@@ -87,7 +87,7 @@ expect it.
| `version` | string | yes | Your plugin's semver. Informational. |
| `plugin_version` | string | yes | Contract lock. Must equal `"gbrain-plugin-v1"` for v0.15. |
| `subagents` | string | no | Subdir name (default `subagents`). Escape-attempts are rejected. |
| `description` | string | no | Shown in a future plugin-listing command. |
| `description` | string | no | Shown in future `gbrain plugin list`. |
## Subagent definition files
+1 -1
View File
@@ -250,7 +250,7 @@ All 30 GBrain operations are available remotely, including `sync_brain` and
directory where `gbrain serve` was launched. Symlinks, `..` traversal, and absolute
paths outside cwd are rejected. Page slugs and filenames are allowlist-validated
(alphanumeric + hyphens; no control chars, RTL overrides, or backslashes). Local
CLI callers (`gbrain files upload ...`) keep unrestricted filesystem access since
CLI callers (`gbrain file upload ...`) keep unrestricted filesystem access since
the user owns the machine.
## Deployment Options
+1 -1
View File
@@ -13,7 +13,7 @@ Step-by-step walkthroughs that take you from zero to a working outcome. Concrete
These are the next tutorials on the roadmap. Open an issue if one of them is the one you need most; that's how we'll prioritize.
- **Set up GBrain for VC dealflow** — the operator's recipe. People pages for founders, companies with typed Facts fence carrying ARR / team-size / runway across dates, meetings auto-ingested, deal pages linking everything. Shows `gbrain whoknows`, `gbrain find-trajectory`, and `gbrain founder scorecard` on real workflows.
- **Set up GBrain for VC dealflow** — the operator's recipe. People pages for founders, companies with typed Facts fence carrying ARR / team-size / runway across dates, meetings auto-ingested, deal pages linking everything. Shows `gbrain whoknows`, `gbrain find_trajectory`, and `gbrain founder scorecard` on real workflows.
- **Migrate your existing vault into GBrain** — for Notion / Obsidian / Roam users with a vault that doesn't match GBrain's default layout. Walks through `gbrain schema detect``suggest``review-candidates` so the brain learns your shape instead of forcing you to learn its.
+1 -1
View File
@@ -554,7 +554,7 @@ What to do next:
- **Wire ingestion** from external systems (Granola, Linear, Slack) using the [ingestion source contract](../skillpack-anatomy.md). Most companies want their meetings auto-ingested so the brain stays current without anyone typing notes.
- **Set up team-specific dashboards** through the admin UI. Each team lead can have their own view of brain health and activity.
- **Explore the rest of the brain layer.** `gbrain whoknows` (find the expert on a topic), `gbrain find-trajectory` (how a metric changed over time), `gbrain founder scorecard` (especially useful for VC and ops teams), the contradiction-detection cycle that surfaces conflicts between different people's notes.
- **Explore the rest of the brain layer.** `gbrain whoknows` (find the expert on a topic), `gbrain find_trajectory` (how a metric changed over time), `gbrain founder scorecard` (especially useful for VC and ops teams), the contradiction-detection cycle that surfaces conflicts between different people's notes.
If you're building in this space (which YC has flagged as the [company-brain category in its Request for Startups](https://www.ycombinator.com/rfs#company-brain)), you might as well build on this. Everything described above is open source, MIT licensed, and what I run in production behind my own AI agents.
+9 -9
View File
@@ -115,21 +115,21 @@ You can use the same keys across multiple agents.
## Step 6: Install GBrain
Once OpenClaw is running, installation is two commands — one in the brain repo, one in the agent workspace:
Once OpenClaw is running:
```bash
# In the BRAIN repo (the git repo that holds your markdown pages):
gbrain init --supabase
# In the AGENT WORKSPACE repo (where OpenClaw runs):
gbrain skillpack scaffold --all
gbrain install
```
`gbrain init --supabase` walks a short wizard that asks for your Supabase connection string and creates the schema. You'll get that connection string in Step 7 — read 7a and 7b first so you paste the right one (the transaction pooler, not the direct connection). If you'd rather try things locally before paying for a database, `gbrain init --pglite` gives you a zero-config embedded engine instead; you can migrate to Supabase later with `gbrain migrate --to supabase`.
This installs:
`gbrain skillpack scaffold --all` copies the ~43 bundled skills into your agent workspace as first-class files you can edit freely. (The old managed-install model was retired in v0.36.0.0; see `docs/INSTALL.md` if you're upgrading from an older release.)
- About 60 skills
- About 9 skill packs
- Default brain structure
- MCP server configuration
- Supabase connection (for embeddings and search)
From this point, the agent has working memory and access to every skill.
GBrain populates the brain repo with its default directory structure, skill files, and configuration. From this point, the agent has working memory and access to every skill.
---
+3 -3
View File
@@ -32,7 +32,7 @@ gbrain schema sync --apply
The sync backfills `page.type = 'meeting'` on all 4000 pages in 1000-row batches. Now:
- `gbrain whoknows "Q3 roadmap discussion"` routes through the meeting type, ranking by `expert_routing` signal (attendees, recency, salience) instead of raw text.
- The `extract_facts` cycle runs on every meeting page automatically (because `extractable: true`), pulling typed facts like `attended_by=alice-example`, `date=2026-05-23`.
- `gbrain extract-facts` runs on every meeting page automatically (because `extractable: true`), pulling typed facts like `attended_by=alice-example`, `date=2026-05-23`.
- The downstream `think` skill can now answer "what did we decide about pricing in the last three roadmap meetings" by querying the meeting graph instead of grep'ing 4000 files.
One command. 4000 pages went from invisible to queryable. The content didn't change. The structure did.
@@ -62,7 +62,7 @@ gbrain schema add-link-type led-by --page-type deal --target-type inves
gbrain schema sync --apply
```
Now `gbrain whoknows "Series A SaaS"` routes through `investor` and `portco` types specifically, not the noisy general type set. `gbrain graph-query alice-example --type intro-from --depth 2` walks two hops of intros to surface "Alice introduced you to Bob who introduced you to Charlie." The `extract_facts` cycle starts producing typed claims from the fence in your deal pages: `(deals/acme-seed, raise=2000000, valuation=15000000, lead=widget-vc, closed_at=2026-05-23)`.
Now `gbrain whoknows "Series A SaaS"` routes through `investor` and `portco` types specifically, not the noisy general type set. `gbrain graph-query alice-example --type intro-from --depth 2` walks two hops of intros to surface "Alice introduced you to Bob who introduced you to Charlie." `gbrain extract-facts` starts producing typed claims from the fence in your deal pages: `(deals/acme-seed, raise=2000000, valuation=15000000, lead=widget-vc, closed_at=2026-05-23)`.
The CRM you've been promising yourself you'll set up next quarter? You just shipped it in 4 commands. It's downstream of your notes, not parallel to them.
@@ -143,7 +143,7 @@ Re-run the same `whoknows` query. Top-3 should shift, because the new type is no
Three things gbrain does that generic note systems can't:
**1. The brain knows the difference between a person and an idea.** Page-type matters at query time. `gbrain whoknows` only considers `expert_routing: true` types. The `extract_facts` cycle only runs on `extractable: true` types. `gbrain graph-query` walks declared link verbs. None of that works on a flat tag system because tags don't have semantics — they're labels. Types are first-class citizens with rules attached.
**1. The brain knows the difference between a person and an idea.** Page-type matters at query time. `gbrain whoknows` only considers `expert_routing: true` types. `gbrain extract-facts` only runs on `extractable: true` types. `gbrain graph-query` walks declared link verbs. None of that works on a flat tag system because tags don't have semantics — they're labels. Types are first-class citizens with rules attached.
**2. Untyped content is invisible content.** If your meetings are typed as `note`, expert routing skips them, facts extraction ignores them, link inference doesn't fire. They exist on disk and they're indexed for text search, but the structural surfaces (whoknows, find_experts, recall, think) treat them as second-class. Adding a type isn't cosmetic; it's structural promotion.
+4 -4
View File
@@ -2316,7 +2316,7 @@ gbrain schema sync --apply
The sync backfills `page.type = 'meeting'` on all 4000 pages in 1000-row batches. Now:
- `gbrain whoknows "Q3 roadmap discussion"` routes through the meeting type, ranking by `expert_routing` signal (attendees, recency, salience) instead of raw text.
- The `extract_facts` cycle runs on every meeting page automatically (because `extractable: true`), pulling typed facts like `attended_by=alice-example`, `date=2026-05-23`.
- `gbrain extract-facts` runs on every meeting page automatically (because `extractable: true`), pulling typed facts like `attended_by=alice-example`, `date=2026-05-23`.
- The downstream `think` skill can now answer "what did we decide about pricing in the last three roadmap meetings" by querying the meeting graph instead of grep'ing 4000 files.
One command. 4000 pages went from invisible to queryable. The content didn't change. The structure did.
@@ -2346,7 +2346,7 @@ gbrain schema add-link-type led-by --page-type deal --target-type inves
gbrain schema sync --apply
```
Now `gbrain whoknows "Series A SaaS"` routes through `investor` and `portco` types specifically, not the noisy general type set. `gbrain graph-query alice-example --type intro-from --depth 2` walks two hops of intros to surface "Alice introduced you to Bob who introduced you to Charlie." The `extract_facts` cycle starts producing typed claims from the fence in your deal pages: `(deals/acme-seed, raise=2000000, valuation=15000000, lead=widget-vc, closed_at=2026-05-23)`.
Now `gbrain whoknows "Series A SaaS"` routes through `investor` and `portco` types specifically, not the noisy general type set. `gbrain graph-query alice-example --type intro-from --depth 2` walks two hops of intros to surface "Alice introduced you to Bob who introduced you to Charlie." `gbrain extract-facts` starts producing typed claims from the fence in your deal pages: `(deals/acme-seed, raise=2000000, valuation=15000000, lead=widget-vc, closed_at=2026-05-23)`.
The CRM you've been promising yourself you'll set up next quarter? You just shipped it in 4 commands. It's downstream of your notes, not parallel to them.
@@ -2427,7 +2427,7 @@ Re-run the same `whoknows` query. Top-3 should shift, because the new type is no
Three things gbrain does that generic note systems can't:
**1. The brain knows the difference between a person and an idea.** Page-type matters at query time. `gbrain whoknows` only considers `expert_routing: true` types. The `extract_facts` cycle only runs on `extractable: true` types. `gbrain graph-query` walks declared link verbs. None of that works on a flat tag system because tags don't have semantics — they're labels. Types are first-class citizens with rules attached.
**1. The brain knows the difference between a person and an idea.** Page-type matters at query time. `gbrain whoknows` only considers `expert_routing: true` types. `gbrain extract-facts` only runs on `extractable: true` types. `gbrain graph-query` walks declared link verbs. None of that works on a flat tag system because tags don't have semantics — they're labels. Types are first-class citizens with rules attached.
**2. Untyped content is invisible content.** If your meetings are typed as `note`, expert routing skips them, facts extraction ignores them, link inference doesn't fire. They exist on disk and they're indexed for text search, but the structural surfaces (whoknows, find_experts, recall, think) treat them as second-class. Adding a type isn't cosmetic; it's structural promotion.
@@ -3897,7 +3897,7 @@ All 30 GBrain operations are available remotely, including `sync_brain` and
directory where `gbrain serve` was launched. Symlinks, `..` traversal, and absolute
paths outside cwd are rejected. Page slugs and filenames are allowlist-validated
(alphanumeric + hyphens; no control chars, RTL overrides, or backslashes). Local
CLI callers (`gbrain files upload ...`) keep unrestricted filesystem access since
CLI callers (`gbrain file upload ...`) keep unrestricted filesystem access since
the user owns the machine.
## Deployment Options
+1 -1
View File
@@ -248,7 +248,7 @@ before submission.
After the brain page is written, render to PDF using `skills/brain-pdf`:
```bash
gbrain put # already done by the CLI; nothing to add here
gbrain put_page # already done by the CLI; nothing to add here
# Then invoke brain-pdf:
# (see skills/brain-pdf/SKILL.md for the make-pdf invocation)
```
+6 -6
View File
@@ -73,13 +73,13 @@ stock worker auto-loads on startup) registers handlers before `start()`.
Users who set `minion_mode: off` in `~/.gbrain/preferences.json` keep
using `agentTurn`. Respect that. No auto-rewrite.
## Forward note
## Forward note (v0.12.0)
A native scheduler loop inside `gbrain jobs work` (owning cron
expressions directly, with no host-scheduler hand-off) has been on the
roadmap since v0.11.1 but has not shipped. The host scheduler keeps
firing on schedule; this convention only replaces the execution layer
(what the cron trigger *does*), not the scheduling layer.
GBrain v0.12.0 ships `gbrain cron`: a scheduler loop inside
`gbrain jobs work` that owns cron expressions natively — no more
handing off to host schedulers. Until v0.12.0 lands, the host
scheduler keeps firing on schedule; v0.11.1 only replaces the execution
layer (what the cron trigger *does*), not the scheduling layer.
## Related
+2 -2
View File
@@ -54,8 +54,8 @@ Ask the user what they want to track. Either:
- Define a custom recipe with: source queries, classification rules, extraction schema,
tracker page path, tracker format
Recipes are YAML files at `~/.gbrain/recipes/{name}.yaml`. Scaffold a new one by
copying a built-in recipe file and editing its fields.
Recipes are YAML files at `~/.gbrain/recipes/{name}.yaml`. Use `gbrain research init`
to scaffold a new one.
### Phase 2: Search Sources
+1 -1
View File
@@ -201,7 +201,7 @@ Use the brain page template. MUST include:
### 4b. Entity pages (people, companies)
For each entity mentioned:
- Check if a brain page exists (`gbrain search "<name>"` or `gbrain get people/<slug>`).
- Check if a brain page exists (`gbrain search "<name>"` or `gbrain get_page people/<slug>`).
- If exists: update State, append Timeline entry citing this research.
- If not: create with enrichment.
+1 -1
View File
@@ -112,7 +112,7 @@ gbrain query "<topic keywords>"
# -d '{"model": "sonar-pro", "messages": [{"role":"user","content":"..."}]}'
# 4. Write the structured research page via put_page:
gbrain put research/<slug> # via the put_page operation
gbrain put_page research/<slug> # via the put_page operation
# 5. Cross-link entities mentioned (people, companies) per Iron Law.
```
+4 -4
View File
@@ -11,7 +11,7 @@ tools:
- gbrain schema active
- gbrain schema use
- gbrain schema stats
- gbrain restore
- gbrain pages restore
- mcp:run_onboard
triggers:
- "unify my types"
@@ -143,7 +143,7 @@ WHERE source_id = 'default' AND frontmatter->>'legacy_type' IS NOT NULL;
Page-to-alias and page-to-link source pages soft-delete with 72h TTL. Restore within that window:
```bash
gbrain restore <slug>
gbrain pages restore <slug>
```
Revert the active pack flip:
@@ -197,7 +197,7 @@ Outputs:
- Active pack flipped to `gbrain-base-v2` atomically at end of successful run.
Side effects:
- Source pages soft-deleted with 72h restore TTL (`gbrain restore <slug>`).
- Source pages soft-deleted with 72h restore TTL (`gbrain pages restore <slug>`).
- One-time cache invalidation on KNOBS_HASH_VERSION bump (5→6); self-healing in `cache.ttl_seconds`.
- Query-time `--type X` alias-expands via `expandTypeFilter` (D14 back-compat).
@@ -212,7 +212,7 @@ DON'T:
- Submit `unify-types` directly via the MCP `submit_job` op without `--allow-protected`. PROTECTED handlers require trusted local callers; remote MCP rejection is the intentional trust boundary.
- Edit `mapping_rules` in `gbrain-base-v2.yaml` to skip clusters you don't trust. Fork the pack instead (`gbrain schema fork`) so the source-of-truth migration stays consistent across brains.
- Run `unify-types` from inside an autopilot tick. The check is `manual_only` per D17 — autopilot deliberately never auto-fires it because pack upgrades are one-time consenting taxonomy decisions.
- Hard-delete soft-deleted source pages before the 72h restore window. Use `gbrain restore <slug>` first if rollback is needed.
- Hard-delete soft-deleted source pages before the 72h restore window. Use `gbrain pages restore <slug>` first if rollback is needed.
- Assume `frontmatter.legacy_type` survives every roundtrip. The marker is canonical for the immediate post-migration window; downstream re-imports may overwrite it.
## Output Format
+4 -6
View File
@@ -43,9 +43,8 @@ The Analysis section can interpret; the transcript section is sacred.
The user sends an audio or voice message via any channel (Telegram, voice
memo upload, openclaw audio attachment). The host agent typically provides
the transcript text. If not, transcribe it with your host's transcription
tool (Groq Whisper is fast and cheap; OpenAI Whisper works too — segment
audio > 25MB via ffmpeg first).
the transcript text. If not, transcribe via `gbrain transcription` (Groq
Whisper by default; OpenAI fallback for audio > 25MB segmented via ffmpeg).
## The pipeline
@@ -53,9 +52,8 @@ audio > 25MB via ffmpeg first).
1. STORE → Upload original audio to gbrain storage backend
(S3 / Supabase Storage / local — pluggable per
src/core/storage.ts).
2. TRANSCRIBE → Use the agent-provided transcript verbatim, OR
transcribe the audio yourself (see "When to invoke")
if no transcript was supplied.
2. TRANSCRIBE → Use the agent-provided transcript verbatim, OR call
gbrain transcription if no transcript was supplied.
3. ROUTE → Apply the decision tree (below) to find the right
destination directory.
4. WRITE → Create / update the destination brain page; preserve the
+13 -125
View File
@@ -9,7 +9,7 @@ installSigchldHandler();
import { installSignalHandlers as installCleanupSignalHandlers } from './core/process-cleanup.ts';
installCleanupSignalHandlers();
import { readFileSync, existsSync, unlinkSync, fstatSync } from 'fs';
import { readFileSync, existsSync, unlinkSync } from 'fs';
import { spawn } from 'child_process';
import {
readUpdateCache,
@@ -55,17 +55,12 @@ export function bigintToStringReplacer(_key: string, value: unknown): unknown {
}
// CLI-only commands that bypass the operation layer
export const CLI_ONLY = new Set(['init', 'reinit-pglite', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'extract-conversation-facts', 'enrich', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'maintain', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'reconcile-links', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'calibration', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'retrieval-upgrade', 'founder', 'brainstorm', 'lsd', 'schema', 'capture', 'onboard', 'conversation-parser', 'status', 'connect', 'skillopt', 'quarantine', 'self-upgrade', 'advisor', 'watch', 'reindex-search-vector', 'pages', 'bench', 'backfill']);
export const CLI_ONLY = new Set(['init', 'reinit-pglite', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'extract-conversation-facts', 'enrich', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'maintain', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'reconcile-links', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'calibration', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'retrieval-upgrade', 'founder', 'brainstorm', 'lsd', 'schema', 'capture', 'onboard', 'conversation-parser', 'status', 'connect', 'skillopt', 'quarantine', 'self-upgrade', 'advisor', 'watch', 'reindex-search-vector']);
// CLI-only commands whose handlers print their own --help text. These are
// excluded from the generic short-circuit so detailed per-command and
// per-subcommand usage stays reachable.
const CLI_ONLY_SELF_HELP = new Set([
'upgrade', 'post-upgrade', 'check-update',
// #3502 sweep: pages + bench print their own usage (pages.ts printHelp,
// bench-publish.ts printHelp). Both were documented but undispatchable —
// `pages` had a live handleCliOnly case but was missing from CLI_ONLY
// (the #2035 calibration bug class); `bench` was never wired at all.
'pages', 'bench',
'embed', 'config',
'skillpack', 'skillpack-check',
'integrations', 'friction',
@@ -349,11 +344,6 @@ async function main() {
// them out of the engine try/catch is safe and unlocks routing.
const params = parseOpArgs(op, subArgs);
// #3513: stdin fill moved out of parseOpArgs so a non-TTY stdin with no
// piped input can't block the parse forever — the bounded read leaves the
// param unset on timeout and the required-param check below fails fast.
await applyStdinParam(op, params);
// v0.27.1 (`gbrain query --image <path>`): swap the `image` param from
// a filesystem path into base64 bytes + mime. The op accepts base64; the
// CLI accepts a path. Helper is exported so tests can exercise the
@@ -814,99 +804,18 @@ export function parseOpArgs(op: Operation, args: string[]): Record<string, unkno
}
}
return params;
}
/**
* #3513: read stdin into an op's stdin-capable param without ever blocking
* forever. The old inline `readFileSync(0)` in parseOpArgs assumed non-TTY
* implies piped content; a non-TTY stdin with NO input (CI step, cron job,
* agent harness holding an unwritten pipe open) blocked the read until kill.
*
* Strategy by fd kind (fstat):
* - TTY: skip, as before (interactive input is not an op-param source).
* - regular file / /dev/null / anything not a pipe or socket: readFileSync
* returns without blocking (`gbrain put x < file`, `< /dev/null` '').
* - FIFO/socket: stream-read with a deadline on the FIRST byte only. A real
* pipe (`echo foo | gbrain put x`, heredocs) delivers its first byte
* within milliseconds; once any data arrives the deadline is lifted and
* we read to EOF like readFileSync did (slow producers stay supported).
* An empty-but-closed pipe (`: | gbrain put x`) EOFs immediately ''.
* A pipe that never delivers a byte times out param stays unset, so
* the existing required-param usage error fires (fail fast, exit 1).
*
* GBRAIN_STDIN_TIMEOUT_MS overrides the first-byte deadline (default 5000).
* Exported for tests; called by the op dispatch right after parseOpArgs.
*/
export async function applyStdinParam(
op: Operation,
params: Record<string, unknown>,
): Promise<void> {
// Branch shape (stdin hint + missing param + `!process.stdin.isTTY` gate +
// 5MB cap) is pinned by the R4 regression test for PR #1325's Windows fix
// (test/cycle/regression-pr-wave-r1-r2-r4.test.ts) — keep the spelling.
// Read stdin for content params
if (op.cliHints?.stdin && !params[op.cliHints.stdin] && !process.stdin.isTTY) {
const content = await readStdinBounded();
if (content === null) return; // no input arrived — let the required-param check fail fast
const stdinContent = readFileSync(0, 'utf-8');
const MAX_STDIN = 5_000_000; // 5MB
if (Buffer.byteLength(content, 'utf-8') > MAX_STDIN) {
if (Buffer.byteLength(stdinContent, 'utf-8') > MAX_STDIN) {
console.error(`Error: stdin content exceeds ${MAX_STDIN} bytes. Split into smaller inputs.`);
process.exit(1);
}
params[op.cliHints.stdin] = content;
params[op.cliHints.stdin] = stdinContent;
}
}
/** First-byte deadline for pipe/socket stdin (#3513). Env-overridable escape hatch. */
function stdinFirstByteTimeoutMs(): number {
const n = Number(process.env.GBRAIN_STDIN_TIMEOUT_MS);
return Number.isFinite(n) && n > 0 ? n : 5000;
}
/**
* Returns the full stdin content, '' for a readable-but-empty stdin, or
* null when stdin is a pipe/socket that never delivered a byte within the
* first-byte deadline (or the fd is closed/unreadable).
*/
export async function readStdinBounded(): Promise<string | null> {
let isPipeOrSocket: boolean;
try {
const st = fstatSync(0);
isPipeOrSocket = st.isFIFO() || st.isSocket();
} catch {
return null; // closed/invalid fd — treat as no input
}
if (!isPipeOrSocket) {
// Regular file redirect, /dev/null, etc. — read returns without blocking.
try {
return readFileSync(0, 'utf-8');
} catch {
return null;
}
}
return await new Promise<string | null>((resolve) => {
const chunks: Buffer[] = [];
let gotData = false;
const timer = setTimeout(() => {
if (!gotData) {
process.stdin.destroy();
resolve(null);
}
}, stdinFirstByteTimeoutMs());
const finish = () => {
clearTimeout(timer);
resolve(Buffer.concat(chunks).toString('utf-8'));
};
process.stdin.on('data', (c: Buffer) => {
if (!gotData) {
gotData = true;
clearTimeout(timer); // deadline applies to the FIRST byte only
}
chunks.push(c);
});
process.stdin.once('end', finish);
process.stdin.once('error', finish);
});
return params;
}
/**
@@ -963,8 +872,7 @@ export function applyThinClientSourceScope(
params.source_id = resolved;
}
// Exported for tests (same import-safety contract as applyThinClientSourceScope).
export async function makeContext(engine: BrainEngine, params: Record<string, unknown>): Promise<OperationContext> {
async function makeContext(engine: BrainEngine, params: Record<string, unknown>): Promise<OperationContext> {
// v0.31.8 (D11): resolve sourceId via the canonical 6-tier chain. Honors
// --source / GBRAIN_SOURCE / .gbrain-source / path-match / brain default /
// 'default'. Wrapped in try/catch so a doctor / single-source brain that
@@ -976,21 +884,16 @@ export async function makeContext(engine: BrainEngine, params: Record<string, un
// trusted local boundary) and consumed by federatedSearchScope in
// operations.ts, which additionally gates on ctx.remote === false.
let localFederated: string[] | undefined;
// params.source is set when a CLI flag was parsed for the op (rare; most
// CLI ops don't take --source). Falls through to env/dotfile/path-match.
const explicit = (params.source as string | undefined) ?? null;
try {
const { resolveSourceWithTier, localFederatedSourceIds } = await import('./core/source-resolver.ts');
// params.source is set when a CLI flag was parsed for the op (rare; most
// CLI ops don't take --source). Falls through to env/dotfile/path-match.
const explicit = (params.source as string | undefined) ?? null;
const resolved = await resolveSourceWithTier(engine, explicit);
sourceId = resolved.source_id;
localFederated = await localFederatedSourceIds(engine, resolved.source_id, resolved.tier);
} catch (err) {
// #1712: an EXPLICIT --source that fails to resolve (invalid id, or a
// source that doesn't exist) must error loudly — the blanket swallow
// turned `--source __all__` and typos into a silent `default` scope,
// which is how three bug reports became debugging sessions.
if (explicit) throw err;
// Ambient resolution failed (e.g. sources table doesn't exist on a fresh
} catch {
// Source resolution failed (e.g. sources table doesn't exist on a fresh
// pre-init brain). Leave sourceId unset; engine read methods fall through
// to the cross-source view (D16 back-compat path).
sourceId = undefined;
@@ -1270,20 +1173,6 @@ async function handleCliOnly(command: string, args: string[]) {
await runInit(args);
return;
}
if (command === 'bench') {
// #3502 sweep: `gbrain bench publish` was documented (docs/eval-bench.md,
// KEY_FILES.md, and eval-gate's own --help text) but never dispatched —
// the promised-but-unwired class retrieval-upgrade (#3390) fixed before.
// Pure file-in/file-out (NDJSON → baseline); no DB, no engine.
if (args[0] === 'publish') {
const { runBenchPublish } = await import('./commands/bench-publish.ts');
await runBenchPublish(args.slice(1));
return;
}
console.error('Usage: gbrain bench publish --from <captured.ndjson> --to <X.baseline.ndjson> [flags]');
console.error('Run `gbrain bench publish --help` for the full flag list.');
process.exit(args[0] === '--help' || args[0] === '-h' ? 0 : 2);
}
// v0.37 fix wave (deferred TODO, shipped): one-command wipe-and-reinit.
// Spawns its own engine internally so no pre-bound engine needed.
if (command === 'reinit-pglite') {
@@ -2559,7 +2448,6 @@ TOOLS
publish <page.md> [--password] Shareable HTML (strips private data, optional AES-256)
check-backlinks <check|fix> [dir] Find/fix missing back-links across brain
lint <dir|file> [--fix] Catch LLM artifacts, placeholder dates, bad frontmatter
backfill <kind|list> v0.30.1: run a registered backfill (effective-date, ...)
orphans [--json] [--count] Find pages with no inbound wikilinks
salience [--days N] [--kind P] v0.29: pages ranked by emotional + activity salience
anomalies [--since D] [--sigma N] v0.29: cohort-based statistical anomalies (tag, type)
+3 -50
View File
@@ -19,26 +19,6 @@ import {
} from '../core/pace-mode.ts';
import { tryAcquireDbLock, type DbLockHandle } from '../core/db-lock.ts';
import { embedBackfillLockId } from '../core/embed-backfill-lock.ts';
import { wrapChunkTextsForStoredMode } from '../core/embedding-context.ts';
import { titleTierCorpusGeneration } from '../core/contextual-retrieval-service.ts';
import type { Page } from '../core/types.ts';
/**
* #3507 after a plain re-embed fully re-embedded a `per_chunk_synopsis`
* page at the title-only tier (see wrapChunkTextsForStoredMode), restamp the
* page's CR state to 'title' so `contextual_retrieval_mode` keeps describing
* the vectors actually in the column. The reindex sweep restores the synopsis
* tier later. No-op for every other mode.
*/
export async function restampIfDemotedToTitleTier(
engine: BrainEngine,
page: Pick<Page, 'contextual_retrieval_mode'> | null | undefined,
slug: string,
sourceId: string,
): Promise<void> {
if (page?.contextual_retrieval_mode !== 'per_chunk_synopsis') return;
await engine.updatePageContextualRetrievalState(slug, sourceId, 'title', titleTierCorpusGeneration());
}
export interface EmbedOpts {
/** Embed ALL pages (every chunk). */
@@ -619,11 +599,7 @@ async function embedPage(
return;
}
// #3507: embed with the page's STORED wrapping convention (title-tier
// contextual prefix when the page was embedded wrapped), not raw
// chunk_text — otherwise a re-embed silently strips the contextual
// prefixes the sync path applied. fenced_code chunks stay unwrapped.
const embeddings = await embedBatch(wrapChunkTextsForStoredMode(page, toEmbed), { abortSignal: signal });
const embeddings = await embedBatch(toEmbed.map(c => c.chunk_text), { abortSignal: signal });
const embeddingMap = new Map<number, Float32Array>();
for (let j = 0; j < toEmbed.length; j++) {
embeddingMap.set(toEmbed[j].chunk_index, embeddings[j]);
@@ -646,9 +622,6 @@ async function embedPage(
// such a page and then stamps it.
if (toEmbed.length === chunks.length) {
await engine.setPageEmbeddingSignature(slug, { sourceId, signature: currentEmbeddingSignature() });
// #3507: a fully re-embedded per_chunk_synopsis page landed at the
// title tier — keep the stamped mode honest.
await restampIfDemotedToTitleTier(engine, page, slug, page.source_id);
}
result.embedded += toEmbed.length;
result.pages_processed++;
@@ -790,8 +763,7 @@ async function embedAll(
}
try {
// #3507: reproduce the page's stored wrapping convention (see embedPage).
const embeddings = await embedBatch(wrapChunkTextsForStoredMode(page, toEmbed));
const embeddings = await embedBatch(toEmbed.map(c => c.chunk_text));
// Build a map of new embeddings by chunk_index
const embeddingMap = new Map<number, Float32Array>();
for (let j = 0; j < toEmbed.length; j++) {
@@ -813,11 +785,6 @@ async function embedAll(
await observed(pacer, () =>
engine.setPageEmbeddingSignature(page.slug, { sourceId: pageSourceId, signature }),
);
// #3507: --all fully re-embeds; a per_chunk_synopsis page landed at
// the title tier — keep the stamped mode honest.
await observed(pacer, () =>
restampIfDemotedToTitleTier(engine, page, page.slug, pageSourceId),
);
result.embedded += toEmbed.length;
} catch (e: unknown) {
serr(`\n Error embedding ${page.slug}: ${e instanceof Error ? e.message : e}`);
@@ -1131,13 +1098,7 @@ async function embedAllStale(
const keySourceId = stale[0]?.source_id ?? 'default';
const slug = stale[0].slug;
try {
// #3507: fetch the page row for its title + stored CR mode so the
// re-embed reproduces the page's wrapping convention instead of
// silently stripping contextual prefixes — `embed --stale` is the
// NORMAL post-model-migration path, so raw-text embedding here
// quietly converted whole corpora to the unwrapped convention.
const pageRow = await observed(pacer, () => engine.getPage(slug, { sourceId: keySourceId }));
const embeddings = await embedBatchWithBackoff(wrapChunkTextsForStoredMode(pageRow, stale), { abortSignal: effectiveSignal });
const embeddings = await embedBatchWithBackoff(stale.map(c => c.chunk_text), { abortSignal: effectiveSignal });
// Re-fetch existing chunks and merge to avoid deleting non-stale chunks.
const existing = await observed(pacer, () => engine.getChunks(slug, { sourceId: keySourceId }));
const staleIdxToEmbedding = new Map<number, Float32Array>();
@@ -1165,14 +1126,6 @@ async function embedAllStale(
engine.setPageEmbeddingSignature(slug, { sourceId: keySourceId, signature }),
);
}
// #3507: a FULLY re-embedded per_chunk_synopsis page landed at the
// title tier — keep the stamped mode honest. Partially-stale pages
// stay stamped as-is (mixed provenance; reindex sweeps fix them).
if (stale.length === existing.length) {
await observed(pacer, () =>
restampIfDemotedToTitleTier(engine, pageRow, slug, keySourceId),
);
}
result.embedded += stale.length;
} catch (e: unknown) {
// Budget/abort-fired cancellations are expected on the way out; don't
+3 -10
View File
@@ -233,7 +233,7 @@ USAGE
gbrain jobs get <id>
gbrain jobs cancel <id>
gbrain jobs retry <id>
gbrain jobs prune [--older-than 30d] [--dry-run]
gbrain jobs prune [--older-than 30d]
gbrain jobs delete <id>
gbrain jobs stats
gbrain jobs smoke
@@ -633,15 +633,8 @@ HANDLER TYPES (built in)
try { await queue.ensureSchema(); }
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
// #2712: --dry-run previews the count without deleting. It used to be
// silently ignored (the destructive default ran anyway).
const dryRun = hasFlag(args, '--dry-run');
const count = await queue.prune({ olderThan: new Date(Date.now() - days * 86400000), dryRun });
if (dryRun) {
console.log(`[dry-run] Would prune ${count} jobs older than ${days} days. Nothing deleted.`);
} else {
console.log(`Pruned ${count} jobs older than ${days} days.`);
}
const count = await queue.prune({ olderThan: new Date(Date.now() - days * 86400000) });
console.log(`Pruned ${count} jobs older than ${days} days.`);
break;
}
+8 -15
View File
@@ -3,7 +3,6 @@
*
* Subcommands:
* takes <slug> list takes for a page
* takes list list all active takes (#2079)
* takes search "<query>" [--who h] keyword search across all takes
* takes add <slug> ...flags append a take (markdown + DB)
* takes update <slug> --row N ...flags update mutable fields
@@ -130,10 +129,11 @@ function writeBody(path: string, body: string): void {
// --- Subcommands ---
async function cmdList(engine: BrainEngine, args: string[]): Promise<void> {
// #2079: slug is optional. `gbrain takes list` (no slug) lists ALL active
// takes — CLI parity with the takes_list operation. A leading flag is not
// a slug.
const slug = args[0] && !args[0].startsWith('-') ? args[0] : undefined;
const slug = args[0];
if (!slug) {
console.error('Usage: gbrain takes <slug> [--json]');
process.exit(1);
}
const json = flagPresent(args, '--json');
const holder = flagValue(args, '--who');
const kind = flagValue(args, '--kind') as string | undefined;
@@ -153,19 +153,17 @@ async function cmdList(engine: BrainEngine, args: string[]): Promise<void> {
return;
}
const scope = slug ?? 'this brain';
if (takes.length === 0) {
console.log(`No takes on ${scope}.`);
console.log(`No takes on ${slug}.`);
return;
}
console.log(`# Takes on ${scope}\n`);
console.log(`# Takes on ${slug}\n`);
for (const t of takes) {
const tag = t.active ? '' : ' [superseded]';
const w = Number(t.weight).toFixed(2);
const since = t.since_date ?? '';
const src = t.source ? `${t.source}` : '';
const where = slug ? '' : `${t.page_slug} `;
console.log(`${where}#${t.row_num} [${t.kind}${t.holder} • w=${w}${since ? `${since}` : ''}]${tag}\n ${t.claim}${src}\n`);
console.log(`#${t.row_num} [${t.kind}${t.holder} • w=${w}${since ? `${since}` : ''}]${tag}\n ${t.claim}${src}\n`);
}
}
@@ -557,8 +555,6 @@ export async function runTakes(engine: BrainEngine, args: string[]): Promise<voi
Subcommands:
takes <slug> [--json] [--who h] [--kind k] [--sort weight|since_date|created_at] [--expired]
List takes for a page
takes list [--json] [--who h] [--kind k] [--sort ...] [--expired]
List all active takes across the brain (#2079)
takes search "<query>" [--limit N] [--json]
Keyword search across all takes
takes add <slug> --claim "..." --kind <fact|take|bet|hunch> --who <holder>
@@ -588,9 +584,6 @@ Common flags:
const rest = args.slice(1);
switch (sub) {
// #2079: `takes list` used to be parsed as page slug "list" and printed
// "No takes on list." — reading exactly like an empty takes table.
case 'list': return cmdList(engine, rest);
case 'search': return cmdSearch(engine, rest);
case 'add': return cmdAdd(engine, rest, await resolveTakesSourceId(engine));
case 'update': return cmdUpdate(engine, rest, await resolveTakesSourceId(engine));
+2 -63
View File
@@ -642,42 +642,8 @@ function warnRecipesMissingBatchTokens(): void {
}
}
/**
* Test-only reset baseline (#3554). The bunfig preload
* (`test/helpers/legacy-embedding-preload.ts`) pins the gateway to the legacy
* OpenAI/1536 config at process start, but `resetGateway()` used to wipe that
* pin to `_config = null`. The next test file's engine connect then
* reconfigured from the SHIPPED default (zembed-1 @ 1280) and every 1536-d
* fixture in that file exploded with `expected 1280 dimensions, not 1536`
* a cross-file mine whose placement depended on shard bin-packing.
*
* When a baseline factory is registered, `resetGateway()` means "back to the
* test baseline" instead of "unconfigured": it clears everything as before,
* then re-applies the factory's config via `configureGateway()`. A factory
* (not a frozen config) so each re-application captures fresh
* `process.env`, matching the preload's original `applyLegacy()` semantics.
*
* Production is untouched: nothing in `src/` calls `resetGateway()` or this
* setter, so in production the baseline is never registered and
* `resetGateway()` still fully unconfigures. Same `__*ForTests` seam
* convention as `__setEmbedTransportForTests` above.
*/
let _resetBaseline: (() => AIGatewayConfig) | null = null;
/**
* Register (or clear, with `null`) the config factory that `resetGateway()`
* re-applies. Called once by the bunfig test preload.
*
* @internal exported for tests; not part of the public gateway API.
*/
export function __setGatewayResetBaselineForTests(
factory: (() => AIGatewayConfig) | null,
): void {
_resetBaseline = factory;
}
/** Clear every piece of module state. Shared by both reset flavors. */
function clearGatewayState(): void {
/** Reset (for tests). */
export function resetGateway(): void {
_config = null;
_modelCache.clear();
_shrinkState.clear();
@@ -689,33 +655,6 @@ function clearGatewayState(): void {
_extendedModels.clear();
}
/**
* Reset (for tests). Clears all module state (config, model cache, shrink
* state, transports, warned recipes, extended models), then if a test
* baseline is registered re-applies it so the gateway returns to the
* process-wide test default instead of an unconfigured limbo (#3554).
*/
export function resetGateway(): void {
clearGatewayState();
// configureGateway re-clears _modelCache/_shrinkState/_extendedModels and
// registers the baseline's models; transports are NOT touched by it, so a
// stale test transport can never leak back in through this path.
if (_resetBaseline) configureGateway(_resetBaseline());
}
/**
* Reset AND stay unconfigured, ignoring any registered baseline. For the
* handful of tests that assert genuine no-gateway behavior
* (`no_gateway_config` diagnosis, `isAvailable() === false`, graceful
* degradation paths). The preload's per-test beforeEach restores the
* baseline before the next test, so this cannot leak across tests.
*
* @internal exported for tests; not part of the public gateway API.
*/
export function __unconfigureGatewayForTests(): void {
clearGatewayState();
}
/**
* Test-only seam. Replaces the function the gateway calls to embed a
* sub-batch. Pass `null` to restore the real `embedMany` from the AI SDK.
+5 -28
View File
@@ -21,35 +21,12 @@ export const CJK_SLUG_CHARS = '一-鿿぀-ゟ゠-ヿ가-힯';
export const CJK_RANGES_REGEX = new RegExp(`[${CJK_SLUG_CHARS}]`);
/**
* Slug "word" character class (#3417): every script's letters, not just
* Latin + CJK. Unicode property escapes REQUIRES the `u` flag on any
* regex composed from this string (without `u`, `\p{Ll}` silently matches
* the literal chars `p`, `L`, `l`, `{`, `}`).
*
* \p{Ll} lowercase letters (a-z, Cyrillic/Greek lowercase, đ, )
* \p{Lm} modifier letters
* \p{Lo} caseless-script letters (Hebrew, Arabic, Thai, CJK, Devanagari, )
* \p{M} combining marks that survive the Latin accent-strip pass
* (Hebrew niqqud, Arabic harakat, Thai/Devanagari vowel signs)
* \p{N} numbers (0-9, Arabic-Indic digits, )
*
* Uppercase (\p{Lu}/\p{Lt}) is deliberately excluded: slugifySegment()
* lowercases before filtering, so validators stay lowercase-canonical.
*
* Distinct from CJK_SLUG_CHARS above, which also drives the
* countCJKAwareWords density heuristic do NOT merge the two, or slug
* grammar changes silently change chunking behavior.
* Page-slug segment grammar (no anchors): alnum-or-CJK lead char, then
* alnum/CJK/hyphen continuation. Single source for validatePageSlug
* (operations.ts), SlugRegistry's SLUG_RE, and the dream-cycle
* SUMMARY_SLUG_RE so every slug validator shares one grammar (#738).
*/
export const SLUG_WORD_CHARS = '\\p{Ll}\\p{Lm}\\p{Lo}\\p{M}\\p{N}';
/**
* Page-slug segment grammar (no anchors): word-char lead, then word-char or
* hyphen continuation. Single source for validatePageSlug (operations.ts),
* SlugRegistry's SLUG_RE, and the dream-cycle SUMMARY_SLUG_RE so every slug
* validator shares one grammar (#738). Compose with the `u` flag see
* SLUG_WORD_CHARS.
*/
export const PAGE_SLUG_SEG = `[${SLUG_WORD_CHARS}][${SLUG_WORD_CHARS}\\-]*`;
export const PAGE_SLUG_SEG = `[a-z0-9${CJK_SLUG_CHARS}][a-z0-9${CJK_SLUG_CHARS}\\-]*`;
export const CJK_SENTENCE_DELIMITERS = ['。', '', '']; // 。!?
export const CJK_CLAUSE_DELIMITERS = ['', '', '', '、']; // ;:,、
-13
View File
@@ -145,19 +145,6 @@ export function computeCorpusGeneration(args: {
return h.digest('hex').slice(0, 16);
}
/**
* #3507 the corpus_generation a page lands on when a plain re-embed path
* (`embed --stale` and friends) re-embeds a `per_chunk_synopsis` page at the
* title-only tier (the D14 fallback tier; synopsis re-generation is a paid
* backfill concern). Callers restamp
* `updatePageContextualRetrievalState(slug, sourceId, 'title', titleTierCorpusGeneration())`
* so the stamped mode keeps describing the vectors actually in the column.
* Matches what the inline import path writes for its title-tier pages.
*/
export function titleTierCorpusGeneration(): string {
return computeCorpusGeneration({ crMode: 'title', haikuModel: DEFAULT_HAIKU_MODEL });
}
/**
* Compute source_text_hash for D27 P1-4 cache key composition. The
* synopsis cache invalidates correctly when adjacent text changes (page
+1 -3
View File
@@ -1179,9 +1179,7 @@ async function runPhaseExtractFacts(
summary: `extract_facts skipped: ${result.legacyRowsPending} legacy v0.31 facts pending fence backfill`,
details: {
legacyRowsPending: result.legacyRowsPending,
// A bare `apply-migrations --yes` no-ops once the v0.32.2 ledger
// entry is complete; the retry marker is what re-runs Phase B.
hint: 'gbrain apply-migrations --force-retry 0.32.2 && gbrain apply-migrations --yes',
hint: 'gbrain apply-migrations --yes',
warnings: result.warnings,
},
};
+11 -32
View File
@@ -26,17 +26,13 @@
*
* Empty-fence guard (Codex R2-#7; #2484; #2646): the phase refuses to do
* its destructive reconciliation pass when genuinely-backfillable legacy
* rows still exist in THIS run's source only (`source_id = sourceId`;
* a pending row in source A must not jam extraction for source B the
* source-isolation invariant) `row_num IS NULL` (never fenced) AND
* `entity_slug` resolves to a live page in this source (so the v0_32_2
* migration's Phase B could fence them) AND the row is not soft-expired
* (`expired_at IS NULL`). Status returns `warn` with a hint to re-run
* the v0.32.2 fence backfill (`apply-migrations --force-retry 0.32.2`
* then `--yes` a bare `--yes` is a no-op once the ledger says
* complete). Without the guard, an interrupted upgrade where v0_32_2
* hasn't run could leave the cycle silently misreporting "0 facts on
* people/alice" while legacy rows linger.
* rows still exist `row_num IS NULL` (never fenced) AND `entity_slug`
* resolves to a live page in this source (so the v0_32_2 migration's
* Phase B could fence them) AND the row is not soft-expired
* (`expired_at IS NULL`). Status returns `warn` with a hint to run
* `gbrain apply-migrations --yes`. Without the guard, an interrupted
* upgrade where v0_32_2 hasn't run could leave the cycle silently
* misreporting "0 facts on people/alice" while legacy rows linger.
*
* The live-page requirement (#2484) is load-bearing: the inline facts
* writer keeps producing `row_num IS NULL, entity_slug IS NOT NULL`
@@ -229,17 +225,10 @@ export async function runExtractFacts(
// soft-expires legacy rows rather than deleting them, so counting
// expired rows would leave the guard permanently stuck with no
// supported way to drain the backlog.
//
// Source isolation (#3526): the count is scoped to THIS run's
// sourceId. The pre-fix query counted brain-wide, so a single pending
// legacy row in any mounted source jammed extract_facts for every
// source — a cross-source leak of one source's migration state into
// another's cycle (CLAUDE.md source-isolation invariant).
const legacy = await engine.executeRaw<{ n: string }>(
`SELECT COUNT(*) AS n
FROM facts f
WHERE f.source_id = $1
AND f.row_num IS NULL
WHERE f.row_num IS NULL
AND f.entity_slug IS NOT NULL
AND f.expired_at IS NULL
AND EXISTS (
@@ -248,25 +237,15 @@ export async function runExtractFacts(
AND p.slug = f.entity_slug
AND p.deleted_at IS NULL
)`,
[sourceId],
);
const legacyCount = parseInt(legacy[0]?.n ?? '0', 10);
result.legacyRowsPending = legacyCount;
if (legacyCount > 0) {
result.guardTriggered = true;
// Drain advice must actually work: a bare `apply-migrations --yes`
// is a no-op once the v0.32.2 ledger entry says complete (the
// runner classifies it as already-applied), so the sanctioned
// re-run path is the explicit retry marker first. Phase B is
// idempotent — it only touches `row_num IS NULL` rows and de-dupes
// against the existing fence — so the re-run is safe. Individual
// rows can instead be drained through `forget_fact` (soft-expired
// rows stop counting).
result.warnings.push(
`extract_facts: ${legacyCount} legacy v0.31 fact rows in source "${sourceId}" ` +
`(entity page present, not yet fenced) pending fence backfill. Re-run the v0.32.2 ` +
`fence backfill: \`gbrain apply-migrations --force-retry 0.32.2\` then ` +
`\`gbrain apply-migrations --yes\`. Or drain individual rows via \`forget_fact\`.`,
`extract_facts: ${legacyCount} legacy v0.31 fact rows (entity page present, not yet ` +
`fenced) pending fence backfill. Run \`gbrain apply-migrations --yes\` to complete ` +
`v0_32_2 before this phase can safely reconcile fence → DB.`,
);
return result;
}
+2 -3
View File
@@ -48,9 +48,8 @@ import { safeSplitIndex } from '../text-safe.ts';
import { PAGE_SLUG_SEG } from '../cjk.ts';
// Slug grammar from validatePageSlug — shared via PAGE_SLUG_SEG (#738).
// Used for the orchestrator-written summary index slug. `u` flag required
// by PAGE_SLUG_SEG's \p{...} classes (#3417).
const SUMMARY_SLUG_RE = new RegExp(`^${PAGE_SLUG_SEG}(\\/${PAGE_SLUG_SEG})*$`, 'u');
// Used for the orchestrator-written summary index slug.
const SUMMARY_SLUG_RE = new RegExp(`^${PAGE_SLUG_SEG}(\\/${PAGE_SLUG_SEG})*$`);
// ── Model context budget (D1, D5, D7, D9) ─────────────────────────────
+2 -17
View File
@@ -19,8 +19,7 @@
import type { BrainEngine } from './engine.ts';
import type { ChunkInput } from './types.ts';
import { embedBatchWithBackoff, restampIfDemotedToTitleTier } from '../commands/embed.ts';
import { wrapChunkTextsForStoredMode } from './embedding-context.ts';
import { embedBatchWithBackoff } from '../commands/embed.ts';
import { type DbPacer, createNoopPacer, observed } from './db-pacer.ts';
import { AbortError } from './abort-check.ts';
@@ -190,15 +189,8 @@ export async function embedStaleForSource(
const keySourceId = stale[0]?.source_id ?? sourceId;
const slug = stale[0].slug;
try {
// #3507: fetch the page row for its title + stored CR mode so the
// re-embed reproduces the page's wrapping convention instead of
// silently stripping contextual prefixes (mirrors
// src/commands/embed.ts:embedAllStale).
const pageRow = await observed(pacer, () =>
engine.getPage(slug, { sourceId: keySourceId }),
);
const embeddings = await embedFn(
wrapChunkTextsForStoredMode(pageRow, stale),
stale.map((c) => c.chunk_text),
{ abortSignal: signal },
);
const existing = await observed(pacer, () =>
@@ -241,13 +233,6 @@ export async function embedStaleForSource(
engine.setPageEmbeddingSignature(slug, { sourceId: keySourceId, signature }),
);
}
// #3507: a FULLY re-embedded per_chunk_synopsis page landed at the
// title tier — keep the stamped mode honest (mixed pages stay as-is).
if (stale.length === existing.length) {
await observed(pacer, () =>
restampIfDemotedToTitleTier(engine, pageRow, slug, keySourceId),
);
}
result.embedded += stale.length;
result.pagesProcessed += 1;
} catch (e: unknown) {
-38
View File
@@ -186,41 +186,3 @@ export function modeRequiresHaiku(mode: CRMode): boolean {
export function modeRequiresWrapper(mode: CRMode): boolean {
return mode !== 'none';
}
/**
* #3507 build the embedding inputs for a re-embed of EXISTING chunk rows,
* reproducing the wrapping convention the page's vectors were originally
* built under (recorded in `pages.contextual_retrieval_mode`).
*
* Used by every plain re-embed path (`embed <slug>`, `embed --all`,
* `embed --stale`, the embed-backfill Minion loop). Before this helper those
* paths embedded raw `chunk_text`, so any re-embed including the NORMAL
* post-model-migration `embed --stale` silently replaced context-wrapped
* vectors with unwrapped ones, degrading retrieval with no signature change
* to show for it.
*
* Convention rules (embed PRESERVES conventions; changing them is
* sync/reindex's job):
* - mode NULL/undefined/'none' raw chunk_text (status quo).
* - mode 'title' title-only prefix (pure string concat).
* - mode 'per_chunk_synopsis' title-only prefix. Re-generating Haiku
* synopses is a paid backfill concern; title-only is the service's own
* documented fallback tier (D14). Callers that fully re-embed a page
* this way should restamp the page to 'title' so the column stays
* honest (see contextual-retrieval-service.ts:titleTierCorpusGeneration).
* - `fenced_code` chunks are NEVER wrapped (D20-T4), same as sync.
*/
export function wrapChunkTextsForStoredMode(
page:
| { title?: string | null; contextual_retrieval_mode?: CRMode | null }
| null
| undefined,
chunks: ReadonlyArray<{ chunk_text: string; chunk_source?: string | null }>,
): string[] {
const mode = page?.contextual_retrieval_mode;
if (mode == null || !modeRequiresWrapper(mode)) {
return chunks.map((c) => c.chunk_text);
}
const prefix = buildContextualPrefix(page?.title ?? '', null);
return chunks.map((c) => wrapChunkForEmbedding(c.chunk_text, prefix, c.chunk_source));
}
+3 -13
View File
@@ -85,21 +85,11 @@ const RUN_ID_SHORT_LEN = 8;
/**
* Truncate a run id to the standard 8-char short form used in slug
* paths. Idempotent passing an already-short id returns it unchanged.
* Non-hex / non-alphanumeric chars survive INSIDE the short form
* (op-checkpoint ids may include dashes or other separators), but
* boundary hyphens are trimmed (#3443): `slugifySegment()` strips
* leading/trailing hyphens during repo sync, so a short form like
* 'propose-' (from propose-<timestamp> run ids) made the DB receipt
* slug and its Git-backed slug disagree writing the receipt through
* to the repo created a normalized sibling instead of materializing
* the existing page. Invariant: slugifySegment(shortRunId(x)) ===
* shortRunId(x) for slug-safe run ids.
* Non-hex / non-alphanumeric chars survive (op-checkpoint ids may
* include dashes or other separators).
*/
export function shortRunId(runId: string): string {
// ponytail: truncation-based discrimination is only as good as the run id's
// first 8 chars; families that need per-run uniqueness must front-load it.
const short = runId.slice(0, RUN_ID_SHORT_LEN).replace(/^-+|-+$/g, '');
return short || (runId ? 'run' : '');
return runId.slice(0, RUN_ID_SHORT_LEN);
}
/**
+2 -2
View File
@@ -1092,8 +1092,8 @@ export async function importFromFile(
chunks: 0,
error:
`Filename "${relativePath}" produces no usable slug. ` +
`Add a "slug:" to the frontmatter, or rename the file to include ` +
`at least one letter or number (any script).`,
`Add a "slug:" to the frontmatter, or rename the file to use ` +
`ASCII / Chinese / Japanese / Korean characters.`,
};
}
} else if (parsed.slug !== expectedSlug) {
-5
View File
@@ -43,11 +43,6 @@ export const HANDLER_DEFAULT_TIMEOUT_MS: Readonly<Record<string, number>> = {
// few writes. Generous 10-min budget (vs the tight null-default) covers a
// slow gateway without the 30-min loop budget.
chronicle_extract: TEN_MIN_MS,
// #3207 — same shape as chronicle_extract: one page = one LLM extraction
// call + a few writes. Was missing from this map, so it inherited the tight
// null-default and got dead-lettered mid-generation on slow chat providers
// (facts silently lost) — exactly the failure this file exists to prevent.
'facts-absorb': TEN_MIN_MS,
// Per-page contextual reindex jobs process chunks sequentially with one
// rate-leased LLM synopsis call per chunk; large transcript pages need more
// than the standard 30-min long-job budget.
+1 -12
View File
@@ -534,21 +534,10 @@ export class MinionQueue {
}
/** Prune old jobs in terminal statuses. Returns count of deleted rows. */
async prune(opts?: { olderThan?: Date; status?: MinionJobStatus[]; dryRun?: boolean }): Promise<number> {
async prune(opts?: { olderThan?: Date; status?: MinionJobStatus[] }): Promise<number> {
const statuses = opts?.status ?? ['completed', 'dead', 'cancelled'];
const olderThan = opts?.olderThan ?? new Date(Date.now() - 30 * 86400000);
// #2712: dryRun counts the would-be-pruned rows without deleting.
// Silent-ignoring a safety flag on a delete path is data loss.
if (opts?.dryRun) {
const rows = await this.engine.executeRaw<{ count: string }>(
`SELECT count(*)::text as count FROM minion_jobs
WHERE status = ANY($1) AND updated_at < $2`,
[statuses, olderThan.toISOString()]
);
return parseInt(rows[0]?.count ?? '0', 10);
}
const rows = await this.engine.executeRaw<{ count: string }>(
`WITH pruned AS (
DELETE FROM minion_jobs
+5 -15
View File
@@ -28,7 +28,6 @@ import { isSearchMode } from './search/mode.ts';
import { stampEvidence } from './search/evidence.ts';
import type { SearchResult } from './types.ts';
import { CJK_SLUG_CHARS, PAGE_SLUG_SEG } from './cjk.ts';
import { ALL_SOURCES } from './source-id.ts';
import * as db from './db.ts';
import { VERSION } from '../version.ts';
import {
@@ -163,11 +162,10 @@ export function validatePageSlug(slug: string): void {
if (slug.length > 255) {
throw new OperationError('invalid_params', 'page_slug exceeds 255 characters');
}
// #3417: letters/numbers from any script allowed in segments (u flag required
// for the \p{...} classes in PAGE_SLUG_SEG). Shape rules (lead char, hyphen
// continuation) preserved.
if (!new RegExp(`^${PAGE_SLUG_SEG}(\\/${PAGE_SLUG_SEG})*$`, 'iu').test(slug)) {
throw new OperationError('invalid_params', `Invalid page_slug: ${slug} (allowed: letters/numbers in any script, hyphens, forward-slash separated segments)`);
// v0.32.7: CJK ranges (Han / Hiragana / Katakana / Hangul Syllables) allowed
// in segments. ASCII shape rules (lead char, hyphen continuation) preserved.
if (!new RegExp(`^${PAGE_SLUG_SEG}(\\/${PAGE_SLUG_SEG})*$`, 'i').test(slug)) {
throw new OperationError('invalid_params', `Invalid page_slug: ${slug} (allowed: alphanumeric, CJK, hyphens, forward-slash separated segments)`);
}
}
@@ -488,14 +486,6 @@ export function sourceScopeOpts(ctx: OperationContext): { sourceId?: string; sou
// value of `[]` MUST NOT widen scope to "all sources" by being interpreted
// as "no filter."
if (allowed && allowed.length > 0) return { sourceIds: allowed };
// #1712: the __all__ sentinel spans the brain — but ONLY for trusted local
// callers (strictly `remote === false`). For remote/untrusted callers the
// literal stays as-is: it can never match a real source id (underscores are
// rejected at creation), so the read fail-closes to empty rather than
// widening past the caller's grant. Do NOT "simplify" this to `{}`.
if (ctx.sourceId === ALL_SOURCES) {
return ctx.remote === false ? {} : { sourceId: ctx.sourceId };
}
if (ctx.sourceId) return { sourceId: ctx.sourceId };
return {};
}
@@ -563,7 +553,7 @@ export function resolveRequestedScope(
sourceIdParam: string | undefined,
allSourcesParam = false,
): { sourceId?: string; sourceIds?: string[] } {
const wantsAll = allSourcesParam || sourceIdParam === ALL_SOURCES;
const wantsAll = allSourcesParam || sourceIdParam === '__all__';
if (wantsAll) {
return ctx.remote === false ? {} : sourceScopeOpts(ctx);
}
+2 -3
View File
@@ -72,10 +72,9 @@ export class SlugRegistryError extends Error {
// SlugRegistry
// ---------------------------------------------------------------------------
// Shares the page-slug segment grammar (all scripts, #738/#3417) with
// Shares the page-slug segment grammar (incl. CJK ranges, #738) with
// validatePageSlug; keeps this site's dir/name shape (>= 2 segments).
// `u` flag required by PAGE_SLUG_SEG's \p{...} classes.
const SLUG_RE = new RegExp(`^${PAGE_SLUG_SEG}(\\/${PAGE_SLUG_SEG})+$`, 'u');
const SLUG_RE = new RegExp(`^${PAGE_SLUG_SEG}(\\/${PAGE_SLUG_SEG})+$`);
export class SlugRegistry {
constructor(private engine: BrainEngine) {}
+8 -32
View File
@@ -420,10 +420,8 @@ export class PGLiteEngine implements BrainEngine {
let model: string = DEFAULT_EMBEDDING_MODEL;
try {
const gw = await import('./ai/gateway.ts');
// Both accessors THROW when the gateway is unconfigured (they never
// return falsy), so the catch below is the only fallback path (#3461).
dims = gw.getEmbeddingDimensions();
model = gw.getEmbeddingModel();
model = gw.getEmbeddingModel() || model;
} catch { /* gateway not configured — use defaults */ }
await this.db.exec(getPGLiteSchema(dims, model));
@@ -981,8 +979,7 @@ export class PGLiteEngine implements BrainEngine {
const { rows } = await this.db.query(
`SELECT id, source_id, slug, type, title, compiled_truth, timeline, frontmatter, content_hash, created_at, updated_at, deleted_at,
effective_date, effective_date_source,
source_kind, source_uri, ingested_via, ingested_at,
contextual_retrieval_mode
source_kind, source_uri, ingested_via, ingested_at
FROM pages WHERE ${where.join(' AND ')} LIMIT 1`,
params
);
@@ -2323,26 +2320,15 @@ export class PGLiteEngine implements BrainEngine {
// Provenance fallback for chunks without an explicit `model`: resolve the
// gateway's runtime model, not the compile-time DEFAULT_EMBEDDING_MODEL.
// #3461: getEmbeddingModel() THROWS when unconfigured (never returns
// falsy) — on the throw path fall back to the brain's own
// `config.embedding_model` row, then the compile-time default as the
// last resort. See postgres-engine.ts _upsertChunksOnce for the full
// rationale — pglite mirrors it for parity.
let resolvedModel: string | null = null;
// See postgres-engine.ts _upsertChunksOnce for the full rationale — pglite
// mirrors it for parity.
let resolvedModel: string = DEFAULT_EMBEDDING_MODEL;
try {
const gw = await import('./ai/gateway.ts');
resolvedModel = gw.getEmbeddingModel();
resolvedModel = gw.getEmbeddingModel() || resolvedModel;
} catch {
try {
const cfg = await this.db.query(
`SELECT value FROM config WHERE key = 'embedding_model'`,
);
resolvedModel = ((cfg.rows[0] as { value?: string } | undefined)?.value) ?? null;
} catch {
// config table unreadable — fall through to the compile-time default.
}
// Gateway unconfigured (unit tests / pre-connect): keep the default.
}
if (!resolvedModel) resolvedModel = DEFAULT_EMBEDDING_MODEL;
for (const chunk of chunks) {
const embeddingStr = chunk.embedding
@@ -2395,9 +2381,6 @@ export class PGLiteEngine implements BrainEngine {
// Code-chunk metadata columns follow the same chunk_text-gated CASE pattern as `embedding`
// (#769). Re-chunk trusts EXCLUDED outright; pure re-embed COALESCEs so a caller carrying
// only embedding-shaped fields doesn't clobber metadata to NULL.
//
// #3461: `model` mirrors the `embedding` CASE branch-for-branch so the label always
// describes whichever vector wins the upsert. See postgres-engine.ts for rationale.
await this.db.query(
`INSERT INTO content_chunks ${cols} VALUES ${rowParts.join(', ')}
ON CONFLICT (page_id, chunk_index) DO UPDATE SET
@@ -2411,14 +2394,7 @@ export class PGLiteEngine implements BrainEngine {
THEN EXCLUDED.embedding
ELSE content_chunks.embedding
END,
model = CASE
WHEN EXCLUDED.chunk_text != content_chunks.chunk_text THEN EXCLUDED.model
WHEN content_chunks.embedding IS NULL THEN EXCLUDED.model
WHEN EXCLUDED.embedded_at IS NOT NULL
AND (content_chunks.embedded_at IS NULL OR EXCLUDED.embedded_at > content_chunks.embedded_at)
THEN EXCLUDED.model
ELSE content_chunks.model
END,
model = COALESCE(EXCLUDED.model, content_chunks.model),
token_count = EXCLUDED.token_count,
embedded_at = CASE
WHEN EXCLUDED.chunk_text != content_chunks.chunk_text AND EXCLUDED.embedding IS NULL THEN NULL
+7 -36
View File
@@ -382,10 +382,8 @@ export class PostgresEngine implements BrainEngine {
let model: string = DEFAULT_EMBEDDING_MODEL;
try {
const gw = await import('./ai/gateway.ts');
// Both accessors THROW when the gateway is unconfigured (they never
// return falsy), so the catch below is the only fallback path (#3461).
dims = gw.getEmbeddingDimensions();
model = gw.getEmbeddingModel();
model = gw.getEmbeddingModel() || model;
} catch { /* gateway not yet configured — use defaults */ }
const sqlText = getPostgresSchema(dims, model);
@@ -1033,8 +1031,7 @@ export class PostgresEngine implements BrainEngine {
const rows = await tx`
SELECT id, source_id, slug, type, title, compiled_truth, timeline, frontmatter, content_hash, created_at, updated_at, deleted_at,
effective_date, effective_date_source,
source_kind, source_uri, ingested_via, ingested_at,
contextual_retrieval_mode
source_kind, source_uri, ingested_via, ingested_at
FROM pages
WHERE slug = ${slug} ${sourceCondition} ${deletedCondition}
LIMIT 1
@@ -2440,28 +2437,14 @@ export class PostgresEngine implements BrainEngine {
// hardcoded default (e.g. zeroentropyai:zembed-1) onto rows whose vectors
// were produced by a different, config-resolved model — corrupting the
// provenance that signature-drift staleness + dim-migration logic trust.
//
// #3461: getEmbeddingModel() THROWS when the gateway is unconfigured —
// it never returns falsy — so an `||` guard here is dead code and the
// catch path used to stamp the compile-time default onto rows whose
// vectors came from the config-resolved provider. On the throw path we
// now fall back to the brain's own `config.embedding_model` row (kept
// current by init / migrate / retrieval-upgrade), which names the model
// that actually produced this brain's vectors. The compile-time default
// is the LAST resort (fresh brain whose config row doesn't exist yet).
let resolvedModel: string | null = null;
// Mirrors the resolve-then-fallback pattern used for schema sizing above.
let resolvedModel: string = DEFAULT_EMBEDDING_MODEL;
try {
const gw = await import('./ai/gateway.ts');
resolvedModel = gw.getEmbeddingModel();
resolvedModel = gw.getEmbeddingModel() || resolvedModel;
} catch {
try {
const cfg = await sql`SELECT value FROM config WHERE key = 'embedding_model'`;
resolvedModel = (cfg[0]?.value as string | undefined) ?? null;
} catch {
// config table unreadable — fall through to the compile-time default.
}
// Gateway unconfigured (unit tests / pre-connect): keep the default.
}
if (!resolvedModel) resolvedModel = DEFAULT_EMBEDDING_MODEL;
for (const chunk of chunks) {
const embeddingStr = chunk.embedding
@@ -2525,11 +2508,6 @@ export class PostgresEngine implements BrainEngine {
// pure re-embed (chunk_text unchanged) COALESCEs so a caller that only carries embedding
// doesn't clobber metadata to NULL. Without this, every embed --stale pass nuked code-def's
// primary index for thousands of chunks at once.
//
// #3461: `model` mirrors the `embedding` CASE branch-for-branch — the label must
// describe whichever vector WINS the upsert. The old COALESCE(EXCLUDED.model, …)
// relabeled preserved (older-model) vectors with the current gateway model on every
// partial re-embed, corrupting provenance without changing the vector.
await sql.unsafe(
`INSERT INTO content_chunks ${cols} VALUES ${rows.join(', ')}
ON CONFLICT (page_id, chunk_index) DO UPDATE SET
@@ -2543,14 +2521,7 @@ export class PostgresEngine implements BrainEngine {
THEN EXCLUDED.embedding
ELSE content_chunks.embedding
END,
model = CASE
WHEN EXCLUDED.chunk_text != content_chunks.chunk_text THEN EXCLUDED.model
WHEN content_chunks.embedding IS NULL THEN EXCLUDED.model
WHEN EXCLUDED.embedded_at IS NOT NULL
AND (content_chunks.embedded_at IS NULL OR EXCLUDED.embedded_at > content_chunks.embedded_at)
THEN EXCLUDED.model
ELSE content_chunks.model
END,
model = COALESCE(EXCLUDED.model, content_chunks.model),
token_count = EXCLUDED.token_count,
embedded_at = CASE
WHEN EXCLUDED.chunk_text != content_chunks.chunk_text AND EXCLUDED.embedding IS NULL THEN NULL
-11
View File
@@ -33,17 +33,6 @@
export const SOURCE_ID_RE = /^[a-z0-9](?:[a-z0-9-]{0,30}[a-z0-9])?$/;
/**
* Sentinel meaning "span every source" (#1712). Deliberately NOT a valid
* source id (underscores are rejected by SOURCE_ID_RE), so it can never
* collide with a real source, be created via `sources add`, or leak into
* lock ids / path joins. The resolver's explicit/env tiers pass it through
* verbatim; `sourceScopeOpts` translates it to an unscoped read for trusted
* local callers and keeps it as an unsatisfiable literal for remote callers
* (fail-closed).
*/
export const ALL_SOURCES = '__all__';
/** Returns true if the string matches the canonical source_id regex. */
export function isValidSourceId(s: unknown): s is string {
return typeof s === 'string' && SOURCE_ID_RE.test(s);
+5 -21
View File
@@ -17,13 +17,9 @@ import { readFileSync, lstatSync, type Stats } from 'fs';
import { join, dirname, resolve } from 'path';
import type { BrainEngine } from './engine.ts';
import { isSourceFederated } from './sources-load.ts';
import { SOURCE_ID_RE, isValidSourceId, ALL_SOURCES } from './source-id.ts';
import { SOURCE_ID_RE, isValidSourceId } from './source-id.ts';
import { isTrustedDotfile, realpathOrResolve } from './path-confine.ts';
// Re-export so scope-resolution call sites can import the sentinel from
// either module (#1712).
export { ALL_SOURCES };
const DOTFILE = '.gbrain-source';
// Canonical SOURCE_ID_RE imported from `source-id.ts` (single source of truth).
// Re-exported below as `__testing.SOURCE_ID_RE` for legacy test imports.
@@ -87,11 +83,8 @@ export async function resolveSourceId(
explicit: string | null | undefined,
cwd: string = process.cwd(),
): Promise<string> {
// 1. Explicit flag wins. The __all__ sentinel passes through verbatim
// (#1712) — it is not a source id, so it skips both the regex and
// assertSourceExists; sourceScopeOpts gives it span-everything semantics.
// 1. Explicit flag wins.
if (explicit) {
if (explicit === ALL_SOURCES) return ALL_SOURCES;
if (!SOURCE_ID_RE.test(explicit)) {
throw new Error(`Invalid --source value "${explicit}". Must match [a-z0-9-]{1,32}.`);
}
@@ -99,10 +92,9 @@ export async function resolveSourceId(
return explicit;
}
// 2. Env var. Same __all__ pass-through (#2140).
// 2. Env var.
const env = process.env.GBRAIN_SOURCE;
if (env && env.length > 0) {
if (env === ALL_SOURCES) return ALL_SOURCES;
if (!SOURCE_ID_RE.test(env)) {
throw new Error(`Invalid GBRAIN_SOURCE value "${env}". Must match [a-z0-9-]{1,32}.`);
}
@@ -181,7 +173,6 @@ export function resolveSourceIdEngineFree(
cwd: string = process.cwd(),
): string | null {
if (explicit) {
if (explicit === ALL_SOURCES) return ALL_SOURCES; // #1712 sentinel pass-through
if (!SOURCE_ID_RE.test(explicit)) {
throw new Error(`Invalid --source value "${explicit}". Must match [a-z0-9-]{1,32}.`);
}
@@ -189,7 +180,6 @@ export function resolveSourceIdEngineFree(
}
const env = process.env.GBRAIN_SOURCE;
if (env && env.length > 0) {
if (env === ALL_SOURCES) return ALL_SOURCES; // #2140 sentinel pass-through
if (!SOURCE_ID_RE.test(env)) {
throw new Error(`Invalid GBRAIN_SOURCE value "${env}". Must match [a-z0-9-]{1,32}.`);
}
@@ -325,11 +315,8 @@ export async function resolveSourceWithTier(
explicit: string | null | undefined,
cwd: string = process.cwd(),
): Promise<{ source_id: string; tier: SourceTier; detail?: string }> {
// 1. Explicit flag wins. __all__ sentinel passes through verbatim (#1712).
// 1. Explicit flag wins.
if (explicit) {
if (explicit === ALL_SOURCES) {
return { source_id: ALL_SOURCES, tier: 'flag', detail: `--source ${ALL_SOURCES} (spans all sources)` };
}
if (!SOURCE_ID_RE.test(explicit)) {
throw new Error(`Invalid --source value "${explicit}". Must match [a-z0-9-]{1,32}.`);
}
@@ -337,12 +324,9 @@ export async function resolveSourceWithTier(
return { source_id: explicit, tier: 'flag', detail: `--source ${explicit}` };
}
// 2. Env var. Same __all__ pass-through (#2140).
// 2. Env var.
const env = process.env.GBRAIN_SOURCE;
if (env && env.length > 0) {
if (env === ALL_SOURCES) {
return { source_id: ALL_SOURCES, tier: 'env', detail: `GBRAIN_SOURCE=${ALL_SOURCES} (spans all sources)` };
}
if (!SOURCE_ID_RE.test(env)) {
throw new Error(`Invalid GBRAIN_SOURCE value "${env}". Must match [a-z0-9-]{1,32}.`);
}
+8 -13
View File
@@ -11,7 +11,7 @@
* pathToSlug() convert file paths to page slugs
*/
import { SLUG_WORD_CHARS } from './cjk.ts';
import { CJK_SLUG_CHARS } from './cjk.ts';
// v0.37.7.0 #1169 submodule-detection helpers. Bottom-of-file already
// aliases existsSync as `_existsSync` for other purposes; the top-of-file
// import keeps the pruneDir helper's deps near its callsite.
@@ -396,10 +396,8 @@ export function unsyncableReason(path: string, opts: SyncableOptions = {}): Sync
/**
* Character class for the lowercase-canonical form of a slug segment after
* slugifySegment() has run. Letters/numbers in any script (lowercase where
* the script has case #3417), dots, underscores, hyphens. Uses \p{...}
* classes, so composed regexes need the `u` flag (this one carries it).
* Exposed so adjacent code (e.g. takes-fence holder validation,
* slugifySegment() has run. Lowercase letters, digits, dots, underscores,
* hyphens. Exposed so adjacent code (e.g. takes-fence holder validation,
* v0.32 EXP-4) can reuse the actual repo slug grammar instead of inventing
* a stricter parallel one and emitting false-positive warnings on legitimate
* `companies/acme.io` / `people/foo_bar` slugs (codex review #3).
@@ -407,18 +405,15 @@ export function unsyncableReason(path: string, opts: SyncableOptions = {}): Sync
* Pattern is the inner character class only (no anchors); callers wrap it
* in `^...$` or compose it with prefixes like `(?:people|companies)/...`.
*/
export const SLUG_SEGMENT_PATTERN = new RegExp(`[${SLUG_WORD_CHARS}._\\-]+`, 'u');
export const SLUG_SEGMENT_PATTERN = new RegExp(`[a-z0-9._\\-${CJK_SLUG_CHARS}]+`);
/**
* Slugify a single path segment: lowercase, strip special chars, spaces hyphens.
* Letters and numbers from EVERY script are preserved (#3417): previously only
* Latin + CJK survived, so Hebrew/Arabic/Cyrillic/Greek/Thai/... filenames
* collapsed to empty segments and distinct files silently merged onto one slug.
* NFC re-normalize after the NFD-strip-accents pass so Hangul Jamo recomposes
* back into precomposed syllables, and so NFD filenames (macOS) and NFC
* filenames (Linux/git) of the same name produce the SAME slug.
* CJK ranges (Han / Hiragana / Katakana / Hangul Syllables) are preserved (v0.32.7).
* NFC re-normalize after the NFD-strip-accents pass so Hangul Jamo recomposes back
* into precomposed syllables that fall inside the whitelist.
*/
const SLUGIFY_KEEP_RE = new RegExp(`[^${SLUG_WORD_CHARS}.\\s_\\-]`, 'gu');
const SLUGIFY_KEEP_RE = new RegExp(`[^a-z0-9.\\s_\\-${CJK_SLUG_CHARS}]`, 'g');
export function slugifySegment(segment: string): string {
return segment
-1
View File
@@ -134,7 +134,6 @@ export const TAKES_FENCE_END = '<!--- gbrain:takes:end -->';
import { SLUG_SEGMENT_PATTERN } from './sync.ts';
export const HOLDER_REGEX = new RegExp(
`^(?:world|brain|(?:people|companies)/${SLUG_SEGMENT_PATTERN.source}|${SLUG_SEGMENT_PATTERN.source})$`,
'u', // required by SLUG_SEGMENT_PATTERN's \p{...} classes (#3417)
);
/**
-7
View File
@@ -110,12 +110,6 @@ export function rowToPage(row: Record<string, unknown>): Page {
const sourceUri = row.source_uri === undefined ? undefined : (row.source_uri as string | null);
const ingestedVia = row.ingested_via === undefined ? undefined : (row.ingested_via as string | null);
const ingestedAt = readOptionalDate(row.ingested_at);
// #3507: the CR tier the page was last embedded under (three-state, same
// pattern as the provenance columns above). Re-embed paths (`embed --stale`
// and friends) read this to reproduce the page's stored wrapping convention.
const contextualRetrievalMode = row.contextual_retrieval_mode === undefined
? undefined
: (row.contextual_retrieval_mode as Page['contextual_retrieval_mode']);
return {
id: row.id as number,
slug: row.slug as string,
@@ -141,7 +135,6 @@ export function rowToPage(row: Record<string, unknown>): Page {
...(sourceUri !== undefined && { source_uri: sourceUri }),
...(ingestedVia !== undefined && { ingested_via: ingestedVia }),
...(ingestedAt !== undefined && { ingested_at: ingestedAt }),
...(contextualRetrievalMode !== undefined && { contextual_retrieval_mode: contextualRetrievalMode }),
// v0.31.12: propagate source_id so downstream callers (embed, reconcile-links)
// can thread it through getChunks / upsertChunks without defaulting to 'default'.
// v0.32.8: Page.source_id is required. Every SELECT feeding rowToPage now
-68
View File
@@ -1,68 +0,0 @@
/**
* #3554 resetGateway() must restore the test baseline, not unconfigure.
*
* The bunfig preload (test/helpers/legacy-embedding-preload.ts) pins the
* gateway to openai:text-embedding-3-large @ 1536 at process start and
* registers that config as the reset baseline. Before the fix,
* resetGateway() wiped the pin to _config = null; the next file's beforeAll
* engine-connect then reconfigured from the SHIPPED default (zembed-1 @
* 1280) and every 1536-d fixture in that file failed with
* `expected 1280 dimensions, not 1536`. Which file pairs collided depended
* on shard bin-packing, so adding ANY test file reshuffled the mines.
*
* These assertions pin the contract so it cannot silently rot again.
*/
import { describe, test, expect, afterEach } from 'bun:test';
import {
configureGateway,
resetGateway,
__unconfigureGatewayForTests,
__setChatTransportForTests,
getEmbeddingModel,
getEmbeddingDimensions,
isAvailable,
} from '../../src/core/ai/gateway.ts';
afterEach(() => resetGateway());
describe('resetGateway baseline restore (#3554)', () => {
test('immediately after resetGateway(), the preload baseline is live', () => {
resetGateway();
expect(getEmbeddingModel()).toBe('openai:text-embedding-3-large');
expect(getEmbeddingDimensions()).toBe(1536);
});
test('resetGateway() overwrites a file-local config back to the baseline', () => {
configureGateway({
embedding_model: 'zeroentropyai:zembed-1',
embedding_dimensions: 1280,
env: {},
});
expect(getEmbeddingDimensions()).toBe(1280);
resetGateway();
expect(getEmbeddingModel()).toBe('openai:text-embedding-3-large');
expect(getEmbeddingDimensions()).toBe(1536);
});
test('resetGateway() still clears test transports (no stale transport leaks back)', () => {
__setChatTransportForTests(async () => {
throw new Error('should have been cleared');
});
resetGateway();
// Baseline config sets no chat key in a keyless env, but the transport
// seam itself must be gone: isAvailable('chat') short-circuits to true
// whenever a chat transport is installed, so with a hard-unconfigured
// gateway it can only be true if the transport survived the reset.
__unconfigureGatewayForTests();
expect(isAvailable('chat')).toBe(false);
});
test('__unconfigureGatewayForTests() gives a genuinely unconfigured gateway', () => {
__unconfigureGatewayForTests();
expect(() => getEmbeddingDimensions()).toThrow(/not configured/);
expect(isAvailable('embedding')).toBe(false);
// And a plain reset brings the baseline back.
resetGateway();
expect(getEmbeddingDimensions()).toBe(1536);
});
});
-4
View File
@@ -2,7 +2,6 @@ import { describe, test, expect, beforeEach, afterAll } from 'bun:test';
import {
configureGateway,
resetGateway,
__unconfigureGatewayForTests,
isAvailable,
embed,
getEmbeddingModel,
@@ -56,9 +55,6 @@ describe('gateway.isAvailable (silent-drop regression surface)', () => {
beforeEach(() => resetGateway());
test('returns false when gateway not configured', () => {
// resetGateway() restores the preload's test baseline (#3554); go
// genuinely unconfigured for this one assertion.
__unconfigureGatewayForTests();
expect(isAvailable('embedding')).toBe(false);
});
-176
View File
@@ -1,176 +0,0 @@
/**
* #1712 (dupes #2289, #2140) the `__all__` sentinel must work in EVERY
* resolution tier, not just as a per-call `source_id` param.
*
* The bug: SOURCE_ID_RE forbids underscores, so `--source __all__` and
* `GBRAIN_SOURCE=__all__` threw in the resolver; the CLI's makeContext
* blanket-caught that and silently fell back to `sourceId: 'default'`
* making the documented span-everything flag STRICTLY NARROWER than passing
* no flag at all (the catch also discarded the #2561/#3242 federated
* widening). Meanwhile sourceScopeOpts treated a ctx.sourceId of '__all__'
* as an unsatisfiable literal.
*
* Uses the literal '__all__' (not the ALL_SOURCES constant) so these tests
* load and run behaviorally against pre-fix trees.
*/
import { describe, test, expect } from 'bun:test';
import { withEnv } from './helpers/with-env.ts';
import {
resolveSourceId,
resolveSourceIdEngineFree,
resolveSourceWithTier,
} from '../src/core/source-resolver.ts';
import {
sourceScopeOpts,
federatedSearchScope,
type OperationContext,
} from '../src/core/operations.ts';
import type { BrainEngine } from '../src/core/engine.ts';
// Stub engine: registered sources + no local_path rows + no default config.
function makeStub(registeredSources: string[]): BrainEngine {
return {
kind: 'pglite',
executeRaw: async <T>(sql: string, params?: unknown[]): Promise<T[]> => {
if (sql.includes('SELECT id FROM sources WHERE id = $1')) {
const target = params?.[0];
return registeredSources.includes(target as string)
? [{ id: target } as unknown as T]
: [];
}
return [];
},
getConfig: async () => null,
} as unknown as BrainEngine;
}
function ctxOf(overrides: Partial<OperationContext> = {}): OperationContext {
return {
engine: {} as any,
config: {} as any,
logger: console as any,
dryRun: false,
remote: true,
sourceId: 'default',
...overrides,
};
}
// ── Resolver tiers pass the sentinel through verbatim ──────────────────
describe('source-resolver — __all__ sentinel pass-through', () => {
test('resolveSourceId: explicit --source __all__ resolves (no regex throw, no existence check)', async () => {
// '__all__' is deliberately NOT in the registered set — the sentinel
// must skip assertSourceExists (it is not a source id).
const id = await resolveSourceId(makeStub(['default']), '__all__', '/nonexistent');
expect(id).toBe('__all__');
});
test('resolveSourceId: GBRAIN_SOURCE=__all__ resolves (#2140)', async () => {
await withEnv({ GBRAIN_SOURCE: '__all__' }, async () => {
const id = await resolveSourceId(makeStub(['default']), null, '/nonexistent');
expect(id).toBe('__all__');
});
});
test('resolveSourceIdEngineFree: explicit + env __all__ (thin-client path)', async () => {
expect(resolveSourceIdEngineFree('__all__', '/nonexistent')).toBe('__all__');
await withEnv({ GBRAIN_SOURCE: '__all__' }, () => {
expect(resolveSourceIdEngineFree(null, '/nonexistent')).toBe('__all__');
});
});
test('resolveSourceWithTier: flag and env tiers carry the sentinel', async () => {
const flag = await resolveSourceWithTier(makeStub(['default']), '__all__', '/nonexistent');
expect(flag).toMatchObject({ source_id: '__all__', tier: 'flag' });
await withEnv({ GBRAIN_SOURCE: '__all__' }, async () => {
const env = await resolveSourceWithTier(makeStub(['default']), null, '/nonexistent');
expect(env).toMatchObject({ source_id: '__all__', tier: 'env' });
});
});
test('a genuinely invalid --source still throws (SOURCE_ID_RE not loosened)', async () => {
await expect(resolveSourceId(makeStub(['default']), 'my_source', '/nonexistent'))
.rejects.toThrow(/Invalid --source/);
expect(() => resolveSourceIdEngineFree('my_source', '/nonexistent'))
.toThrow(/Invalid --source/);
});
});
// ── sourceScopeOpts — the single read-scope choke point ─────────────────
describe('sourceScopeOpts — __all__ sentinel', () => {
test('trusted local (remote === false): spans the whole brain (empty scope)', () => {
expect(sourceScopeOpts(ctxOf({ remote: false, sourceId: '__all__' }))).toEqual({});
});
test('remote: keeps the unsatisfiable literal — fail-closed, never widens', () => {
expect(sourceScopeOpts(ctxOf({ remote: true, sourceId: '__all__' })))
.toEqual({ sourceId: '__all__' });
});
test('anything not strictly remote === false is untrusted (fail-closed)', () => {
// undefined / missing remote must behave like remote, per the trust rule.
const ctx = ctxOf({ sourceId: '__all__' });
(ctx as any).remote = undefined;
expect(sourceScopeOpts(ctx)).toEqual({ sourceId: '__all__' });
});
test('a federated grant always wins over the sentinel', () => {
const ctx = ctxOf({
remote: true,
sourceId: '__all__',
auth: { token: 't', clientId: 'c', scopes: [], allowedSources: ['a', 'b'] } as any,
});
expect(sourceScopeOpts(ctx)).toEqual({ sourceIds: ['a', 'b'] });
});
});
// ── Never narrower than passing no flag (#2561 regression shape) ────────
describe('__all__ is never narrower than an unqualified read', () => {
test('local __all__ spans the brain even when federated widening exists', () => {
// Unqualified read on a federated brain widens to the federated array…
const unqualified = ctxOf({
remote: false,
sourceId: 'default',
localFederatedSourceIds: ['default', 'src-a', 'src-b'],
});
expect(federatedSearchScope(unqualified)).toEqual({
sourceIds: ['default', 'src-a', 'src-b'],
});
// …and __all__ must be a superset of that: the whole brain ({}).
const all = ctxOf({ remote: false, sourceId: '__all__' });
expect(federatedSearchScope(all)).toEqual({});
});
});
// ── makeContext — explicit --source failures error loudly ───────────────
describe('cli makeContext — no silent default fallback for explicit --source', () => {
test('--source __all__ produces ctx.sourceId __all__ (was: silent default)', async () => {
const { makeContext } = await import('../src/cli.ts');
const ctx = await makeContext(makeStub(['default']), { source: '__all__' });
expect(ctx.sourceId).toBe('__all__');
expect(ctx.remote).toBe(false);
});
test('an explicit --source that fails to resolve throws instead of becoming default', async () => {
const { makeContext } = await import('../src/cli.ts');
await expect(makeContext(makeStub(['default']), { source: 'ghost' }))
.rejects.toThrow(/not found/);
await expect(makeContext(makeStub(['default']), { source: 'my_source' }))
.rejects.toThrow(/Invalid --source/);
});
test('ambient resolution failure still falls back silently (pre-init brains)', async () => {
const { makeContext } = await import('../src/cli.ts');
const broken = {
kind: 'pglite',
executeRaw: async () => { throw new Error('relation "sources" does not exist'); },
getConfig: async () => { throw new Error('relation "config" does not exist'); },
} as unknown as BrainEngine;
const ctx = await makeContext(broken, {});
expect(ctx.sourceId).toBe('default');
});
});
+10 -48
View File
@@ -11,34 +11,15 @@ import { tmpdir } from 'os';
import { execFileSync } from 'child_process';
import { hardenBrainRepo } from '../src/core/brain-repo-durability.ts';
// #2943 root cause: `env: process.env` is REQUIRED here. Bun snapshots
// process.env at startup, so without it the spawned git — and any post-commit
// hook it fires — is blind to beforeEach's HOME/GBRAIN_HOME mutations (the
// same Bun quirk as #2747, see resolveGbrainCliPath in brain-repo-durability).
// Pre-fix, the hook under test resolved ${GBRAIN_HOME:-$HOME/.gbrain} to the
// OPERATOR'S REAL ~/.gbrain: it wrote its log lines there (polluting the real
// brain-push.log on every run), the LOCAL-ONLY test never saw them in the
// temp log it polls, and the assertion only passed when the scaffolding push
// from beforeEach (spawned by hardenBrainRepo WITH explicit env) happened to
// still be in flight, lose the ref race, and retry AFTER the test had pointed
// origin at the dead path — an accidental, load-dependent signal. That race
// is the CI flake.
function git(cwd: string, ...args: string[]): string {
return execFileSync('git', ['-C', cwd, '-c', 'protocol.file.allow=always', ...args], {
stdio: ['ignore', 'pipe', 'pipe'], encoding: 'utf-8', env: process.env,
stdio: ['ignore', 'pipe', 'pipe'], encoding: 'utf-8',
}).trim();
}
function originHead(bare: string): string {
return git(bare, 'rev-parse', 'refs/heads/main');
}
// #2943: 30s poll deadlines (was 8s) for headroom under loaded CI shards —
// the unreachable-origin path runs ~6 sequential process spawns after the
// hook detaches. Every hook test also passes an explicit 60_000 third-arg
// timeout: bun 1.3.14 IGNORES bunfig.toml's `timeout` key, so a bare
// `bun test` enforces its 5000ms default and killed these tests before the
// internal deadline could even elapse (the runner scripts pass --timeout
// explicitly, which is why the inversion only bit direct local runs).
async function waitForOrigin(bare: string, expectSha: string, ms = 30_000): Promise<boolean> {
async function waitForOrigin(bare: string, expectSha: string, ms = 8000): Promise<boolean> {
const deadline = Date.now() + ms;
while (Date.now() < deadline) {
try { if (originHead(bare) === expectSha) return true; } catch { /* */ }
@@ -47,24 +28,6 @@ async function waitForOrigin(bare: string, expectSha: string, ms = 30_000): Prom
return false;
}
/** #2943 (index.lock form): hardenBrainRepo installs the post-commit hook
* BEFORE committing the scaffolding, so that commit fires the hook and
* detaches a background brain_push. If that push loses the ref race against
* hardenBrainRepo's own synchronous push, it falls back to `git pull
* --rebase`, which takes .git/index.lock — racing the test body's first git
* calls ("Unable to create '.../.git/index.lock': File exists"). Wait for the
* detached push's terminal log line before handing the repo to the test. */
async function waitForHookPushSettled(ms = 30_000): Promise<void> {
const log = join(process.env.GBRAIN_HOME!, 'brain-push.log');
const terminal = /\[push\] (ok|lock-timeout|LOCAL-ONLY)/;
const deadline = Date.now() + ms;
while (Date.now() < deadline) {
if (existsSync(log) && terminal.test(readFileSync(log, 'utf-8'))) return;
await new Promise(r => setTimeout(r, 150));
}
throw new Error(`detached hook push did not settle within ${ms}ms (${log})`);
}
let root: string, work: string, bare: string;
let oldHome: string | undefined, oldGbrainHome: string | undefined;
@@ -75,15 +38,14 @@ beforeEach(async () => {
process.env.GBRAIN_HOME = join(process.env.HOME, '.gbrain');
process.env.GBRAIN_GIT_ALLOW_FILE_TRANSPORT = '1';
bare = mkdtempSync(join(root, 'origin-')) + '.git';
execFileSync('git', ['init', '-q', '--bare', '-b', 'main', bare], { stdio: 'ignore', env: process.env });
execFileSync('git', ['init', '-q', '--bare', '-b', 'main', bare], { stdio: 'ignore' });
work = mkdtempSync(join(root, 'work-'));
execFileSync('git', ['-c', 'protocol.file.allow=always', 'clone', '-q', bare, work], { stdio: 'ignore', env: process.env });
execFileSync('git', ['-c', 'protocol.file.allow=always', 'clone', '-q', bare, work], { stdio: 'ignore' });
git(work, 'config', 'user.email', 't@t.t'); git(work, 'config', 'user.name', 'tester');
writeFileSync(join(work, 'README.md'), 'init\n');
git(work, 'add', 'README.md'); git(work, 'commit', '-qm', 'init'); git(work, 'push', '-q', 'origin', 'main');
git(work, 'remote', 'set-head', 'origin', 'main');
await hardenBrainRepo({ repoPath: work, sourceId: 'wiki', pat: 'ghp_x', installCron: false });
await waitForHookPushSettled();
});
afterEach(() => {
if (oldHome === undefined) delete process.env.HOME; else process.env.HOME = oldHome;
@@ -103,7 +65,7 @@ describe('brain-commit-push.sh (D13 guarantee)', () => {
expect(originHead(bare)).toBe(git(work, 'rev-parse', 'HEAD'));
// origin actually has the file
const verify = mkdtempSync(join(root, 'verify-'));
execFileSync('git', ['-c', 'protocol.file.allow=always', 'clone', '-q', bare, verify], { stdio: 'ignore', env: process.env });
execFileSync('git', ['-c', 'protocol.file.allow=always', 'clone', '-q', bare, verify], { stdio: 'ignore' });
expect(existsSync(join(verify, 'people', 'alice.md'))).toBe(true);
});
@@ -140,7 +102,7 @@ describe('brain-commit-push.sh (D13 guarantee)', () => {
rmSync(join(work, '.git', 'hooks', 'post-commit'));
// Advance the remote from a second clone so a pull is genuinely needed.
const other = mkdtempSync(join(root, 'other-'));
execFileSync('git', ['-c', 'protocol.file.allow=always', 'clone', '-q', bare, other], { stdio: 'ignore', env: process.env });
execFileSync('git', ['-c', 'protocol.file.allow=always', 'clone', '-q', bare, other], { stdio: 'ignore' });
git(other, 'config', 'user.email', 'o@o.o'); git(other, 'config', 'user.name', 'other');
writeFileSync(join(other, 'remote.md'), 'from other\n');
git(other, 'add', 'remote.md'); git(other, 'commit', '-qm', 'remote change'); git(other, 'push', '-q', 'origin', 'main');
@@ -166,26 +128,26 @@ describe('post-commit hook (D9 local, D7 self-contained)', () => {
git(work, 'add', 'note.md'); git(work, 'commit', '-qm', 'note'); // fires .git/hooks/post-commit
const head = git(work, 'rev-parse', 'HEAD');
expect(await waitForOrigin(bare, head)).toBe(true);
}, 60_000);
});
test('the hook works even with the committed helper deleted (self-contained)', async () => {
rmSync(join(work, 'scripts', 'brain-commit-push.sh'));
git(work, 'add', '-A'); git(work, 'commit', '-qm', 'remove helper');
const head = git(work, 'rev-parse', 'HEAD');
expect(await waitForOrigin(bare, head)).toBe(true);
}, 60_000);
});
test('logs a clear LOCAL-ONLY line when origin is unreachable', async () => {
git(work, 'remote', 'set-url', 'origin', join(root, 'gone2.git'));
writeFileSync(join(work, 'orphan.md'), 'o\n');
git(work, 'add', 'orphan.md'); git(work, 'commit', '-qm', 'orphan');
const log = join(process.env.GBRAIN_HOME!, 'brain-push.log');
const deadline = Date.now() + 30_000;
const deadline = Date.now() + 8000;
let found = false;
while (Date.now() < deadline) {
if (existsSync(log) && readFileSync(log, 'utf-8').includes('NEEDS ATTENTION')) { found = true; break; }
await new Promise(r => setTimeout(r, 150));
}
expect(found).toBe(true);
}, 60_000);
});
});
-173
View File
@@ -1,173 +0,0 @@
/**
* #3513: parseOpArgs' stdin read must never block forever.
*
* In a non-TTY with no piped input a CI step, a cron job, an agent
* harness that inherits a non-TTY stdin without writing to it the old
* inline `readFileSync(0)` never returned. The fix bounds the read with a
* first-byte deadline (pipes/sockets only) and falls through to the
* existing required-param usage error on timeout.
*
* The load-bearing regression test spawns the REAL CLI with a held-open,
* never-written pipe: on pre-fix code it hangs until our observation window
* kills it; on fixed code it exits 1 with the usage error well inside the
* window. The stdin read + required-param check both run BEFORE engine
* connect, so no brain/DB is touched.
*/
import { describe, expect, test } from 'bun:test';
import { dirname, join } from 'path';
const REPO = dirname(import.meta.dir);
const CLI = join(REPO, 'src', 'cli.ts');
interface CliRun {
exited: boolean;
exitCode: number | null;
stderr: string;
}
/** Narrow Bun's `number | FileSink` stdin union to the pipe sink. */
function pipeSink(proc: { stdin: unknown }): { write(d: string): unknown; end(): unknown } {
const s = proc.stdin;
if (!s || typeof s === 'number') throw new Error('expected a piped stdin sink');
return s as { write(d: string): unknown; end(): unknown };
}
/**
* Spawn the CLI with the given stdin wiring. `holdPipeOpen` keeps the write
* end of the stdin pipe alive without ever writing the #3513 repro. The
* observation window kills the child if it hasn't exited (pre-fix hang).
*/
async function runCliWithStdin(
args: string[],
stdin: 'hold-open' | 'closed-empty' | { data: string } | { file: string },
windowMs: number,
): Promise<CliRun> {
const proc = Bun.spawn(['bun', 'run', CLI, ...args], {
cwd: REPO,
env: { ...process.env, GBRAIN_STDIN_TIMEOUT_MS: '500' },
stdin: typeof stdin === 'object' && 'file' in stdin ? Bun.file(stdin.file) : 'pipe',
stdout: 'pipe',
stderr: 'pipe',
});
if (typeof stdin === 'object' && 'data' in stdin) {
pipeSink(proc).write(stdin.data);
await pipeSink(proc).end();
} else if (stdin === 'closed-empty') {
await pipeSink(proc).end();
}
// 'hold-open': never write, never close — the CI/cron/agent-harness shape.
let exited = true;
const killer = setTimeout(() => {
exited = false;
try { proc.kill('SIGKILL'); } catch { /* already dead */ }
}, windowMs);
const [exitCode, stderr] = await Promise.all([
proc.exited,
new Response(proc.stderr).text(),
]);
clearTimeout(killer);
try { pipeSink(proc).end(); } catch { /* hold-open cleanup */ }
return { exited, exitCode: exited ? exitCode : null, stderr };
}
describe('#3513 — stdin-capable op with a non-TTY, never-written stdin', () => {
test('exits fast with the usage error instead of blocking forever', async () => {
// `put` declares stdin:'content' (required). No inline content, no piped
// input → the bounded read times out at 500ms, content stays unset, and
// the required-param check prints usage and exits 1. Pre-fix: readFileSync(0)
// blocks until the 20s window kills the child.
const run = await runCliWithStdin(['put', 'stdin-hang-test-slug'], 'hold-open', 20_000);
expect(run.exited).toBe(true); // pre-#3513 this is false: the read never returns
expect(run.exitCode).toBe(1);
expect(run.stderr).toContain('Usage: gbrain put');
}, 30_000);
test('a genuine pipe with data is still consumed (no hang, no crash)', async () => {
// Piped content fills `content`; the missing positional slug then fails
// the required check — proving the stream path read stdin and moved on.
const run = await runCliWithStdin(['put'], { data: '# hello\n' }, 20_000);
expect(run.exited).toBe(true);
expect(run.exitCode).toBe(1);
expect(run.stderr).toContain('Usage: gbrain put');
}, 30_000);
test('empty-but-real input (`< /dev/null`) does not hang', async () => {
const run = await runCliWithStdin(['put'], { file: '/dev/null' }, 20_000);
expect(run.exited).toBe(true);
expect(run.exitCode).toBe(1);
}, 30_000);
test('an empty pipe that closes immediately does not hang', async () => {
const run = await runCliWithStdin(['put'], 'closed-empty', 20_000);
expect(run.exited).toBe(true);
expect(run.exitCode).toBe(1);
}, 30_000);
});
describe('#3513 — applyStdinParam content preservation (subprocess driver)', () => {
// Drive the exported helper in a child process so we control the child's
// real fd 0 — bun test's own stdin is not a reliable fixture.
const DRIVER = `
const { applyStdinParam } = await import(${JSON.stringify(CLI)});
const op = { name: 'put', params: { content: { type: 'string', required: true } }, cliHints: { stdin: 'content' } };
const params = {};
await applyStdinParam(op, params);
console.log(JSON.stringify(params));
process.exit(0);
`;
async function runDriver(
stdin: 'hold-open' | 'closed-empty' | { data: string } | { file: string },
): Promise<{ exited: boolean; params: Record<string, unknown> | null }> {
const proc = Bun.spawn(['bun', '-e', DRIVER], {
cwd: REPO,
env: { ...process.env, GBRAIN_STDIN_TIMEOUT_MS: '500' },
stdin: typeof stdin === 'object' && 'file' in stdin ? Bun.file(stdin.file) : 'pipe',
stdout: 'pipe',
stderr: 'pipe',
});
if (typeof stdin === 'object' && 'data' in stdin) {
pipeSink(proc).write(stdin.data);
await pipeSink(proc).end();
} else if (stdin === 'closed-empty') {
await pipeSink(proc).end();
}
let exited = true;
const killer = setTimeout(() => {
exited = false;
try { proc.kill('SIGKILL'); } catch { /* already dead */ }
}, 20_000);
const [stdout] = await Promise.all([new Response(proc.stdout).text(), proc.exited]);
clearTimeout(killer);
try { pipeSink(proc).end(); } catch { /* hold-open cleanup */ }
const line = stdout.trim().split('\n').pop() ?? '';
let params: Record<string, unknown> | null = null;
try { params = JSON.parse(line); } catch { /* child killed before printing */ }
return { exited, params };
}
test('piped data lands in the stdin param verbatim', async () => {
const { exited, params } = await runDriver({ data: '---\ntitle: x\n---\nbody' });
expect(exited).toBe(true);
expect(params?.content).toBe('---\ntitle: x\n---\nbody');
}, 30_000);
test('/dev/null yields empty-string content (readable, empty — pre-fix parity)', async () => {
const { exited, params } = await runDriver({ file: '/dev/null' });
expect(exited).toBe(true);
expect(params?.content).toBe('');
}, 30_000);
test('empty closed pipe yields empty-string content', async () => {
const { exited, params } = await runDriver('closed-empty');
expect(exited).toBe(true);
expect(params?.content).toBe('');
}, 30_000);
test('held-open pipe times out and leaves the param unset', async () => {
const { exited, params } = await runDriver('hold-open');
expect(exited).toBe(true); // completes inside the window instead of hanging
expect(params).toEqual({});
}, 30_000);
});
-138
View File
@@ -1,138 +0,0 @@
/**
* #3502: docs must not reference nonexistent gbrain commands.
*
* `docs/tutorials/personal-brain.md` shipped a `gbrain install` step for two
* months after the command it replaced was retired every reader hit
* "Unknown command: install". This guard scans README.md, docs/, and skills/
* for `gbrain <verb>` invocations in code (fenced blocks + inline code spans)
* and checks each verb against the live CLI surface: CLI_ONLY, operation
* cliHints names (non-hidden), and aliases.
*
* Deliberately excluded (historical or speculative by design, per CLAUDE.md's
* "historical docs are never rewritten" rule):
* - docs/GBRAIN_V0.md the v0 spec; documents v0's CLI
* - docs/designs/, docs/plans/ future/speculative design docs
* - docs/migrations/, skills/migrations/ per-release migration notes,
* written against that release's CLI
* - docs/UPGRADING_DOWNSTREAM_AGENTS.md per-release upgrade chronicle
*
* Heuristics keep prose out: only fenced code + inline spans are scanned,
* comment lines and diagram lines are skipped, and the verb must sit in
* command position (start of command text, or after a shell operator).
*/
import { describe, expect, test } from 'bun:test';
import { readdirSync, readFileSync, statSync } from 'fs';
import { dirname, join, relative } from 'path';
import { CLI_ONLY, cliAliases } from '../src/cli.ts';
import { operations } from '../src/core/operations.ts';
const ROOT = dirname(import.meta.dir);
const EXCLUDED = [
'docs/GBRAIN_V0.md',
'docs/UPGRADING_DOWNSTREAM_AGENTS.md',
'docs/designs/',
'docs/plans/',
'docs/migrations/',
'skills/migrations/',
];
/** Known-intentional references to commands that deliberately don't exist. */
const ALLOWLIST: Record<string, string[]> = {
// The doc explains that gbrain does NOT ship this command, on purpose.
'docs/guides/rls-and-you.md': ['rls-exempt'],
};
function validCommands(): Set<string> {
const valid = new Set<string>(CLI_ONLY);
for (const op of operations) {
const name = op.cliHints?.name;
if (name && !op.cliHints?.hidden) valid.add(name);
}
for (const alias of cliAliases.keys()) valid.add(alias);
return valid;
}
function* mdFiles(dir: string): Generator<string> {
for (const entry of readdirSync(dir)) {
const p = join(dir, entry);
if (statSync(p).isDirectory()) yield* mdFiles(p);
else if (p.endsWith('.md')) yield p;
}
}
interface CodeLine { code: string; line: number }
/** Fenced-block lines + inline code spans that START with `gbrain `. */
function codeRegions(text: string): CodeLine[] {
const out: CodeLine[] = [];
const lines = text.split('\n');
let inFence = false;
for (let i = 0; i < lines.length; i++) {
const l = lines[i];
if (/^\s*(```|~~~)/.test(l)) { inFence = !inFence; continue; }
if (inFence) {
const t = l.trim();
if (/^(#|\/\/|--|\*)/.test(t)) continue; // comment lines
if (/[│┌┐└┘├┤─═╔╗╚╝]/.test(l)) continue; // ASCII-art diagrams
out.push({ code: l, line: i + 1 });
continue;
}
for (const m of l.matchAll(/`(gbrain [^`]+)`/g)) out.push({ code: m[1], line: i + 1 });
}
return out;
}
/** True when `gbrain` sits at command position (not mid-prose). */
function commandPosition(prefix: string): boolean {
const p = prefix.trimEnd();
return p === '' || /[|;&`(={[]$/.test(p) || /\$$/.test(p);
}
function scan(): string[] {
const valid = validCommands();
const violations: string[] = [];
const files = [
join(ROOT, 'README.md'),
...mdFiles(join(ROOT, 'docs')),
...mdFiles(join(ROOT, 'skills')),
];
for (const file of files) {
const rel = relative(ROOT, file);
if (EXCLUDED.some((e) => rel === e || rel.startsWith(e))) continue;
const text = readFileSync(file, 'utf-8');
for (const { code, line } of codeRegions(text)) {
for (const m of code.matchAll(/\bgbrain\s+([A-Za-z][\w-]*)/g)) {
const verb = m[1];
if (!/^[a-z][a-z0-9_-]{2,}$/.test(verb)) continue; // flags, <slots>, v0.x
if (!commandPosition(code.slice(0, m.index))) continue;
if (valid.has(verb)) continue;
if (ALLOWLIST[rel]?.includes(verb)) continue;
violations.push(`${rel}:${line}: \`gbrain ${verb}\` is not a real command — ${code.trim().slice(0, 90)}`);
}
}
}
return violations;
}
describe('#3502 — docs reference only real gbrain commands', () => {
test('every `gbrain <verb>` in README/docs/skills resolves to a live command', () => {
const violations = scan();
expect(violations).toEqual([]);
});
test('the sanity anchors: install is dead, init/put/skillpack are live', () => {
const valid = validCommands();
expect(valid.has('install')).toBe(false); // retired v0.36.0.0 — the #3502 bug
expect(valid.has('init')).toBe(true);
expect(valid.has('put')).toBe(true);
expect(valid.has('skillpack')).toBe(true);
});
test('pages + bench are dispatchable (documented surfaces; #2035 bug class)', () => {
// `pages` had a live handleCliOnly case but was dropped from CLI_ONLY;
// `bench` (bench-publish.ts) was documented but never wired at all.
expect(CLI_ONLY.has('pages')).toBe(true);
expect(CLI_ONLY.has('bench')).toBe(true);
});
});
+3 -4
View File
@@ -143,10 +143,9 @@ describe('checkEmbeddingWidthConsistency', () => {
});
test('gateway unconfigured: skips with ok', async () => {
// Hard-unconfigure so requireConfig() throws — resetGateway() would
// restore the preload's test baseline (#3554).
const { __unconfigureGatewayForTests } = await import('../src/core/ai/gateway.ts');
__unconfigureGatewayForTests();
// Reset gateway so requireConfig() throws.
const { resetGateway } = await import('../src/core/ai/gateway.ts');
resetGateway();
const check = await checkEmbeddingWidthConsistency(engine);
expect(check.status).toBe('ok');
expect(check.message).toContain('gateway not configured');
-81
View File
@@ -252,87 +252,6 @@ describe('upsertChunks — model provenance uses gateway-resolved model, not com
resetGateway();
});
// #3461: getEmbeddingModel() THROWS when the gateway is unconfigured — it
// never returns falsy — so the reland's `|| resolvedModel` guard was dead
// code and the catch path still stamped the compile-time default onto rows
// whose vectors came from the config-resolved provider. The engine must
// fall back to the brain's own `config.embedding_model` row instead.
test('#3461: unconfigured gateway falls back to the brain config model, never the compiled default', async () => {
await engine.setConfig('embedding_model', 'voyage:voyage-3-large');
// The preload's beforeEach re-configures the gateway before every test,
// so the reset must happen INSIDE the test body.
resetGateway();
await engine.putPage('docs/provenance-throw-path', {
type: 'concept',
title: 'Provenance throw-path page',
compiled_truth: 'Chunk written while the gateway is unconfigured.',
});
await engine.upsertChunks('docs/provenance-throw-path', [
{ chunk_index: 0, chunk_text: 'throw-path provenance chunk', chunk_source: 'compiled_truth' },
]);
const rows = await engine.executeRaw<{ model: string }>(
`SELECT cc.model FROM content_chunks cc
JOIN pages p ON p.id = cc.page_id
WHERE p.slug = 'docs/provenance-throw-path'`,
);
expect(rows.length).toBe(1);
expect(rows[0].model).toBe('voyage:voyage-3-large');
// Restore the value initSchema wrote for the rest of the file.
await engine.setConfig('embedding_model', 'openai:text-embedding-3-large');
});
// #3461 sibling: on a partial re-upsert that carries NO new embedding (the
// exact shape `embed --stale` produces for a page's non-stale chunks), the
// preserved vector must KEEP its original model label. The old
// COALESCE(EXCLUDED.model, …) relabeled it with the current gateway model.
test('#3461: preserved vector keeps its original model label on a no-embedding re-upsert', async () => {
configureGateway({
embedding_model: 'openai:text-embedding-3-large',
embedding_dimensions: 1536,
env: { OPENAI_API_KEY: 'sk-test' },
});
await engine.putPage('docs/provenance-preserve', {
type: 'concept',
title: 'Provenance preserve page',
compiled_truth: 'Chunk embedded under model A, re-upserted under model B.',
});
await engine.upsertChunks('docs/provenance-preserve', [
{
chunk_index: 0,
chunk_text: 'stable chunk text',
chunk_source: 'compiled_truth',
embedding: new Float32Array(VEC1536_A),
},
]);
// Model swap: the gateway now resolves a different model, and the
// re-upsert (same chunk_text) carries no new embedding.
configureGateway({
embedding_model: 'voyage:voyage-3-large',
embedding_dimensions: 1536,
env: { VOYAGE_API_KEY: 'test' },
});
await engine.upsertChunks('docs/provenance-preserve', [
{ chunk_index: 0, chunk_text: 'stable chunk text', chunk_source: 'compiled_truth' },
]);
const rows = await engine.executeRaw<{ model: string; has_embedding: boolean }>(
`SELECT cc.model, cc.embedding IS NOT NULL AS has_embedding
FROM content_chunks cc
JOIN pages p ON p.id = cc.page_id
WHERE p.slug = 'docs/provenance-preserve'`,
);
expect(rows.length).toBe(1);
expect(rows[0].has_embedding).toBe(true); // vector preserved…
expect(rows[0].model).toBe('openai:text-embedding-3-large'); // …and its label still describes it
resetGateway();
});
});
describe('buildVectorCastFragment — engine SQL composer (D3)', () => {
+4 -4
View File
@@ -69,7 +69,7 @@ describeE2E('E2E: JSONB roundtrip — v0.12.1 reliability wave', () => {
`;
expect(row.t).toBe('object');
expect(row.marker).toBe('rawdata-value');
}, 30_000);
});
test('logIngest writes pages_updated as array, not double-encoded string', async () => {
const engine = getEngine();
@@ -91,7 +91,7 @@ describeE2E('E2E: JSONB roundtrip — v0.12.1 reliability wave', () => {
expect(row.t).toBe('array');
expect(Number(row.n)).toBe(3);
expect(row.first).toBe('test/a');
}, 30_000);
});
// files.ts:254 (uploadRaw's cloud-upload branch) was changed from
// `${JSON.stringify({...})}::jsonb` to `${sql.json({...})}` in v0.12.1.
@@ -114,7 +114,7 @@ describeE2E('E2E: JSONB roundtrip — v0.12.1 reliability wave', () => {
expect(row.t).toBe('object');
expect(row.type).toBe('pdf');
expect(row.method).toBe('TUS resumable');
}, 30_000);
});
// Source-level tripwire: if anyone re-introduces the old `${JSON.stringify(x)}::jsonb`
// pattern for the fixed sites, fail loudly. Greps actual source files per the
@@ -129,5 +129,5 @@ describeE2E('E2E: JSONB roundtrip — v0.12.1 reliability wave', () => {
const source = await Bun.file(new URL(rel, import.meta.url)).text();
expect(source.match(bad)?.[0] ?? null).toBeNull();
}
}, 30_000);
});
});
+1 -7
View File
@@ -81,11 +81,6 @@ function copyFixturesIntoTempWorkspace(): Workspace {
let workspace: Workspace;
// Restore (not delete) after each test: the audit-dir preload sets
// GBRAIN_AUDIT_DIR once at process start, and deleting it leaks the
// operator's real ~/.gbrain/audit/ to every later file in the shard.
const priorAuditDir = process.env.GBRAIN_AUDIT_DIR;
beforeEach(() => {
workspace = copyFixturesIntoTempWorkspace();
// Redirect audit dir to the tempdir so the snapshot file doesn't pollute
@@ -94,8 +89,7 @@ beforeEach(() => {
});
afterEach(() => {
if (priorAuditDir === undefined) delete process.env.GBRAIN_AUDIT_DIR;
else process.env.GBRAIN_AUDIT_DIR = priorAuditDir;
delete process.env.GBRAIN_AUDIT_DIR;
workspace.cleanup();
});
+12 -12
View File
@@ -6,11 +6,7 @@
* process.env.
*/
import { describe, test, expect, beforeEach, afterAll } from 'bun:test';
import {
configureGateway,
resetGateway,
__unconfigureGatewayForTests,
} from '../src/core/ai/gateway.ts';
import { configureGateway, resetGateway } from '../src/core/ai/gateway.ts';
import {
validateEmbeddingCreds,
formatEmbeddingCredsError,
@@ -26,12 +22,18 @@ import type { AIGatewayConfig } from '../src/core/ai/types.ts';
// isAvailable('embedding') check. That's what made facts-backstop-gating
// fail intermittently (bin-pack-dependent) on CI shard 10.
//
// #3554: resetGateway() now restores the preload's legacy pin itself (the
// preload registers it via __setGatewayResetBaselineForTests), so a bare
// reset is safe here the NEXT file's beforeAll sees the 1536-d baseline,
// not a null gateway that would seed 1280-d schemas under 1536-d fixtures.
// Don't end on a bare resetGateway() either: the NEXT file's beforeAll
// (often engine.initSchema, which sizes vector columns from ambient gateway
// state) runs before the legacy-embedding-preload's per-test restore, so a
// null gateway here would seed 1280-d schemas under 1536-d fixtures.
// Restore the preload's legacy pin instead.
afterAll(() => {
resetGateway();
configureGateway({
embedding_model: 'openai:text-embedding-3-large',
embedding_dimensions: 1536,
env: { ...process.env },
});
});
function baseConfig(overrides: Partial<AIGatewayConfig> = {}): AIGatewayConfig {
@@ -136,9 +138,7 @@ describe('validateEmbeddingCreds', () => {
});
test('throws no_gateway_config when gateway was not configured', () => {
// resetGateway() restores the preload's test baseline (#3554), so this
// test needs the hard variant to get a genuinely unconfigured gateway.
__unconfigureGatewayForTests();
// resetGateway() in beforeEach already cleared _config.
let caught: unknown;
try { validateEmbeddingCreds(); } catch (e) { caught = e; }
expect(caught).toBeInstanceOf(EmbeddingCredentialError);
-76
View File
@@ -277,79 +277,3 @@ describe('embedStaleForSource', () => {
expect(txtRow.embedded_at).not.toBeNull();
});
});
// ────────────────────────────────────────────────────────────────
// #3507 — re-embed must reproduce the page's STORED contextual-retrieval
// wrapping convention. Before the fix, every plain re-embed (including the
// normal post-model-migration `embed --stale`) embedded raw chunk_text,
// silently replacing context-wrapped vectors with unwrapped ones.
// ────────────────────────────────────────────────────────────────
describe('contextual-retrieval wrapping on re-embed (#3507)', () => {
/** embedFn that records every text it is asked to embed. */
function capturingEmbedFn(seen: string[]) {
return (texts: string[]): Promise<Float32Array[]> => {
seen.push(...texts);
return fakeEmbedFn(texts);
};
}
async function seedWrappablePage(slug: string, title: string): Promise<void> {
await engine.putPage(slug, { type: 'note', title, compiled_truth: 'seeded' });
await engine.upsertChunks(slug, [
{ chunk_index: 0, chunk_text: 'prose chunk about widgets', chunk_source: 'compiled_truth', token_count: 4 },
{ chunk_index: 1, chunk_text: 'const x = 1;', chunk_source: 'fenced_code', token_count: 4 },
]);
}
test('title-mode page: stale re-embed sends title-wrapped texts; fenced_code stays raw', async () => {
await seedWrappablePage('wrapped-page', 'Widget Notes');
await engine.updatePageContextualRetrievalState('wrapped-page', 'default', 'title', 'gen-title');
const seen: string[] = [];
const result = await embedStaleForSource(engine, 'default', { embedFn: capturingEmbedFn(seen) });
expect(result.embedded).toBe(2);
expect(seen).toContain('<context>Widget Notes\n</context>\nprose chunk about widgets');
expect(seen).toContain('const x = 1;'); // fenced_code is NEVER wrapped (D20-T4)
// D20-T1: the canonical chunk_text is NOT rewritten — wrapping is embed-input-only.
const chunks = await engine.getChunks('wrapped-page');
expect(chunks.map((c) => c.chunk_text).sort()).toEqual(['const x = 1;', 'prose chunk about widgets']);
// Mode stamp unchanged for title-tier pages.
const rows = await engine.executeRaw<{ contextual_retrieval_mode: string }>(
`SELECT contextual_retrieval_mode FROM pages WHERE slug = 'wrapped-page'`,
);
expect(rows[0].contextual_retrieval_mode).toBe('title');
});
test('per_chunk_synopsis page: re-embed applies the title-tier wrapper and restamps honestly', async () => {
await seedWrappablePage('synopsis-page', 'Synopsis Notes');
await engine.updatePageContextualRetrievalState('synopsis-page', 'default', 'per_chunk_synopsis', 'gen-synopsis');
const seen: string[] = [];
const result = await embedStaleForSource(engine, 'default', { embedFn: capturingEmbedFn(seen) });
expect(result.embedded).toBe(2);
// Synopsis re-generation is a paid backfill concern; the plain re-embed
// lands at the title tier (the service's own D14 fallback tier)…
expect(seen).toContain('<context>Synopsis Notes\n</context>\nprose chunk about widgets');
// …and the stamped mode is updated so it keeps describing the vectors.
const rows = await engine.executeRaw<{ contextual_retrieval_mode: string }>(
`SELECT contextual_retrieval_mode FROM pages WHERE slug = 'synopsis-page'`,
);
expect(rows[0].contextual_retrieval_mode).toBe('title');
});
test('unstamped page (NULL mode) embeds raw chunk_text — convention preserved', async () => {
await seedWrappablePage('plain-page', 'Plain Notes');
// No updatePageContextualRetrievalState call: pre-CR page.
const seen: string[] = [];
const result = await embedStaleForSource(engine, 'default', { embedFn: capturingEmbedFn(seen) });
expect(result.embedded).toBe(2);
expect(seen).toContain('prose chunk about widgets');
expect(seen.some((t) => t.startsWith('<context>'))).toBe(false);
});
});
-73
View File
@@ -907,76 +907,3 @@ describe('runEmbed preserves code-chunk metadata across re-embed (regression for
expect(metadataOf(upsertChunkArgs![0])).toEqual(metadataOf(fullCodeChunk));
});
});
// ────────────────────────────────────────────────────────────────
// #3507 — `embed --stale` must reproduce the page's STORED
// contextual-retrieval wrapping convention instead of embedding raw
// chunk_text (which silently stripped contextual prefixes on every
// re-embed, including the normal post-model-migration path).
// ────────────────────────────────────────────────────────────────
describe('embed --stale contextual-retrieval wrapping (#3507)', () => {
const wrapChunks = [
{ chunk_index: 0, chunk_text: 'prose chunk', chunk_source: 'compiled_truth', embedded_at: null, token_count: 1 },
{ chunk_index: 1, chunk_text: 'const x = 1;', chunk_source: 'fenced_code', embedded_at: null, token_count: 1 },
];
const wrapStale = [
{ slug: 'wrapped', chunk_index: 0, chunk_text: 'prose chunk', chunk_source: 'compiled_truth' as const, model: null, token_count: 1, source_id: 'default', page_id: 1 },
{ slug: 'wrapped', chunk_index: 1, chunk_text: 'const x = 1;', chunk_source: 'fenced_code' as any, model: null, token_count: 1, source_id: 'default', page_id: 1 },
];
function wrappingHarness(mode: string | null) {
const seen: string[] = [];
const restamps: any[][] = [];
embedBatchBehavior = async (texts: string[]) => {
seen.push(...texts);
return texts.map(() => new Float32Array(1536));
};
const engine = mockEngine({
countStaleChunks: async () => 2,
listStaleChunks: async () => wrapStale,
getPage: async () => ({
slug: 'wrapped',
title: 'Widget Notes',
source_id: 'default',
compiled_truth: 'x',
timeline: '',
contextual_retrieval_mode: mode,
}),
getChunks: async () => wrapChunks,
upsertChunks: async () => {},
updatePageContextualRetrievalState: async (...args: any[]) => { restamps.push(args); },
});
return { engine, seen, restamps };
}
test('title-mode page: stale re-embed wraps prose with the title prefix; fenced_code stays raw', async () => {
const { engine, seen, restamps } = wrappingHarness('title');
const result = await runEmbedCore(engine, { stale: true });
expect(result.embedded).toBe(2);
expect(seen).toContain('<context>Widget Notes\n</context>\nprose chunk');
expect(seen).toContain('const x = 1;');
expect(restamps).toHaveLength(0); // title tier: stamp already honest
});
test('per_chunk_synopsis page: fully re-embedded page restamps to the title tier', async () => {
const { engine, seen, restamps } = wrappingHarness('per_chunk_synopsis');
const result = await runEmbedCore(engine, { stale: true });
expect(result.embedded).toBe(2);
expect(seen).toContain('<context>Widget Notes\n</context>\nprose chunk');
expect(restamps).toHaveLength(1);
const [slug, sourceId, newMode] = restamps[0];
expect(slug).toBe('wrapped');
expect(sourceId).toBe('default');
expect(newMode).toBe('title');
});
test('page with no stored CR mode embeds raw chunk_text (convention preserved)', async () => {
const { engine, seen, restamps } = wrappingHarness(null);
const result = await runEmbedCore(engine, { stale: true });
expect(result.embedded).toBe(2);
expect(seen).toContain('prose chunk');
expect(seen.some((t) => t.startsWith('<context>'))).toBe(false);
expect(restamps).toHaveLength(0);
});
});
-45
View File
@@ -672,51 +672,6 @@ describe('runExtractFacts — empty-fence guard (Codex R2-#7)', () => {
});
describe('runExtractFacts — multi-source isolation', () => {
test('a pending legacy row in source A does NOT jam extraction for source B (#2646 source-scope)', async () => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
await (engine as any).db.query(
`INSERT INTO sources (id, name, config) VALUES ('work', 'work', '{}'::jsonb)
ON CONFLICT (id) DO NOTHING`,
);
// Source "work": a genuine pending legacy row (row_num NULL, active,
// live backing page) — the exact shape that must gate work's cycle.
await engine.putPage('people/alice', {
title: 'people/alice', type: 'person',
compiled_truth: FACT_FENCE(`| 1 | work fence fact | fact | 1.0 | world | high | 2026-01-01 | | s | |`),
frontmatter: {}, timeline: '',
}, { sourceId: 'work' });
// eslint-disable-next-line @typescript-eslint/no-explicit-any
await (engine as any).db.query(
`INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, notability,
valid_from, source, confidence)
VALUES ('work', 'people/alice', 'work legacy claim', 'fact', 'private', 'medium',
now(), 'mcp:put_page', 1.0)`,
);
// Source "default": clean — no legacy rows, one fenced page.
await putPage('people/bob', FACT_FENCE(
`| 1 | default fact | fact | 1.0 | world | high | 2026-01-01 | | s | |`,
));
// default's run must NOT be jammed by work's pending backlog.
const rDefault = await runExtractFacts(engine, { slugs: ['people/bob'], sourceId: 'default' });
expect(rDefault.guardTriggered).toBe(false);
expect(rDefault.legacyRowsPending).toBe(0);
expect(rDefault.factsInserted).toBe(1);
// work's own run still gates (discriminator stays sharp).
const rWork = await runExtractFacts(engine, { slugs: ['people/alice'], sourceId: 'work' });
expect(rWork.guardTriggered).toBe(true);
expect(rWork.legacyRowsPending).toBe(1);
expect(rWork.factsInserted).toBe(0);
// The drain advice must be one that actually re-runs Phase B — a bare
// `apply-migrations --yes` no-ops once the ledger says complete.
expect(rWork.warnings.some(w => w.includes('--force-retry 0.32.2'))).toBe(true);
expect(rWork.warnings.some(w => w.includes('forget_fact'))).toBe(true);
expect(rWork.warnings.some(w => w.includes('source "work"'))).toBe(true);
});
test('deleteFactsForPage scoping does not affect other sources', async () => {
// Seed sources work + home.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
-26
View File
@@ -20,7 +20,6 @@ import {
writeReceipt,
type ExtractReceiptInput,
} from '../../src/core/extract/receipt-writer.ts';
import { slugifySegment } from '../../src/core/sync.ts';
const BASE_INPUT: ExtractReceiptInput = {
kind: 'facts.conversation',
@@ -82,31 +81,6 @@ describe('shortRunId / dateFromIso — pure helpers', () => {
expect(shortRunId('op_check_abc')).toBe('op_check');
});
// #3443 — a short form ending in '-' (e.g. propose-<timestamp> run ids)
// desynced the DB receipt slug from its Git-backed slug: slugifySegment()
// strips boundary hyphens during repo sync, so the write-through created a
// normalized sibling instead of materializing the existing page.
test('shortRunId is canonical under slugifySegment for every receipt-producing run-id family (#3443)', () => {
const familyRunIds = [
'propose-20260724103000-ab12cd34', // cycle/propose-takes.ts
`atoms-${Date.now().toString(36)}-pers`, // cycle/extract-atoms.ts
`efacts-${Date.now().toString(36)}-pers`, // cycle/extract-facts.ts
`concepts-${Date.now().toString(36)}`, // cycle/synthesize-concepts.ts
`ecf-${Date.now().toString(36)}-pers`, // extract-conversation-facts.ts
];
for (const runId of familyRunIds) {
const short = shortRunId(runId);
expect(slugifySegment(short)).toBe(short);
expect(short.length).toBeGreaterThan(0);
}
});
test('shortRunId trims boundary hyphens introduced by truncation', () => {
expect(shortRunId('propose-20260724103000-ab12cd34')).toBe('propose');
// Pathological all-separator prefix still yields a non-empty segment.
expect(shortRunId('--------tail')).toBe('run');
});
test('dateFromIso extracts YYYY-MM-DD prefix', () => {
expect(dateFromIso('2026-05-27T14:30:00Z')).toBe('2026-05-27');
expect(dateFromIso('2026-05-27T14:30:00.123456Z')).toBe('2026-05-27');
@@ -9,11 +9,7 @@ import { afterEach, beforeEach, describe, expect, spyOn, test } from 'bun:test';
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import {
__unconfigureGatewayForTests,
isAvailable,
resetGateway,
} from '../src/core/ai/gateway.ts';
import { isAvailable, resetGateway } from '../src/core/ai/gateway.ts';
import { runExtractConversationFacts } from '../src/commands/extract-conversation-facts.ts';
import { runEnrich } from '../src/commands/enrich.ts';
@@ -31,10 +27,7 @@ beforeEach(() => {
}));
process.env.GBRAIN_HOME = home;
process.env.OPENAI_API_KEY = 'test-key';
// Hard-unconfigure: this suite exists to exercise the COLD-gateway path
// (#2590), and resetGateway() now restores the preload's test baseline
// (#3554), which would make configureGatewayIfUninitialized a no-op.
__unconfigureGatewayForTests();
resetGateway();
});
afterEach(() => {
+4 -22
View File
@@ -18,11 +18,7 @@
* `configureGateway()` explicitly in their own beforeAll, which
* overwrites this preload.
*/
import {
configureGateway,
getEmbeddingDimensions,
__setGatewayResetBaselineForTests,
} from '../../src/core/ai/gateway.ts';
import { configureGateway, getEmbeddingDimensions } from '../../src/core/ai/gateway.ts';
import { beforeEach } from 'bun:test';
const LEGACY_CONFIG = {
@@ -30,16 +26,12 @@ const LEGACY_CONFIG = {
embedding_dimensions: 1536,
} as const;
function legacyGatewayConfig() {
return {
function applyLegacy() {
configureGateway({
embedding_model: LEGACY_CONFIG.embedding_model,
embedding_dimensions: LEGACY_CONFIG.embedding_dimensions,
env: { ...process.env },
};
}
function applyLegacy() {
configureGateway(legacyGatewayConfig());
});
}
if (process.env.GBRAIN_DEBUG_PRELOAD === '1') {
@@ -49,16 +41,6 @@ if (process.env.GBRAIN_DEBUG_PRELOAD === '1') {
// Initial application — covers tests that don't reset the gateway.
applyLegacy();
// #3554: make resetGateway() mean "back to this baseline" instead of
// "unconfigured". Without this, a file whose teardown calls resetGateway()
// leaves _config = null; the NEXT file's beforeAll engine-connect then
// reconfigures from the shipped default (zembed-1 @ 1280) BEFORE the
// beforeEach below can fire, and the 1280-sized schema rejects the file's
// 1536-d fixtures. Which file pairs collide depends on shard bin-packing,
// so adding any test file reshuffles the mines. A factory (not a frozen
// config) so each re-application captures fresh process.env.
__setGatewayResetBaselineForTests(legacyGatewayConfig);
// Per-test re-application — handles tests that call `resetGateway()`
// in their setup/teardown. Bun's preload allows registering global
// hooks; this fires before every test in every file in the shard.
+1 -1
View File
@@ -522,7 +522,7 @@ just content.
const result = await importFile(engine, filePath, '🌟🚀.md', { noEmbed: true });
expect(result.status).toBe('skipped');
expect(result.error).toContain('no usable slug');
expect(result.error).toContain('at least one letter or number (any script)');
expect(result.error).toContain('ASCII / Chinese / Japanese / Korean');
expect((engine as any)._calls.length).toBe(0);
});
+1 -4
View File
@@ -15,7 +15,6 @@ import {
} from '../src/core/search/llm-intent.ts';
import {
__setChatTransportForTests,
__unconfigureGatewayForTests,
configureGateway,
resetGateway,
} from '../src/core/ai/gateway.ts';
@@ -123,9 +122,7 @@ describe('classifyModalityWithLLM — fail-open', () => {
});
test('Gateway not configured → returns fallback', async () => {
// Hard-unconfigure: resetGateway() would restore the preload's test
// baseline (#3554), whose {...process.env} could make chat available.
__unconfigureGatewayForTests();
resetGateway();
// No configureGateway called → isAvailable('chat') returns false.
expect(await classifyModalityWithLLM('q', 'text')).toBe('text');
});
+1 -8
View File
@@ -282,19 +282,12 @@ describe('shell-audit: computeAuditFilename', () => {
describe('shell-audit: write', () => {
let tmpDir: string;
// #3554-sibling: the audit-dir preload sets GBRAIN_AUDIT_DIR once at
// process start; deleting it here (instead of restoring) let every file
// AFTER this one in the shard write audit fixtures to the operator's
// real ~/.gbrain/audit/ — and failed audit-dir-preload.test.ts whenever
// bin-packing placed it later in the shard. Restore the prior value.
const priorAuditDir = process.env.GBRAIN_AUDIT_DIR;
beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'shell-audit-test-'));
process.env.GBRAIN_AUDIT_DIR = tmpDir;
});
afterAll(() => {
if (priorAuditDir === undefined) delete process.env.GBRAIN_AUDIT_DIR;
else process.env.GBRAIN_AUDIT_DIR = priorAuditDir;
delete process.env.GBRAIN_AUDIT_DIR;
});
test('GBRAIN_AUDIT_DIR env override resolves to the custom dir', () => {
-29
View File
@@ -354,15 +354,6 @@ describe('MinionQueue: #1737 per-handler default timeout', () => {
expect(sub.timeout_ms).toBe(30 * 60 * 1000);
});
// #3207 — facts-absorb is one LLM extraction call per page (same shape as
// chronicle_extract) but was missing from HANDLER_DEFAULT_TIMEOUT_MS, so it
// inherited the tight null-default wall-clock and was dead-lettered
// mid-generation on slow chat providers (facts silently lost).
test('facts-absorb gets the 10-min LLM-extraction default (#3207)', async () => {
const job = await queue.add('facts-absorb', { slug: 'people/alice-example' });
expect(job.timeout_ms).toBe(10 * 60 * 1000);
});
test('contextual per-chunk reindex gets the 60-min default', async () => {
const job = await queue.add('contextual_reindex_per_chunk', { page_slug: 'large-transcript' }, undefined, {
allowProtectedSubmit: true,
@@ -718,26 +709,6 @@ describe('MinionQueue: Prune', () => {
const count = await queue.prune({ olderThan: new Date(Date.now() + 86400000) }); // future date = prune everything old enough
expect(count).toBe(1); // only the cancelled one
});
// #2712: --dry-run used to be silently ignored — the destructive default
// ran and deleted rows while the operator believed they were previewing.
test('dryRun counts prunable jobs without deleting', async () => {
const job1 = await queue.add('sync', {});
await queue.cancelJob(job1.id); // terminal → prunable
const wouldPrune = await queue.prune({ olderThan: new Date(Date.now() + 86400000), dryRun: true });
expect(wouldPrune).toBe(1);
// The row must still exist after a dry run.
const stillThere = await queue.getJob(job1.id);
expect(stillThere).not.toBeNull();
expect(stillThere!.status).toBe('cancelled');
// A real prune afterwards actually deletes it.
const pruned = await queue.prune({ olderThan: new Date(Date.now() + 86400000) });
expect(pruned).toBe(1);
expect(await queue.getJob(job1.id)).toBeNull();
});
});
// --- Stats (1 test) ---
@@ -19,23 +19,3 @@ describe('CLI_ONLY command reachability (#2900)', () => {
expect(CLI_ONLY.has('reconcile-links')).toBe(true);
});
});
// #3224 — same drift class: `backfill` has a full `case 'backfill'` handler
// (cli.ts, dispatching to commands/backfill.ts) but was missing from CLI_ONLY,
// so every invocation hit the generic "Unknown command" branch.
describe('CLI_ONLY command reachability (#3224)', () => {
test('`backfill` is in CLI_ONLY so dispatch reaches its handler', () => {
expect(CLI_ONLY.has('backfill')).toBe(true);
});
test('`gbrain backfill --help` is dispatched, not rejected as unknown', () => {
const { spawnSync } = require('node:child_process') as typeof import('node:child_process');
const result = spawnSync('bun', ['run', 'src/cli.ts', 'backfill', '--help'], {
cwd: process.cwd(),
encoding: 'utf8',
env: { ...process.env, GBRAIN_HOME: '/tmp/gbrain-test-backfill-nonexistent' },
});
expect(result.stderr ?? '').not.toContain('Unknown command');
expect(result.status).toBe(0);
});
});
-115
View File
@@ -1,115 +0,0 @@
import { describe, test, expect } from 'bun:test';
import { slugifySegment, slugifyPath } from '../src/core/sync.ts';
import { validatePageSlug } from '../src/core/operations.ts';
import { isValidHolder } from '../src/core/takes-fence.ts';
/**
* #3417 silent data loss for non-Latin, non-CJK scripts.
*
* Pre-fix, slugifySegment stripped every character outside [a-z0-9._-] + CJK,
* so whole filenames in Hebrew / Arabic / Cyrillic / Greek / Thai collapsed to
* empty segments. Distinct files then mapped to the SAME slug (their shared
* directory prefix) and last-writer-wins overwrote each other with `import`
* reporting 0 errors.
*
* Every assertion here is behavioral (input output), so this file FAILS on
* pre-fix master and passes with the Unicode-property-escape grammar.
*/
describe('#3417: non-Latin scripts survive slugification', () => {
// The six script families from the issue, before/after.
const cases: Array<[string, string, string]> = [
['Hebrew', 'notes/רשימת קניות.md', 'notes/רשימת-קניות'],
['Arabic', 'notes/قائمة المهام.md', 'notes/قائمة-المهام'],
['Cyrillic', 'notes/Список задач.md', 'notes/список-задач'],
// Greek: tonos marks decompose to U+0301 under NFD and are stripped by the
// same combining-accent pass that turns café → cafe. Consistent, stable.
['Greek', 'notes/Λίστα εργασιών.md', 'notes/λιστα-εργασιων'],
['Thai', 'notes/รายการซื้อของ.md', 'notes/รายการซื้อของ'],
['Hebrew + digits', 'notes/תוכנית עבודה 2026.md', 'notes/תוכנית-עבודה-2026'],
];
for (const [name, input, expected] of cases) {
test(`${name}: ${input}${expected}`, () => {
expect(slugifyPath(input)).toBe(expected);
});
}
test('distinct same-directory files no longer collapse onto one slug', () => {
// Pre-fix ALL of these slugified to "notes" — one page, last writer wins.
const slugs = [
slugifyPath('notes/רשימת קניות.md'),
slugifyPath('notes/قائمة المهام.md'),
slugifyPath('notes/Список задач.md'),
slugifyPath('notes/Λίστα εργασιών.md'),
slugifyPath('notes/รายการซื้อของ.md'),
];
expect(new Set(slugs).size).toBe(slugs.length);
for (const s of slugs) expect(s).not.toBe('notes');
});
test('emitted slugs are ACCEPTED by validatePageSlug (three-grammar coherence)', () => {
// The trap: fixing only sync.ts makes sync emit slugs put_page rejects.
for (const [, input] of cases) {
const slug = slugifyPath(input);
expect(() => validatePageSlug(slug)).not.toThrow();
}
});
test('takes-fence holder grammar accepts non-Latin slugs', () => {
expect(isValidHolder('people/גארי-כהן')).toBe(true);
expect(isValidHolder('companies/شركة-مثال')).toBe(true);
// Uppercase still rejected (lowercase-canonical contract preserved).
expect(isValidHolder('people/Garry-Tan')).toBe(false);
});
});
describe('#3417: normalization — NFD (macOS) and NFC (git/Linux) converge', () => {
test('Hebrew NFD filename produces the same slug as NFC', () => {
const nfc = 'notes/רשימת קניות.md'.normalize('NFC');
const nfd = 'notes/רשימת קניות.md'.normalize('NFD');
expect(slugifyPath(nfd)).toBe(slugifyPath(nfc));
});
test('Vietnamese NFD filename produces the same slug as NFC', () => {
const nfc = 'notes/người dùng.md'.normalize('NFC');
const nfd = 'notes/người dùng.md'.normalize('NFD');
expect(slugifyPath(nfd)).toBe(slugifyPath(nfc));
});
});
describe('#3417: regressions — existing behavior unchanged', () => {
test('ASCII kebab-casing, lowercasing, dots, underscores', () => {
expect(slugifyPath('notes/Shopping List.md')).toBe('notes/shopping-list');
expect(slugifyPath('notes/v1.0.0.md')).toBe('notes/v1.0.0');
expect(slugifySegment('my_file_name')).toBe('my_file_name');
expect(slugifySegment('notes (march 2024)')).toBe('notes-march-2024');
});
test('Latin accents still strip (café → cafe)', () => {
expect(slugifySegment('café résumé')).toBe('cafe-resume');
});
test('CJK still preserved', () => {
expect(slugifyPath('notes/购物清单.md')).toBe('notes/购物清单');
expect(slugifyPath('inbox/品牌圣经.md')).toBe('inbox/品牌圣经');
expect(slugifySegment('한글테스트'.normalize('NFD'))).toBe('한글테스트');
});
test('all-symbol input still collapses to empty (frontmatter-fallback path intact)', () => {
expect(slugifySegment('!!!')).toBe('');
expect(slugifySegment('🎉🎉')).toBe('');
});
test('control chars, RTL override, punctuation still stripped', () => {
expect(slugifySegment('evilgnp')).toBe('evilgnp');
expect(slugifySegment('a\u0000b')).toBe('ab');
});
test('validatePageSlug still rejects traversal, backslash, RTL override, uppercase-only weirdness', () => {
expect(() => validatePageSlug('../etc/passwd')).toThrow();
expect(() => validatePageSlug('notes\\file')).toThrow();
expect(() => validatePageSlug('notes/evil')).toThrow();
expect(() => validatePageSlug('notes/a\u0007b')).toThrow();
});
});
+5 -5
View File
@@ -259,10 +259,10 @@ describe('SLUG_SEGMENT_PATTERN (v0.32.7)', () => {
expect(SLUG_SEGMENT_PATTERN.test('icp-理想客户画像')).toBe(true);
});
test('accepts non-CJK Unicode (Vietnamese) since the #3417 all-script widening', () => {
// Pre-#3417 this was rejected (scope was CJK only). The grammar now uses
// Unicode property escapes, so đ/ư/etc. are valid slug characters.
const result = 'người-dùng'.match(new RegExp(`^${SLUG_SEGMENT_PATTERN.source}$`, 'u'));
expect(result).not.toBeNull();
test('REGRESSION: rejects non-CJK Unicode (Vietnamese)', () => {
// Scope is CJK only; Vietnamese with combining diacritics stays rejected
// until we widen to Unicode property escapes in v0.33+.
const result = 'người-dùng'.match(new RegExp(`^${SLUG_SEGMENT_PATTERN.source}$`));
expect(result).toBeNull();
});
});
-76
View File
@@ -1,76 +0,0 @@
/**
* #2079 `gbrain takes list` used to parse "list" as a PAGE SLUG: cmdList
* looked up a page named "list" and printed "No takes on list." even when the
* brain held many takes reading exactly like an empty takes table, so
* agents concluded there were no takes and moved on.
*
* Fix: `list` is a real subcommand (CLI parity with the takes_list op).
* Bare `takes <slug>` still lists per-page.
*/
import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { runTakes } from '../src/commands/takes.ts';
let engine: PGLiteEngine;
async function captureStdout(fn: () => Promise<void>): Promise<string> {
const lines: string[] = [];
const orig = console.log;
console.log = (...args: unknown[]) => { lines.push(args.join(' ')); };
try {
await fn();
} finally {
console.log = orig;
}
return lines.join('\n');
}
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
await engine.putPage('companies/acme-example', {
type: 'company',
title: 'Acme Example',
compiled_truth: 'Acme Example is a test company.',
});
const [row] = await engine.executeRaw<{ id: number }>(
`SELECT id FROM pages WHERE slug = 'companies/acme-example'`,
);
await engine.addTakesBatch([{
page_id: row.id,
row_num: 1,
claim: 'Acme will ship the widget by Q3.',
kind: 'bet',
holder: 'self',
weight: 0.7,
}]);
});
afterAll(async () => {
await engine.disconnect();
});
describe('gbrain takes list (#2079)', () => {
test('`takes list` lists all takes instead of slug-ifying "list"', async () => {
const out = await captureStdout(() => runTakes(engine, ['list']));
expect(out).not.toContain('No takes on list.');
expect(out).toContain('Acme will ship the widget by Q3.');
expect(out).toContain('companies/acme-example');
});
test('`takes list --json` returns the full take rows', async () => {
const out = await captureStdout(() => runTakes(engine, ['list', '--json']));
const parsed = JSON.parse(out);
expect(Array.isArray(parsed)).toBe(true);
expect(parsed.length).toBe(1);
expect(parsed[0].claim).toContain('Acme will ship');
});
test('per-page form still works: `takes <slug>`', async () => {
const out = await captureStdout(() => runTakes(engine, ['companies/acme-example']));
expect(out).toContain('# Takes on companies/acme-example');
expect(out).toContain('Acme will ship the widget by Q3.');
});
});
+11 -7
View File
@@ -55,20 +55,24 @@ describe('v0.37 Lane A — defaults sweep', () => {
test('A.5: embedding-column registry builtin defaults to ZE/1280 on empty config + gateway', async () => {
// The registry's resolution chain is cfg > gateway > DEFAULT. With
// no cfg AND no gateway, it should fall through to the canonical
// default (ZE/1280). Hard-unconfigure first to exercise that path
// resetGateway() would restore the preload's 1536 baseline (#3554).
const { __unconfigureGatewayForTests, resetGateway } = await import('../src/core/ai/gateway.ts');
// default (ZE/1280). Reset gateway first to exercise that path.
const { resetGateway } = await import('../src/core/ai/gateway.ts');
const { getEmbeddingColumnRegistry } = await import('../src/core/search/embedding-column.ts');
__unconfigureGatewayForTests();
resetGateway();
try {
const reg = getEmbeddingColumnRegistry({ engine: 'pglite' } as any);
expect(reg['embedding']).toBeDefined();
expect(reg['embedding'].provider).toBe('zeroentropyai:zembed-1');
expect(reg['embedding'].dimensions).toBe(1280);
} finally {
// Restore the preload's legacy baseline so the rest of the file's
// tests (and subsequent files in this shard) see a configured gateway.
resetGateway();
// Re-apply legacy preload defaults so the rest of the file's tests
// (and subsequent files in this shard) see a configured gateway.
const { configureGateway } = await import('../src/core/ai/gateway.ts');
configureGateway({
embedding_model: 'openai:text-embedding-3-large',
embedding_dimensions: 1536,
env: { ...process.env },
});
}
});