mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-16 01:42:23 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0c78213d92 |
@@ -1,4 +1,4 @@
|
||||
<!-- gbrain-runbook-stamp: 0.46.1.0 -->
|
||||
<!-- gbrain-runbook-stamp: 0.46.2.0 -->
|
||||
<!-- This stamp must equal the VERSION file at every release; CI enforces it
|
||||
(scripts/check-bootstrap-tag.sh). `gbrain bootstrap status` compares it to
|
||||
the installed binary and warns on skew. -->
|
||||
|
||||
@@ -2,6 +2,72 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.46.2.0] - 2026-08-15
|
||||
|
||||
**Dream synthesis now triages before it spends.**
|
||||
([#4152](https://github.com/garrytan/gbrain/issues/4152)) The synthesize
|
||||
phase used to point its most expensive model at every transcript that
|
||||
cleared a yes/no check — on a busy brain that meant an unbounded queue of
|
||||
long frontier-model jobs grinding through logistics and small talk. It is
|
||||
now a two-stage cascade: a cheap scored triage reads every file first, and
|
||||
only what scores above your threshold reaches the synthesis model, which
|
||||
starts from a map of the noteworthy passages instead of hunting through raw
|
||||
transcript.
|
||||
|
||||
### Added
|
||||
- **Scored triage gate.** Every transcript gets a 0–1 salience score,
|
||||
content type, candidate quotes, and entity candidates from the utility-tier
|
||||
model (one call per new file, cached in `dream_verdicts` with the judging
|
||||
model + prompt version — migration v129). The gate
|
||||
(`dream.triage.threshold`, default 0.5) is applied at read time: retune it
|
||||
any time and re-gating costs **zero** new LLM calls. Provider hiccups
|
||||
(truncation, refusal, unparseable output) are never cached as rejections —
|
||||
those files are re-judged next cycle, and an outage reports as "triage
|
||||
degraded", never as "everything scored low".
|
||||
- **`gbrain dream retriage`** — re-score the corpus and reconcile the queued
|
||||
synthesis backlog. `--dry-run` previews from cached scores with zero LLM
|
||||
calls; `--reconcile-queue` cancels queued jobs that score below the gate
|
||||
AND converts jobs stranded in dead per-run queues so the next cycle
|
||||
actually re-submits them; `--audit-rejects <n>` gets a frontier-model
|
||||
second opinion on a sample of rejects (the threshold-calibration loop).
|
||||
Every sweep prints an upfront cost estimate and asks before spending more
|
||||
than a few dollars (`--yes` to skip, `--max-usd` for an estimate-based
|
||||
budget stop that counts every paid call, including unreliable ones — it
|
||||
can overshoot by up to the configured triage concurrency). Guardrails: queues
|
||||
younger than an hour are treated as possibly-live and never touched;
|
||||
`--cancel-unmatched` refuses to run off a truncated or empty corpus scan.
|
||||
- **Triage map in the synthesis prompt.** Passing files hand the synthesis
|
||||
subagent their pre-extracted quotes and entities (verbatim-verified against
|
||||
the chunk text) so it works from signal instead of re-scanning sludge.
|
||||
- **Cost knobs.** `dream.synthesize.max_turns` (default now 16, was a
|
||||
hardcoded 30 — set it back via config if your written-page counts drop;
|
||||
`details.synthesis.avg_turns` shows cap pressure),
|
||||
`dream.triage.max_ms` (per-cycle triage time budget, default 5 min — a big
|
||||
cold corpus triages across a few cycles, with deferred files labeled "not
|
||||
yet triaged", never silently rejected), and an opt-in per-source daily
|
||||
synthesis cap (`dream.synthesize.max_submissions_per_source_per_day`,
|
||||
default off; 200/day is a sane value for busy deployments). The intended
|
||||
pairing is the shipped mid-tier synthesis default — frontier-model
|
||||
overrides are unnecessary with triage doing the reading.
|
||||
|
||||
### Fixed
|
||||
- A run whose submissions were all skipped (cap, already-synthesized) no
|
||||
longer starts the 12-hour cooldown, so the skipped files retry on the next
|
||||
cycle instead of waiting half a day.
|
||||
- Synthesis jobs stranded in a dead per-run queue by a killed cycle are
|
||||
self-healed on the next run (cancelled and re-submitted into the live
|
||||
queue) instead of stalling the phase for the full 35-minute wait.
|
||||
- `gbrain dream retriage --help` (and richer `gbrain dream --help`) now
|
||||
print real usage instead of the generic one-line stub, with no brain
|
||||
configured.
|
||||
|
||||
To take advantage of v0.46.2.0: upgrade and run `gbrain dream` as usual —
|
||||
existing verdicts are re-scored automatically on the next cycle (cheap,
|
||||
utility-tier). If you have a queued synthesis backlog, run
|
||||
`gbrain dream retriage --dry-run` to preview, then
|
||||
`gbrain dream retriage --reconcile-queue` to drain it for pennies. Tune
|
||||
`dream.triage.threshold` freely; re-gating is free.
|
||||
|
||||
## [0.46.1.0] - 2026-08-15
|
||||
|
||||
**A stuck job can no longer take down your whole worker.** Field reports from
|
||||
|
||||
@@ -6080,3 +6080,66 @@ covers DEAD logs; go-forward capture beyond Claude Code is deliberately absent.
|
||||
carry verbatim observation transcripts; consider extending check-privacy.sh
|
||||
(or a dedicated check) to assert pin docs use `<tmp>`/placeholder paths and
|
||||
never carry key material or account ids. Effort: S.
|
||||
|
||||
## Dream triage cascade follow-ups (#4152, filed at implementation)
|
||||
|
||||
- [ ] **P2 — Incremental submit-drain + deadline threading in synthesize
|
||||
fan-out.** What: restructure the fan-out to submit bounded batches and
|
||||
drain each before submitting more, stopping against the parent job's
|
||||
`deadlineAtMs`. Why: today the phase bulk-submits every accepted child
|
||||
then drains sequentially inside `autopilot-cycle`'s 30-min wall clock
|
||||
(`handler-timeouts.ts:44`); a timeout mid-drain strands the remainder in
|
||||
the run's private queue (the C1 self-heal + retriage conversion now
|
||||
recover them, but not creating strands beats recovering them). Blocked
|
||||
by: `runCycle` does not thread deadline/abort into phases (verified
|
||||
absent at the synthesize call site, cycle.ts ~2030). Context: outside
|
||||
voice C2 on the #4152 eng review; the triage `max_ms` budget bounds the
|
||||
cheap half, this bounds the expensive half. Effort: M/L.
|
||||
- [ ] **P2 — Scheduled reject sample-audit with spend-posture
|
||||
integration.** What: automate `dream retriage --audit-rejects N` on a
|
||||
cadence (weekly cron or post-cycle sampling) writing disagreement-rate
|
||||
telemetry, gated by `spend.posture`. Why: the threshold is an
|
||||
intuition-set 0.5 until real false-negative data exists; the cascade
|
||||
literature is unanimous that unaudited gates drift (eng-review search
|
||||
check). The manual flag ships with #4152; this files the loop that runs
|
||||
without an operator remembering. Depends on: a few weeks of production
|
||||
score distributions. Effort: M.
|
||||
- [ ] **P3 — Borderline-band routing (0.30–0.49 → mid-tier model or batch
|
||||
digest).** What: a second lane where near-threshold files get a cheaper
|
||||
treatment instead of the binary keep/drop. Why: the issue marked it
|
||||
optional; it adds a third model lane + a second threshold pair, which
|
||||
should be tuned from `details.triage` score distributions rather than
|
||||
guessed. Blocked by: production calibration data (see the audit TODO
|
||||
above). Effort: M.
|
||||
- [ ] **P3 — Source×corpus multiplier: per-source corpus mapping or
|
||||
explicit fan-out consent.** What: `dream.synthesize.session_corpus_dir`
|
||||
is GLOBAL config while synth idempotency keys are SOURCE-namespaced, so
|
||||
N registered sources each re-fan the same corpus (a live deployment saw
|
||||
3 × ~1,250 jobs/day of the same files). Triage verdicts are
|
||||
source-agnostic (judged once) and the cascade cuts each source's fanout
|
||||
by the pass rate, but total synthesis is still N× the corpus. Why
|
||||
deferred: pages land per-source, so per-source synthesis may be intended
|
||||
semantics for some operators — needs its own issue + design (per-source
|
||||
corpus config keys vs an explicit multi-source consent flag). Diagnostic:
|
||||
`dream retriage --reconcile-queue --json` reports `queue.by_source`.
|
||||
Context: outside voice C3 argued root-cause-first; scoped out twice
|
||||
during the #4152 review. Comment on #4152 after ship. Effort: M.
|
||||
- [ ] **P3 — Dream triage perf follow-ups (from the #4152 ship review).**
|
||||
What: (a) batch the per-file `getDreamVerdict` PK probes in `runTriagePass`
|
||||
into one prefetch (unnest join on (file_path, content_hash)) and reuse it
|
||||
for retriage's spend-estimate loop (currently 2×N sequential roundtrips on
|
||||
the operator sweep); (b) a partial index for `countRecentSynthSubmissions`
|
||||
(`(created_at) WHERE name='subagent' AND idempotency_key LIKE
|
||||
'dream:synth-v2:%'`) so the opt-in daily cap's count is index-served on
|
||||
busy brains; (c) a shared `seedTriageVerdict` test helper to collapse the
|
||||
five hand-rolled triage-v1 seed blocks. Why: all flagged by the ship
|
||||
review's performance/maintainability specialists; none block — cache
|
||||
probes are ~0.1% of adjacent LLM latency and the cap is default-off.
|
||||
Effort: M.
|
||||
- [ ] **P3 — Per-file single-flight for triage cache misses.** What:
|
||||
concurrent passes (retriage while a cycle runs) can double-judge the same
|
||||
uncached file (~1¢/file, last-write-wins converges — benign but untidy);
|
||||
a per-(file,hash) advisory claim would dedupe. Why deferred: real locks
|
||||
are heavy machinery for a benign-cost race; the retriage help documents
|
||||
the behavior. Context: outside-voice CX5 on the #4152 ship review.
|
||||
Effort: M.
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -86,7 +86,7 @@ the repo. The architectural rule still holds — these aren't
|
||||
| `mcp_request_log` | Audit trail. Volatile by design. |
|
||||
| `minion_jobs` / `minion_inbox` / `minion_attachments` | Job queue. Restarts re-enqueue or drop. |
|
||||
| `eval_candidates` / `eval_capture_failures` | Contributor-mode dev loop; opt-in capture. |
|
||||
| `dream_verdicts` | Cheap verdict cache. Rebuildable by re-running Haiku. |
|
||||
| `dream_verdicts` | Scored triage cache (salience score, quotes, entities, judging model + prompt version). Rebuildable via `gbrain dream retriage --force`. |
|
||||
| `gbrain_cycle_locks` / migration ledger | Infrastructure. |
|
||||
| `op_checkpoint_paths` | Sync-resume checkpoint. Append-only progress banking; a completed sync makes it irrelevant. |
|
||||
| `config` (some keys) | Site-local routing config (e.g. `sync.repo_path`). |
|
||||
|
||||
@@ -115,6 +115,49 @@ it nightly and Phase 4 below (plus most of Phase 2's hygiene checks) is
|
||||
covered. The pseudocode that follows is the harness-side variant for agents
|
||||
that also do LLM-driven entity sweeps and memory consolidation on top.
|
||||
|
||||
### Synthesis cost control: the triage cascade
|
||||
|
||||
The synthesize phase is a two-stage cascade: a cheap scored triage
|
||||
(utility-tier model, one call per new transcript) gates the expensive
|
||||
per-transcript synthesis subagents. The dials:
|
||||
|
||||
- `dream.triage.threshold` (default 0.5) — the gate. Scores are cached, so
|
||||
retuning it re-gates instantly with **zero** new LLM calls. Raise it if too
|
||||
much routine content synthesizes; lower it if real signal is being skipped.
|
||||
- `models.dream.triage` — the triage model (default: utility tier / Haiku).
|
||||
- `dream.triage.max_chars` (default 24000, floor 1000) — per-transcript
|
||||
sample window (head/middle/tail) sent to the judge. Not part of cache
|
||||
validity — after changing it, `gbrain dream retriage --force` re-judges
|
||||
under the new sampling.
|
||||
- `dream.triage.max_tokens` (default 2048, floor 256) — judge output budget.
|
||||
- `dream.triage.concurrency` (default 4, clamped 1–16) — concurrent judge
|
||||
calls.
|
||||
- `dream.synthesize.max_turns` (default 16) — synthesis turn budget. The
|
||||
triage map hands the subagent pre-extracted segments, so the mid-tier
|
||||
default model (`models.dream.synthesize`, tier `reasoning`) with a 16-turn
|
||||
budget is the intended pairing — frontier-model overrides are unnecessary
|
||||
and slow the queue. Completeness comes from triage coverage (every file
|
||||
scored, minus files deferred under the `max_ms` budget below) plus
|
||||
segment-guided prompts, not model size. If written-page counts
|
||||
drop after upgrading, set it back to 30 and check
|
||||
`details.synthesis.avg_turns` for cap pressure.
|
||||
- `dream.triage.max_ms` (default 5 min) — per-cycle wall-clock budget for
|
||||
judging NEW files; a big cold corpus triages across a few cycles (cached
|
||||
files are free). Deferred files are labeled "not yet triaged", never
|
||||
silently rejected.
|
||||
- `dream.synthesize.max_submissions_per_source_per_day` (default 0 = off) —
|
||||
opt-in backstop cap on synthesis jobs per source; 200/day is a sane value
|
||||
for busy deployments.
|
||||
|
||||
Maintenance recipe — after changing the threshold, upgrading through a
|
||||
`TRIAGE_VERSION` bump, or to drain a queued synthesis backlog:
|
||||
|
||||
```bash
|
||||
gbrain dream retriage --dry-run # what would change (zero LLM calls)
|
||||
gbrain dream retriage --reconcile-queue # re-score + cancel below-threshold queued jobs
|
||||
gbrain dream retriage --audit-rejects 20 # synthesis-model second opinion on 20 rejects
|
||||
```
|
||||
|
||||
### What It Does
|
||||
|
||||
```
|
||||
|
||||
@@ -23,7 +23,7 @@ gbrain config set spend.posture gated # default — gates enforce
|
||||
| Value | Effect |
|
||||
|-------|--------|
|
||||
| `gated` (default) | Every cost gate enforces its limit as documented below. |
|
||||
| `tokenmax` | Every embedding-spend gate in the table below prints its estimate and **proceeds** — informational only. Spend is still recorded to the ledger; posture removes the *ceiling*, not the *accounting*. (Commands with their own LLM cost caps outside this doc's embedding scope — e.g. `extract-conversation-facts --max-cost-usd` — don't resolve posture; their per-call flags govern.) |
|
||||
| `tokenmax` | Every embedding-spend gate in the table below prints its estimate and **proceeds** — informational only. Spend is still recorded to the ledger; posture removes the *ceiling*, not the *accounting*. (Commands with their own LLM cost caps outside this doc's embedding scope — e.g. `extract-conversation-facts --max-cost-usd`, `dream retriage --max-usd` (an estimate-based soft stop) — don't resolve posture; their per-call flags govern.) |
|
||||
|
||||
`spend.posture` is deliberately separate from `search.mode=tokenmax` (which governs
|
||||
retrieval payload size, not embedding spend). When a gate fires and
|
||||
|
||||
+44
-1
@@ -1543,7 +1543,7 @@ wins; fix the row.
|
||||
| "Migrate from Obsidian/Notion/Logseq" | `skills/migrate/SKILL.md` |
|
||||
| Brain health check, maintenance run | `skills/maintain/SKILL.md` |
|
||||
| "Extract links", "build link graph", "populate timeline" | `skills/maintain/SKILL.md` (extraction sections) |
|
||||
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run" | `skills/maintain/SKILL.md` (dream cycle section) |
|
||||
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run", "retriage the backlog", "re-score the triage" | `skills/maintain/SKILL.md` (dream cycle section) |
|
||||
| "Brain health", "what features am I missing", "brain score" | Run `gbrain features --json` |
|
||||
| "Set up autopilot", "run brain maintenance", "keep brain updated" | Run `gbrain autopilot --install --repo ~/brain` |
|
||||
| "Upgrade gbrain", "update gbrain", "gbrain update available", `UPGRADE_AVAILABLE`, "is gbrain up to date" | `skills/gbrain-upgrade/SKILL.md` |
|
||||
@@ -3234,6 +3234,49 @@ it nightly and Phase 4 below (plus most of Phase 2's hygiene checks) is
|
||||
covered. The pseudocode that follows is the harness-side variant for agents
|
||||
that also do LLM-driven entity sweeps and memory consolidation on top.
|
||||
|
||||
### Synthesis cost control: the triage cascade
|
||||
|
||||
The synthesize phase is a two-stage cascade: a cheap scored triage
|
||||
(utility-tier model, one call per new transcript) gates the expensive
|
||||
per-transcript synthesis subagents. The dials:
|
||||
|
||||
- `dream.triage.threshold` (default 0.5) — the gate. Scores are cached, so
|
||||
retuning it re-gates instantly with **zero** new LLM calls. Raise it if too
|
||||
much routine content synthesizes; lower it if real signal is being skipped.
|
||||
- `models.dream.triage` — the triage model (default: utility tier / Haiku).
|
||||
- `dream.triage.max_chars` (default 24000, floor 1000) — per-transcript
|
||||
sample window (head/middle/tail) sent to the judge. Not part of cache
|
||||
validity — after changing it, `gbrain dream retriage --force` re-judges
|
||||
under the new sampling.
|
||||
- `dream.triage.max_tokens` (default 2048, floor 256) — judge output budget.
|
||||
- `dream.triage.concurrency` (default 4, clamped 1–16) — concurrent judge
|
||||
calls.
|
||||
- `dream.synthesize.max_turns` (default 16) — synthesis turn budget. The
|
||||
triage map hands the subagent pre-extracted segments, so the mid-tier
|
||||
default model (`models.dream.synthesize`, tier `reasoning`) with a 16-turn
|
||||
budget is the intended pairing — frontier-model overrides are unnecessary
|
||||
and slow the queue. Completeness comes from triage coverage (every file
|
||||
scored, minus files deferred under the `max_ms` budget below) plus
|
||||
segment-guided prompts, not model size. If written-page counts
|
||||
drop after upgrading, set it back to 30 and check
|
||||
`details.synthesis.avg_turns` for cap pressure.
|
||||
- `dream.triage.max_ms` (default 5 min) — per-cycle wall-clock budget for
|
||||
judging NEW files; a big cold corpus triages across a few cycles (cached
|
||||
files are free). Deferred files are labeled "not yet triaged", never
|
||||
silently rejected.
|
||||
- `dream.synthesize.max_submissions_per_source_per_day` (default 0 = off) —
|
||||
opt-in backstop cap on synthesis jobs per source; 200/day is a sane value
|
||||
for busy deployments.
|
||||
|
||||
Maintenance recipe — after changing the threshold, upgrading through a
|
||||
`TRIAGE_VERSION` bump, or to drain a queued synthesis backlog:
|
||||
|
||||
```bash
|
||||
gbrain dream retriage --dry-run # what would change (zero LLM calls)
|
||||
gbrain dream retriage --reconcile-queue # re-score + cancel below-threshold queued jobs
|
||||
gbrain dream retriage --audit-rejects 20 # synthesis-model second opinion on 20 rejects
|
||||
```
|
||||
|
||||
### What It Does
|
||||
|
||||
```
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"id": "gbrain-context-engine",
|
||||
"name": "gbrain",
|
||||
"version": "0.46.1.0",
|
||||
"version": "0.46.2.0",
|
||||
"description": "Personal knowledge brain with Postgres + pgvector hybrid search",
|
||||
"family": "bundle-plugin",
|
||||
"configSchema": {
|
||||
|
||||
+1
-1
@@ -157,7 +157,7 @@
|
||||
"bun": ">=1.3.10"
|
||||
},
|
||||
"license": "MIT",
|
||||
"version": "0.46.1.0",
|
||||
"version": "0.46.2.0",
|
||||
"overrides": {
|
||||
"@hono/node-server": "^2.0.5",
|
||||
"fast-uri": "^3.1.5",
|
||||
|
||||
+1
-1
@@ -107,7 +107,7 @@ wins; fix the row.
|
||||
| "Migrate from Obsidian/Notion/Logseq" | `skills/migrate/SKILL.md` |
|
||||
| Brain health check, maintenance run | `skills/maintain/SKILL.md` |
|
||||
| "Extract links", "build link graph", "populate timeline" | `skills/maintain/SKILL.md` (extraction sections) |
|
||||
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run" | `skills/maintain/SKILL.md` (dream cycle section) |
|
||||
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run", "retriage the backlog", "re-score the triage" | `skills/maintain/SKILL.md` (dream cycle section) |
|
||||
| "Brain health", "what features am I missing", "brain score" | Run `gbrain features --json` |
|
||||
| "Set up autopilot", "run brain maintenance", "keep brain updated" | Run `gbrain autopilot --install --repo ~/brain` |
|
||||
| "Upgrade gbrain", "update gbrain", "gbrain update available", `UPGRADE_AVAILABLE`, "is gbrain up to date" | `skills/gbrain-upgrade/SKILL.md` |
|
||||
|
||||
@@ -12,7 +12,7 @@ Four tiers:
|
||||
|
||||
| Tier | Purpose | Default | Examples |
|
||||
|---|---|---|---|
|
||||
| `utility` | fast classification, expansion, verdict, dedup | `claude-haiku-4-5-20251001` | query expansion, facts contradiction classifier, dream synthesize verdict |
|
||||
| `utility` | fast classification, expansion, verdict, dedup | `claude-haiku-4-5-20251001` | query expansion, facts contradiction classifier, dream triage judge (prefers `models.dream.triage`) |
|
||||
| `reasoning` | default chat, synthesis, generation | `claude-sonnet-4-6` | gateway chat, dream synthesize, patterns, facts extraction |
|
||||
| `deep` | slow, expensive reasoning | `claude-opus-4-7` | `gbrain think`, auto-think, cross-modal eval slot B |
|
||||
| `subagent` | Anthropic-only multi-turn tool loop | `claude-sonnet-4-6` | `gbrain agent run` |
|
||||
@@ -28,6 +28,10 @@ Override priority (highest first):
|
||||
7. Tier default (the table above)
|
||||
8. Hardcoded caller fallback
|
||||
|
||||
One exception: the dream triage judge pre-reads `models.dream.triage` first —
|
||||
when that key is set, it wins over this entire chain (`gbrain models` reports
|
||||
it as the effective route).
|
||||
|
||||
Power-user recipes:
|
||||
|
||||
```bash
|
||||
|
||||
+28
-12
@@ -18,6 +18,8 @@ triggers:
|
||||
- "populate links"
|
||||
- "backfill graph"
|
||||
- "extract timeline entries"
|
||||
- "retriage the backlog"
|
||||
- "re-score the triage"
|
||||
- "run dream"
|
||||
- "process today's session"
|
||||
- "process yesterday's transcripts"
|
||||
@@ -116,7 +118,8 @@ gbrain extract timeline --dir ~/brain
|
||||
|
||||
### Dream cycle (v0.23): synthesize + patterns
|
||||
|
||||
`gbrain dream` runs the full 8-phase maintenance cycle:
|
||||
`gbrain dream` runs the full maintenance cycle (core phases shown; opt-in
|
||||
phases like atoms/concepts/drift slot in between):
|
||||
|
||||
```
|
||||
lint -> backlinks -> sync -> synthesize -> extract -> patterns -> embed -> orphans
|
||||
@@ -124,14 +127,25 @@ lint -> backlinks -> sync -> synthesize -> extract -> patterns -> embed -> orpha
|
||||
|
||||
The two new phases consolidate yesterday's conversations into long-term memory:
|
||||
|
||||
**Synthesize phase:** reads transcripts from `dream.synthesize.session_corpus_dir`,
|
||||
runs a cheap Haiku verdict (cached in `dream_verdicts`) to filter routine
|
||||
ops sessions, then fans out one Sonnet subagent per worth-processing
|
||||
transcript. Each subagent writes reflections (`wiki/personal/reflections/...`),
|
||||
originals (`wiki/originals/ideas/...`), and people timeline entries. The
|
||||
orchestrator collects the slugs from `subagent_tool_executions` (NOT
|
||||
`pages.updated_at` — that would pick up unrelated writes) and reverse-renders
|
||||
each new page from DB → markdown on disk.
|
||||
**Synthesize phase (two-stage cascade):** reads transcripts from
|
||||
`dream.synthesize.session_corpus_dir`, then triages before it spends: a cheap
|
||||
utility-tier judge (`models.dream.triage`) scores every new file 0–1 for
|
||||
salience and pre-extracts candidate quotes + entities, cached in
|
||||
`dream_verdicts` with the judging model + prompt version (bounded per cycle
|
||||
by `dream.triage.max_ms`, default 5 min — deferred files retry next cycle,
|
||||
never silently rejected). Only files scoring
|
||||
at or above `dream.triage.threshold` (default 0.5 — applied at read time, so
|
||||
retuning the threshold re-gates with zero new LLM calls) fan out one synthesis
|
||||
subagent per transcript chunk, each primed with the triage map and capped at
|
||||
`dream.synthesize.max_turns` (default 16). Each subagent writes reflections
|
||||
(`wiki/personal/reflections/...`), originals (`wiki/originals/ideas/...`), and
|
||||
people timeline entries. The orchestrator collects the slugs from
|
||||
`subagent_tool_executions` (NOT `pages.updated_at` — that would pick up
|
||||
unrelated writes) and reverse-renders each new page from DB → markdown on
|
||||
disk. To re-apply the gate after retuning the threshold or drain a queued
|
||||
backlog, run `gbrain dream retriage --dry-run` (zero LLM calls, cached
|
||||
scores only) then `gbrain dream retriage --reconcile-queue`; `--force`
|
||||
re-judges everything from scratch.
|
||||
|
||||
**Patterns phase:** runs after `extract` (so the graph state is fresh).
|
||||
Reads recent reflections within `dream.patterns.lookback_days` (default 30),
|
||||
@@ -164,15 +178,17 @@ timestamp is stored in `dream.synthesize.last_completion_ts` and is written
|
||||
ONLY on successful runs (not on skipped/failed). Explicit `--input` /
|
||||
`--date` / `--from` / `--to` invocations bypass cooldown.
|
||||
|
||||
**`--dry-run` semantics:** runs the cheap Haiku significance filter (caches
|
||||
verdicts) but skips the Sonnet synthesis pass. NOT zero LLM calls.
|
||||
**`--dry-run` semantics:** runs the scored triage pass (judges + caches
|
||||
verdicts for new files) but skips the synthesis subagents. NOT zero LLM
|
||||
calls — for a zero-call preview from cached scores use
|
||||
`gbrain dream retriage --dry-run` instead.
|
||||
|
||||
**Configure synthesize on a fresh brain:**
|
||||
```bash
|
||||
gbrain config set dream.synthesize.session_corpus_dir /path/to/transcripts
|
||||
gbrain config set dream.synthesize.enabled true
|
||||
gbrain dream --phase synthesize --dry-run --json # preview
|
||||
gbrain dream # full 8-phase cycle
|
||||
gbrain dream # full cycle
|
||||
```
|
||||
|
||||
**Invocation patterns:**
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"RESOLVER.md": "8e54195c109c764d2954186ee92a30a62cd91e961233be53f828db5a25ebe710",
|
||||
"RESOLVER.md": "36b43c65a41e6fce894b9559db2bce0a53f06a99450410e498323c12e12e92bb",
|
||||
"_AGENT_README.md": "62613f7f1e061576b6c1b18844f59bd35f2df96ca5c45c8c41fae0772b9ce4d3",
|
||||
"_brain-filing-rules.json": "cf850df6a7425464c6d63b3ace71991cc93497fa0cc8cd21acd31883e17939c6",
|
||||
"_brain-filing-rules.md": "2d2d75b7c76081c56f41b2c0a5a978c355ce957300f9b0a5575dc4079ef1f877",
|
||||
@@ -48,7 +48,7 @@
|
||||
"conventions/cron-via-minions.md": "badb1cd6cd825d6f1ac0b6b28cc47e5d80facc783a3e59a14146ae901ee0f933",
|
||||
"conventions/cross-modal.yaml": "c012c3d72614a87b1ee698173dce2a0fb0d057a54df7aab87993c4b07fff6280",
|
||||
"conventions/exec-output.md": "2bf371ac3ec4987eff7cc13cd3ea8cc97c46bd43f588eec024ff27f3171bc58f",
|
||||
"conventions/model-routing.md": "fb7ae8746a578500d6789b68ff40049037aa4d337b65b42f7c1745ae7080c2db",
|
||||
"conventions/model-routing.md": "8b28aa706436e7b68493ec481e12be6309b0af1a553930e8b1029d675fa4b3ad",
|
||||
"conventions/path-discipline.md": "8af5415721bd115e6979c96688bcf542a32675809ada9aaedbb6706a40926954",
|
||||
"conventions/quality.md": "8aa681001114689d34268ccadaf0e2ff07b8f68aa5987c093a8c4a7a744f12a6",
|
||||
"conventions/regex-discipline.md": "d96a9baa6f27184e165889a9c655607366a739d851684d9f41cdec294f99edac",
|
||||
@@ -88,7 +88,7 @@
|
||||
"idea-lineage/routing-eval.jsonl": "ee2e00704b9accb7dd58bb8f126a3bc04a2c40be499180fa505dbf6d5061cd41",
|
||||
"ingest/SKILL.md": "dc40ecc0072806fb8c7bb6ab9cf1f103842e05653eb55d67632d7e3ffc4dd7d2",
|
||||
"install/SKILL.md": "881bd0a422f34c6df4642aae66c51e2a4cc18ad5ca6d0b52d44b4de93512a3c4",
|
||||
"maintain/SKILL.md": "59da3f0227a2b41ed9c3beb334322f1ef1dbd80af733a587c0a4687a707b9815",
|
||||
"maintain/SKILL.md": "33e48e31baf89b6b257ad863cdb9de444777bc1272f5ed8c2b28be3a54cbaa14",
|
||||
"manifest.json": "03471868cce05fa38af6f793da54e2fc11f77ef778271a596d75bc29f9ec4c73",
|
||||
"measure-before-you-fix/SKILL.md": "1fd3b40ab65cbd08f50dea16107701859165469be3c85c57d779c7b4bbf92db8",
|
||||
"measure-before-you-fix/routing-eval.jsonl": "0661df9974a9cfe31216d574b1db0ef341945c2eb844ebf4ab6920fcbbc90d6c",
|
||||
|
||||
@@ -162,6 +162,11 @@ const CLI_ONLY_SELF_HELP = new Set([
|
||||
// so `jobs work --help` prints help instead of starting a worker daemon.
|
||||
// Without this entry the generic stub hid the worker entry point entirely.
|
||||
'jobs',
|
||||
// #4152: dream ships its own printHelp AND the `dream retriage --help`
|
||||
// subverb help (dispatched engine-free before parseArgs). The generic stub
|
||||
// would hide both — `gbrain dream retriage --help` printed the one-line
|
||||
// dream stub instead of the retriage contract (outside-voice CX9).
|
||||
'dream',
|
||||
]);
|
||||
|
||||
/**
|
||||
@@ -184,6 +189,9 @@ const SELF_HELP_WITHOUT_ENGINE: Record<string, () => Promise<(engine: never, arg
|
||||
// runJobs accepts BrainEngine | null and its help guard returns before any
|
||||
// engine (or subcommand body) is touched.
|
||||
jobs: async () => (await import('./commands/jobs.ts')).runJobs as never,
|
||||
// runDream accepts BrainEngine | null; --help (and `retriage --help`) is
|
||||
// answered before any engine-bearing work per the dream.ts IRON RULE.
|
||||
dream: async () => (await import('./commands/dream.ts')).runDream as never,
|
||||
};
|
||||
|
||||
/** Returns true when the command's own help was printed. */
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
/**
|
||||
* Spend-gate constants for `gbrain dream retriage` (#4152, outside-voice C12).
|
||||
* Split from dream-retriage.ts so tests can pin them without importing the
|
||||
* command's engine-bearing module graph. The chars→tokens ratio lives in
|
||||
* synthesize.ts (`CHARS_PER_TOKEN`, exported) — the command imports it from
|
||||
* there so the two estimates can't drift.
|
||||
*/
|
||||
|
||||
/** Estimated sweeps above this ask for confirmation unless --yes. */
|
||||
export const SPEND_CONFIRM_USD = 5;
|
||||
|
||||
/** When the model has no CANONICAL_PRICING entry, gate on file count instead. */
|
||||
export const UNPRICED_CONFIRM_FILES = 500;
|
||||
@@ -0,0 +1,697 @@
|
||||
/**
|
||||
* `gbrain dream retriage` (#4152) — re-score the corpus and reconcile the
|
||||
* synth-v2 job backlog against the triage gate.
|
||||
*
|
||||
* Two halves, both optional:
|
||||
* 1. Re-judge: sweep the discovered corpus through runTriagePass
|
||||
* (`--force` ignores the cache; `--since` treats older verdicts as
|
||||
* stale; `--dry-run` performs ZERO judge calls and reads cached scores
|
||||
* only). Spend-gated: prints an upfront estimate and asks for
|
||||
* confirmation above ~$5 (`--yes` skips; `--max-usd` soft-stops).
|
||||
* 2. `--reconcile-queue` (opt-in — cancels queued work): parse every
|
||||
* waiting/delayed/paused `dream:synth-v2:*` job across ALL queues (the
|
||||
* live backlog largely sits in dead per-run `dream-inline-*` queues no
|
||||
* worker will ever drain), match (basename, hash16) to discovered
|
||||
* transcripts, then:
|
||||
* - matched below threshold → cancel (frontier job not worth it)
|
||||
* - matched above, stale queue → cancel as `converted_for_resubmit`
|
||||
* (cancelled rows release their idempotency slot, so the next cycle
|
||||
* re-adds them into ITS live private drain — this is what actually
|
||||
* migrates the backlog, outside-voice C1)
|
||||
* - matched above, live queue → keep
|
||||
* - matched but unscored → keep (never cancel on no data)
|
||||
* - unmatched → keep unless `--cancel-unmatched`
|
||||
* - key-source ≠ data.source_id → skip + count (C9 hardening)
|
||||
* Cancellation is best-effort: status is re-checked immediately before
|
||||
* each cancel and rows that turned `active` are skipped; the residual
|
||||
* claim-vs-cancel race matches cancelJob's own BullMQ-style contract.
|
||||
*
|
||||
* `--audit-rejects <n>` (C6): re-judges N stride-sampled (deterministic) below-threshold files with
|
||||
* the SYNTHESIS model and reports the disagreement rate — the operator-run
|
||||
* calibration loop for `dream.triage.threshold`.
|
||||
*
|
||||
* Exit codes: 0 success (even when nothing cancelled), 1 missing corpus
|
||||
* config / engine, 2 usage error.
|
||||
*/
|
||||
|
||||
import { basename } from 'node:path';
|
||||
import { createInterface } from 'node:readline';
|
||||
import type { BrainEngine, DreamVerdict } from '../core/engine.ts';
|
||||
import {
|
||||
loadSynthConfig,
|
||||
runTriagePass,
|
||||
parseSynthV2Key,
|
||||
makeJudgeClient,
|
||||
judgeSignificance,
|
||||
isTriageCacheValid,
|
||||
dreamInlineQueueAgeMs,
|
||||
DREAM_INLINE_LIVE_GRACE_MS,
|
||||
CHARS_PER_TOKEN,
|
||||
type TriageFileReport,
|
||||
} from '../core/cycle/synthesize.ts';
|
||||
import { discoverTranscripts } from '../core/cycle/transcript-discovery.ts';
|
||||
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
|
||||
import { MinionQueue } from '../core/minions/queue.ts';
|
||||
import { canonicalLookup } from '../core/model-pricing.ts';
|
||||
import { SPEND_CONFIRM_USD, UNPRICED_CONFIRM_FILES } from './dream-retriage-constants.ts';
|
||||
|
||||
interface RetriageArgs {
|
||||
help: boolean;
|
||||
threshold: number | null;
|
||||
since: Date | null;
|
||||
force: boolean;
|
||||
reconcileQueue: boolean;
|
||||
cancelUnmatched: boolean;
|
||||
dryRun: boolean;
|
||||
limit: number | null;
|
||||
source: string | null;
|
||||
json: boolean;
|
||||
yes: boolean;
|
||||
maxUsd: number | null;
|
||||
auditRejects: number | null;
|
||||
}
|
||||
|
||||
class UsageError extends Error {}
|
||||
|
||||
function parseRetriageArgs(args: string[]): RetriageArgs {
|
||||
const out: RetriageArgs = {
|
||||
help: false,
|
||||
threshold: null,
|
||||
since: null,
|
||||
force: false,
|
||||
reconcileQueue: false,
|
||||
cancelUnmatched: false,
|
||||
dryRun: false,
|
||||
limit: null,
|
||||
source: null,
|
||||
json: false,
|
||||
yes: false,
|
||||
maxUsd: null,
|
||||
auditRejects: null,
|
||||
};
|
||||
const takeValue = (flag: string, i: number): string => {
|
||||
const v = args[i + 1];
|
||||
if (v === undefined || v.startsWith('--')) throw new UsageError(`${flag} requires a value`);
|
||||
return v;
|
||||
};
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
switch (a) {
|
||||
case '--help': case '-h': out.help = true; break;
|
||||
case '--force': out.force = true; break;
|
||||
case '--reconcile-queue': out.reconcileQueue = true; break;
|
||||
case '--cancel-unmatched': out.cancelUnmatched = true; break;
|
||||
case '--dry-run': out.dryRun = true; break;
|
||||
case '--json': out.json = true; break;
|
||||
case '--yes': out.yes = true; break;
|
||||
case '--threshold': {
|
||||
const v = Number(takeValue(a, i)); i++;
|
||||
if (!Number.isFinite(v) || v < 0 || v > 1) throw new UsageError('--threshold must be a number in [0,1]');
|
||||
out.threshold = v;
|
||||
break;
|
||||
}
|
||||
case '--since': {
|
||||
const raw = takeValue(a, i); i++;
|
||||
const ms = Date.parse(raw);
|
||||
if (Number.isNaN(ms)) throw new UsageError(`--since could not parse date: ${raw}`);
|
||||
out.since = new Date(ms);
|
||||
break;
|
||||
}
|
||||
case '--limit': {
|
||||
const v = parseInt(takeValue(a, i), 10); i++;
|
||||
if (!Number.isFinite(v) || v < 1) throw new UsageError('--limit must be a positive integer');
|
||||
out.limit = v;
|
||||
break;
|
||||
}
|
||||
case '--source': case '--source-id': {
|
||||
out.source = takeValue(a, i); i++;
|
||||
break;
|
||||
}
|
||||
case '--max-usd': {
|
||||
const v = Number(takeValue(a, i)); i++;
|
||||
if (!Number.isFinite(v) || v <= 0) throw new UsageError('--max-usd must be a positive number');
|
||||
out.maxUsd = v;
|
||||
break;
|
||||
}
|
||||
case '--audit-rejects': {
|
||||
const v = parseInt(takeValue(a, i), 10); i++;
|
||||
if (!Number.isFinite(v) || v < 1) throw new UsageError('--audit-rejects must be a positive integer');
|
||||
out.auditRejects = v;
|
||||
break;
|
||||
}
|
||||
default:
|
||||
throw new UsageError(`unknown flag for dream retriage: ${a}`);
|
||||
}
|
||||
}
|
||||
if (out.cancelUnmatched && !out.reconcileQueue) {
|
||||
throw new UsageError('--cancel-unmatched requires --reconcile-queue');
|
||||
}
|
||||
// CX2: a corpus scan truncated by --limit would misclassify every file outside
|
||||
// the slice as "unmatched" — combining it with --cancel-unmatched would
|
||||
// mass-cancel valid backlog.
|
||||
if (out.cancelUnmatched && out.limit !== null) {
|
||||
throw new UsageError('--cancel-unmatched cannot combine with --limit: a truncated scan misclassifies files as unmatched');
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function printRetriageHelp(): void {
|
||||
console.log(`gbrain dream retriage — re-score the corpus, reconcile the synth backlog
|
||||
|
||||
USAGE
|
||||
gbrain dream retriage [flags]
|
||||
|
||||
FLAGS
|
||||
--threshold <0..1> Gate override for this sweep (default: dream.triage.threshold)
|
||||
--since <date> Treat verdicts judged before <date> as stale (re-judge)
|
||||
--force Re-judge every discovered file regardless of cache
|
||||
--limit <n> Only consider the first n discovered transcripts
|
||||
--dry-run Zero judge calls, zero cancels; report from cached scores
|
||||
--reconcile-queue Cancel waiting synth jobs per the gate (opt-in; see below)
|
||||
--cancel-unmatched With --reconcile-queue: also cancel jobs whose file no
|
||||
longer matches any discovered transcript
|
||||
--source <id> Scope queue reconciliation to one source's jobs
|
||||
--yes Skip the spend confirmation
|
||||
--max-usd <n> Soft-stop judging when the ESTIMATED spend crosses n
|
||||
(estimate-based; every judge attempt counts, including
|
||||
unreliable responses; may overshoot by up to the
|
||||
configured concurrency; requires a priced model; also
|
||||
bounds --audit-rejects)
|
||||
--audit-rejects <n> Re-judge n stride-sampled (deterministic) below-threshold
|
||||
files with the SYNTHESIS model; report the disagreement
|
||||
rate. Skipped under --dry-run. Counted in the spend gate.
|
||||
--json Machine-readable output
|
||||
--help This text
|
||||
|
||||
NOTES
|
||||
--cancel-unmatched cannot combine with --limit (a truncated corpus scan
|
||||
would misclassify everything outside the slice as unmatched), and refuses
|
||||
to run when discovery finds zero transcripts (a corpus-mount outage must
|
||||
not erase the queued retry frontier). dream-inline-* queues younger than
|
||||
1h are treated as possibly LIVE (a running cycle's drain) and are never
|
||||
cancelled — they count as kept_live_queue. Running retriage while a cycle
|
||||
is active may double-judge some cache misses (benign: last write wins).
|
||||
|
||||
RECONCILE SEMANTICS
|
||||
matched + score < threshold cancel
|
||||
matched + score >= threshold, dead dream-inline-* queue (older than 1h)
|
||||
cancel (converted_for_resubmit —
|
||||
next cycle re-adds it into a live drain)
|
||||
matched + score >= threshold, live queue keep
|
||||
matched, no reliable score keep
|
||||
unmatched / unparseable key keep (cancel with --cancel-unmatched)
|
||||
legacy dream:synth: (v1) keys never touched
|
||||
|
||||
Cancelled rows release their idempotency slot, so lowering the threshold
|
||||
later cleanly re-submits the work.`);
|
||||
}
|
||||
|
||||
interface QueueCandidate {
|
||||
id: number;
|
||||
queue: string;
|
||||
status: string;
|
||||
idempotency_key: string;
|
||||
source_id: string;
|
||||
}
|
||||
|
||||
interface ReconcileStats {
|
||||
candidates: number;
|
||||
cancelled: number;
|
||||
converted_for_resubmit: number;
|
||||
kept_above_threshold: number;
|
||||
kept_unscored: number;
|
||||
/** Rows in a dream-inline-* queue younger than the liveness grace — possibly a running cycle's; never cancelled (CX1). */
|
||||
kept_live_queue: number;
|
||||
unmatched: number;
|
||||
unmatched_cancelled: number;
|
||||
source_mismatch: number;
|
||||
other_source: number;
|
||||
already_terminal: number;
|
||||
by_source: Record<string, number>;
|
||||
by_queue_kind: { dream_inline: number; other: number };
|
||||
}
|
||||
|
||||
/** Per-file cost estimate in USD for one triage judge call; null when the model is unpriced. */
|
||||
function estimatePerFileUsd(model: string, maxChars: number, maxTokens: number): number | null {
|
||||
const pricing = canonicalLookup(model);
|
||||
if (!pricing) return null;
|
||||
const inputTokens = maxChars / CHARS_PER_TOKEN;
|
||||
return (inputTokens / 1_000_000) * pricing.input + (maxTokens / 1_000_000) * pricing.output;
|
||||
}
|
||||
|
||||
async function confirmOnTty(prompt: string): Promise<boolean> {
|
||||
if (!process.stdin.isTTY) return false;
|
||||
const rl = createInterface({ input: process.stdin, output: process.stderr });
|
||||
const answer = await new Promise<string>(resolve => rl.question(`${prompt} [y/N] `, resolve));
|
||||
rl.close();
|
||||
return /^y(es)?$/i.test(answer.trim());
|
||||
}
|
||||
|
||||
export async function runDreamRetriage(engine: BrainEngine | null, args: string[]): Promise<void> {
|
||||
let parsed: RetriageArgs;
|
||||
try {
|
||||
parsed = parseRetriageArgs(args);
|
||||
} catch (e) {
|
||||
if (e instanceof UsageError) {
|
||||
console.error(`dream retriage: ${e.message} (see: gbrain dream retriage --help)`);
|
||||
setCliExitVerdict(2);
|
||||
return;
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
// IRON RULE: --help short-circuits before any engine-bearing work.
|
||||
if (parsed.help) {
|
||||
printRetriageHelp();
|
||||
return;
|
||||
}
|
||||
if (engine === null) {
|
||||
console.error('gbrain dream retriage requires a connected brain; run `gbrain init` first');
|
||||
setCliExitVerdict(1);
|
||||
return;
|
||||
}
|
||||
|
||||
const config = await loadSynthConfig(engine);
|
||||
if (!config.corpusDir) {
|
||||
console.error('dream retriage: dream.synthesize.session_corpus_dir is unset — nothing to retriage');
|
||||
setCliExitVerdict(1);
|
||||
return;
|
||||
}
|
||||
const threshold = parsed.threshold ?? config.triage.threshold;
|
||||
|
||||
let transcripts = discoverTranscripts({
|
||||
corpusDir: config.corpusDir,
|
||||
meetingTranscriptsDir: config.meetingTranscriptsDir ?? undefined,
|
||||
minChars: config.minChars,
|
||||
excludePatterns: config.excludePatterns,
|
||||
});
|
||||
if (parsed.limit !== null) transcripts = transcripts.slice(0, parsed.limit);
|
||||
|
||||
// ── Half 1: score the corpus (cached reads in --dry-run; judged otherwise) ──
|
||||
let reports: TriageFileReport[];
|
||||
const byPath = new Map<string, DreamVerdict>();
|
||||
let passStats = { judged: 0, cacheHits: 0, unreliable: 0, deferred: 0 };
|
||||
// Estimated spend accumulated across the triage sweep AND the reject audit —
|
||||
// one budget spans both halves (CX3 + security review).
|
||||
let estimatedSpendUsd = 0;
|
||||
|
||||
if (parsed.dryRun) {
|
||||
// Zero judge calls: read cached verdicts only. Files without a valid
|
||||
// triage-v1 score report as needs_triage.
|
||||
reports = [];
|
||||
for (const t of transcripts) {
|
||||
const cached = await engine.getDreamVerdict(t.filePath, t.contentHash);
|
||||
const valid = cached !== null && !parsed.force
|
||||
&& isTriageCacheValid(cached, config.triage.model, parsed.since ?? undefined);
|
||||
if (cached && valid) {
|
||||
passStats.cacheHits++;
|
||||
byPath.set(t.filePath, cached);
|
||||
reports.push({
|
||||
filePath: t.filePath,
|
||||
worth: cached.score !== null && cached.score >= threshold,
|
||||
score: cached.score,
|
||||
content_type: cached.content_type,
|
||||
reasons: cached.reasons,
|
||||
cached: true,
|
||||
});
|
||||
} else {
|
||||
reports.push({
|
||||
filePath: t.filePath,
|
||||
worth: false,
|
||||
score: null,
|
||||
content_type: null,
|
||||
reasons: ['needs_triage (dry-run performs no judge calls)'],
|
||||
cached: false,
|
||||
deferred: true,
|
||||
});
|
||||
passStats.deferred++;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Spend gate (outside-voice C12): estimate the miss count upfront and
|
||||
// confirm above SPEND_CONFIRM_USD unless --yes.
|
||||
let missCount = 0;
|
||||
for (const t of transcripts) {
|
||||
if (parsed.force) { missCount++; continue; }
|
||||
const cached = await engine.getDreamVerdict(t.filePath, t.contentHash);
|
||||
const valid = cached !== null
|
||||
&& isTriageCacheValid(cached, config.triage.model, parsed.since ?? undefined);
|
||||
if (!valid) missCount++;
|
||||
}
|
||||
const perFileUsd = estimatePerFileUsd(config.triage.model, config.triage.maxChars, config.triage.maxTokens);
|
||||
// CX3: --max-usd is estimate-based; an unpriced model would silently
|
||||
// disable the budget the operator explicitly asked for — refuse instead.
|
||||
if (parsed.maxUsd !== null && perFileUsd === null) {
|
||||
console.error(
|
||||
`dream retriage: --max-usd requires a priced model; "${config.triage.model}" has no CANONICAL_PRICING entry`,
|
||||
);
|
||||
setCliExitVerdict(2);
|
||||
return;
|
||||
}
|
||||
// The frontier audit spends too (security review): fold its worst case
|
||||
// into the gated estimate so --audit-rejects can't ride around the gate.
|
||||
const auditPerFileUsd = parsed.auditRejects !== null
|
||||
? estimatePerFileUsd(config.model, config.triage.maxChars, config.triage.maxTokens)
|
||||
: null;
|
||||
// Codex structured review P1: an unpriced SYNTHESIS model would zero out
|
||||
// the audit's share of the estimate AND silently disable --max-usd inside
|
||||
// the audit loop — refuse the budget flag, and always confirm when the
|
||||
// audit spend cannot be estimated.
|
||||
const auditUnpriced = parsed.auditRejects !== null && auditPerFileUsd === null;
|
||||
if (parsed.maxUsd !== null && auditUnpriced) {
|
||||
console.error(
|
||||
`dream retriage: --max-usd with --audit-rejects requires a priced synthesis model; ` +
|
||||
`"${config.model}" has no CANONICAL_PRICING entry`,
|
||||
);
|
||||
setCliExitVerdict(2);
|
||||
return;
|
||||
}
|
||||
const auditEstimateUsd = parsed.auditRejects !== null && auditPerFileUsd !== null
|
||||
? parsed.auditRejects * auditPerFileUsd
|
||||
: 0;
|
||||
// Structured-review round 2 P1: the KNOWN portion of the estimate gates
|
||||
// independently of whether the triage model is priced — an unpriced
|
||||
// triage model with a large PRICED audit must still confirm on the audit
|
||||
// dollars, not slide through the file-count gate on cached rejects.
|
||||
const knownEstimateUsd = (perFileUsd ?? 0) * missCount + auditEstimateUsd;
|
||||
const estimateUsd = perFileUsd === null ? null : knownEstimateUsd;
|
||||
const gateTriggered = knownEstimateUsd > SPEND_CONFIRM_USD
|
||||
|| (perFileUsd === null && missCount > UNPRICED_CONFIRM_FILES)
|
||||
|| auditUnpriced; // un-estimable audit spend always confirms
|
||||
const auditSuffix = auditUnpriced
|
||||
? ` (audit model "${config.model}" unpriced — audit spend cannot be estimated)`
|
||||
: auditEstimateUsd > 0 ? ` (incl. ≤ $${auditEstimateUsd.toFixed(2)} audit)` : '';
|
||||
const estimateLine = estimateUsd !== null
|
||||
? `[retriage] ${missCount} file(s) to judge with ${config.triage.model} — estimated ≤ $${estimateUsd.toFixed(2)}${auditSuffix}`
|
||||
: `[retriage] ${missCount} file(s) to judge with ${config.triage.model} — no pricing entry for this model (cannot estimate; the ${UNPRICED_CONFIRM_FILES}-file confirmation gate applies)${auditSuffix}`;
|
||||
process.stderr.write(estimateLine + '\n');
|
||||
if (gateTriggered && !parsed.yes) {
|
||||
if (parsed.json || !process.stdin.isTTY) {
|
||||
console.error('dream retriage: spend estimate exceeds the confirmation gate; re-run with --yes (non-interactive)');
|
||||
setCliExitVerdict(2);
|
||||
return;
|
||||
}
|
||||
const ok = await confirmOnTty(`Proceed with ~$${estimateUsd?.toFixed(2) ?? '?'} of triage spend?`);
|
||||
if (!ok) {
|
||||
console.error('dream retriage: aborted at spend confirmation');
|
||||
setCliExitVerdict(2);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// --max-usd soft-stop: estimate-based (usage isn't threaded through the
|
||||
// judge seam); stops pulling new misses once attempts × per-file estimate
|
||||
// crosses the budget. runTriagePass ticks shouldStop on EVERY judge
|
||||
// attempt (CX3 — unreliable responses are paid calls too); may overshoot
|
||||
// by up to the configured concurrency. Remaining files report as deferred.
|
||||
const shouldStop = parsed.maxUsd !== null && perFileUsd !== null
|
||||
? (): boolean => {
|
||||
estimatedSpendUsd += perFileUsd;
|
||||
return estimatedSpendUsd >= parsed.maxUsd!;
|
||||
}
|
||||
: undefined;
|
||||
|
||||
const pass = await runTriagePass(engine, transcripts, {
|
||||
model: config.triage.model,
|
||||
maxChars: config.triage.maxChars,
|
||||
maxTokens: config.triage.maxTokens,
|
||||
threshold,
|
||||
concurrency: config.triage.concurrency,
|
||||
maxMs: 0, // operator sweep runs to completion; --limit / --max-usd bound it
|
||||
force: parsed.force,
|
||||
staleBefore: parsed.since ?? undefined,
|
||||
shouldStop,
|
||||
});
|
||||
reports = pass.reports;
|
||||
for (const [k, v] of pass.byPath) byPath.set(k, v);
|
||||
passStats = { judged: pass.judged, cacheHits: pass.cacheHits, unreliable: pass.unreliable, deferred: pass.deferred };
|
||||
}
|
||||
|
||||
// ── Half 2: queue reconciliation (opt-in) ──
|
||||
let reconcile: ReconcileStats | null = null;
|
||||
if (parsed.reconcileQueue) {
|
||||
const queue = new MinionQueue(engine);
|
||||
const rows = await engine.executeRaw<QueueCandidate>(
|
||||
`SELECT id, queue, status, idempotency_key,
|
||||
COALESCE(NULLIF(data->>'source_id', ''), 'default') AS source_id
|
||||
FROM minion_jobs
|
||||
WHERE name = 'subagent'
|
||||
AND status IN ('waiting', 'delayed', 'paused')
|
||||
AND idempotency_key LIKE 'dream:synth-v2:%'`,
|
||||
);
|
||||
// Two lookups: membership in the discovered corpus (matched at all?) vs a
|
||||
// usable scored verdict. A discovered file with no reliable score is
|
||||
// "matched but unscored" — kept, never cancelled on missing data. The
|
||||
// '|' join is unambiguous even for basenames containing '|': hash16 is
|
||||
// fixed-width hex after the final separator.
|
||||
const discoveredKeys = new Set<string>();
|
||||
const verdictByKey = new Map<string, DreamVerdict>();
|
||||
for (const t of transcripts) {
|
||||
const k = `${basename(t.filePath)}|${t.contentHash.slice(0, 16)}`;
|
||||
discoveredKeys.add(k);
|
||||
const v = byPath.get(t.filePath);
|
||||
if (v) verdictByKey.set(k, v);
|
||||
}
|
||||
// CX2: an empty discovery result alongside a non-empty backlog means the
|
||||
// corpus is unreachable (mount outage, permissions, wrong dir) far more
|
||||
// often than it means every file was deleted. Refuse to cancel-unmatched
|
||||
// in that state — a transient outage must not erase the retry frontier.
|
||||
if (parsed.cancelUnmatched && transcripts.length === 0 && rows.length > 0) {
|
||||
console.error(
|
||||
`dream retriage: discovery found 0 transcripts but ${rows.length} queued job(s) exist; ` +
|
||||
'refusing --cancel-unmatched (corpus may be unreachable). Fix discovery or drop the flag.',
|
||||
);
|
||||
setCliExitVerdict(2);
|
||||
return;
|
||||
}
|
||||
reconcile = {
|
||||
candidates: rows.length,
|
||||
cancelled: 0,
|
||||
converted_for_resubmit: 0,
|
||||
kept_above_threshold: 0,
|
||||
kept_unscored: 0,
|
||||
kept_live_queue: 0,
|
||||
unmatched: 0,
|
||||
unmatched_cancelled: 0,
|
||||
source_mismatch: 0,
|
||||
other_source: 0,
|
||||
already_terminal: 0,
|
||||
by_source: {},
|
||||
by_queue_kind: { dream_inline: 0, other: 0 },
|
||||
};
|
||||
// Codex structured review P2: queue age alone is not liveness — a cycle
|
||||
// with several slow sequential children can legitimately exceed the 1h
|
||||
// grace. Consult the REAL signal: a live (unexpired) cycle lock means a
|
||||
// cycle is running right now, so no dream-inline queue is provably dead.
|
||||
const liveLocks = await engine.executeRaw<{ id: string }>(
|
||||
`SELECT id FROM gbrain_cycle_locks WHERE ttl_expires_at > NOW() AND id LIKE 'gbrain-cycle%'`,
|
||||
);
|
||||
// Structured-review round 2 P2: cycle locks are per-source
|
||||
// (`gbrain-cycle:<source>`) — a cycle running for source A must not
|
||||
// suppress conversions for source B indefinitely. Only the legacy bare
|
||||
// `gbrain-cycle` lock is global.
|
||||
const globalLockLive = liveLocks.some(l => l.id === 'gbrain-cycle');
|
||||
const liveLockSources = new Set(
|
||||
liveLocks
|
||||
.map(l => (l.id.startsWith('gbrain-cycle:') ? l.id.slice('gbrain-cycle:'.length) : null))
|
||||
.filter((s): s is string => s !== null),
|
||||
);
|
||||
if (liveLocks.length > 0) {
|
||||
process.stderr.write(
|
||||
`[retriage] live cycle lock(s) detected (${liveLocks.map(l => l.id).join(', ')}); ` +
|
||||
`dream-inline queues for those sources are treated as possibly-live — conversions skipped\n`,
|
||||
);
|
||||
}
|
||||
const cancelRow = async (id: number): Promise<'cancelled' | 'already_terminal'> => {
|
||||
// Pre-cancel status re-check (C9): a candidate claimed by a live worker
|
||||
// between the snapshot SELECT and now is skipped, not killed.
|
||||
const fresh = await engine.executeRaw<{ status: string }>(
|
||||
`SELECT status FROM minion_jobs WHERE id = $1`, [id],
|
||||
);
|
||||
const status = fresh[0]?.status;
|
||||
if (status !== 'waiting' && status !== 'delayed' && status !== 'paused') return 'already_terminal';
|
||||
const r = await queue.cancelJob(id);
|
||||
return r ? 'cancelled' : 'already_terminal';
|
||||
};
|
||||
for (const row of rows) {
|
||||
reconcile.by_source[row.source_id] = (reconcile.by_source[row.source_id] ?? 0) + 1;
|
||||
const inlineQueueAge = dreamInlineQueueAgeMs(row.queue);
|
||||
const isInlineQueue = inlineQueueAge !== null || row.queue.startsWith('dream-inline-');
|
||||
if (isInlineQueue) reconcile.by_queue_kind.dream_inline++;
|
||||
else reconcile.by_queue_kind.other++;
|
||||
// CX1: a dream-inline-* queue younger than the liveness grace may belong
|
||||
// to a cycle that is RUNNING right now — its inline drain will claim
|
||||
// these rows. Never cancel anything in a possibly-live private queue
|
||||
// (unparseable-timestamp names count as possibly-live, fail-safe), and
|
||||
// a live cycle lock FOR THIS ROW'S SOURCE marks its inline queues
|
||||
// possibly-live regardless of age (structured-review P2: slow
|
||||
// sequential children can outlive the grace; round 2: per-source, so a
|
||||
// busy source A never suppresses source B's cleanup indefinitely).
|
||||
const lockLiveForRow = globalLockLive || liveLockSources.has(row.source_id);
|
||||
const possiblyLiveQueue = isInlineQueue
|
||||
&& (lockLiveForRow || inlineQueueAge === null || inlineQueueAge <= DREAM_INLINE_LIVE_GRACE_MS);
|
||||
if (possiblyLiveQueue) {
|
||||
reconcile.kept_live_queue++;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (parsed.source !== null && row.source_id !== parsed.source) {
|
||||
reconcile.other_source++;
|
||||
continue;
|
||||
}
|
||||
const key = parseSynthV2Key(row.idempotency_key);
|
||||
if (!key) {
|
||||
reconcile.unmatched++;
|
||||
continue;
|
||||
}
|
||||
// C9 hardening: the key's encoded source must agree with the payload's
|
||||
// source_id — never cancel on a disagreement.
|
||||
if ((key.source || 'default') !== row.source_id) {
|
||||
reconcile.source_mismatch++;
|
||||
continue;
|
||||
}
|
||||
const matchKey = `${key.basename}|${key.hash16}`;
|
||||
const verdict = verdictByKey.get(matchKey);
|
||||
if (!verdict || verdict.score === null) {
|
||||
// Unmatched file OR matched-but-unscored (deferred/degraded): only the
|
||||
// truly-unmatched are cancellable, and only behind --cancel-unmatched.
|
||||
const isMatchedUnscored = discoveredKeys.has(matchKey);
|
||||
if (isMatchedUnscored) {
|
||||
reconcile.kept_unscored++;
|
||||
} else if (parsed.cancelUnmatched) {
|
||||
// Structured-review P2: the dry-run preview must count would-cancel
|
||||
// unmatched rows the same way the below-threshold branch does — a
|
||||
// destructive preview that understates its impact is worse than none.
|
||||
if (parsed.dryRun) {
|
||||
reconcile.unmatched_cancelled++; // dry-run: would cancel
|
||||
} else {
|
||||
const outcome = await cancelRow(row.id);
|
||||
if (outcome === 'cancelled') reconcile.unmatched_cancelled++;
|
||||
else reconcile.already_terminal++;
|
||||
}
|
||||
} else {
|
||||
reconcile.unmatched++;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (verdict.score < threshold) {
|
||||
if (!parsed.dryRun) {
|
||||
const outcome = await cancelRow(row.id);
|
||||
if (outcome === 'cancelled') reconcile.cancelled++;
|
||||
else reconcile.already_terminal++;
|
||||
} else {
|
||||
reconcile.cancelled++; // dry-run: would cancel
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// Above threshold. C1: a row stranded in a provably-dead per-run
|
||||
// dream-inline-* queue (older than the liveness grace, no live cycle
|
||||
// lock — the possibly-live case was already kept above) will never be
|
||||
// claimed — cancel it so the next cycle's queue.add re-creates it in a
|
||||
// live drain (the cancelled row releases its idempotency slot).
|
||||
// `delayed` counts too (structured-review P2): a transient-failure
|
||||
// retry parked in a dead queue has no worker to promote or drain it.
|
||||
if (isInlineQueue && (row.status === 'waiting' || row.status === 'delayed')) {
|
||||
if (!parsed.dryRun) {
|
||||
const outcome = await cancelRow(row.id);
|
||||
if (outcome === 'cancelled') reconcile.converted_for_resubmit++;
|
||||
else reconcile.already_terminal++;
|
||||
} else {
|
||||
reconcile.converted_for_resubmit++; // dry-run: would convert
|
||||
}
|
||||
continue;
|
||||
}
|
||||
reconcile.kept_above_threshold++;
|
||||
}
|
||||
}
|
||||
|
||||
// ── --audit-rejects (C6): frontier second opinion on N stride-sampled rejects ──
|
||||
let audit: { sampled: number; disagreements: number; disagreement_rate: number | null } | null = null;
|
||||
if (parsed.auditRejects !== null && parsed.dryRun) {
|
||||
// Loud no-op instead of a silently-null audit field (maintainability review).
|
||||
process.stderr.write('[retriage] --audit-rejects skipped under --dry-run (the audit spends frontier-model calls)\n');
|
||||
}
|
||||
if (parsed.auditRejects !== null && !parsed.dryRun) {
|
||||
const rejects = reports.filter(r => r.score !== null && r.score < threshold);
|
||||
// Deterministic stride-sample over the rejects in discovery order — no
|
||||
// randomness, so repeated audits compare like with like.
|
||||
const sample: TriageFileReport[] = [];
|
||||
const stride = Math.max(1, Math.floor(rejects.length / parsed.auditRejects));
|
||||
for (let i = 0; i < rejects.length && sample.length < parsed.auditRejects; i += stride) sample.push(rejects[i]);
|
||||
const frontier = makeJudgeClient(config.model);
|
||||
if (!frontier) {
|
||||
process.stderr.write(`[retriage] --audit-rejects: no reachable provider for ${config.model}; skipping audit\n`);
|
||||
} else {
|
||||
const byFilePath = new Map(transcripts.map(t => [t.filePath, t]));
|
||||
const auditPerFileUsd = estimatePerFileUsd(config.model, config.triage.maxChars, config.triage.maxTokens);
|
||||
let disagreements = 0;
|
||||
let judged = 0;
|
||||
for (const r of sample) {
|
||||
const t = byFilePath.get(r.filePath);
|
||||
if (!t) continue;
|
||||
// --max-usd spans the audit too (CX3): stop before the next frontier
|
||||
// call would cross the budget.
|
||||
if (parsed.maxUsd !== null && auditPerFileUsd !== null
|
||||
&& estimatedSpendUsd + auditPerFileUsd > parsed.maxUsd) {
|
||||
process.stderr.write(`[retriage] --audit-rejects stopped at --max-usd $${parsed.maxUsd.toFixed(2)} (audited ${judged})\n`);
|
||||
break;
|
||||
}
|
||||
try {
|
||||
const second = await judgeSignificance(frontier, t, config.model, {
|
||||
maxChars: config.triage.maxChars,
|
||||
maxTokens: config.triage.maxTokens,
|
||||
});
|
||||
estimatedSpendUsd += auditPerFileUsd ?? 0;
|
||||
if (second.unreliable) continue;
|
||||
judged++;
|
||||
if (second.score >= threshold) disagreements++;
|
||||
} catch {
|
||||
// A failed call is still an attempt — count its estimated cost.
|
||||
estimatedSpendUsd += auditPerFileUsd ?? 0;
|
||||
// Audit is best-effort; a failed second opinion is skipped.
|
||||
}
|
||||
}
|
||||
audit = {
|
||||
sampled: judged,
|
||||
disagreements,
|
||||
disagreement_rate: judged > 0 ? Math.round((disagreements / judged) * 1000) / 1000 : null,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const passCount = reports.filter(r => r.worth).length;
|
||||
const summary = {
|
||||
discovered: transcripts.length,
|
||||
threshold,
|
||||
pass: passCount,
|
||||
below_threshold: reports.filter(r => r.score !== null && !r.worth).length,
|
||||
needs_triage: reports.filter(r => r.deferred).length,
|
||||
retriaged: passStats.judged,
|
||||
cache_hits: passStats.cacheHits,
|
||||
unreliable: passStats.unreliable,
|
||||
deferred: passStats.deferred,
|
||||
dry_run: parsed.dryRun,
|
||||
queue: reconcile,
|
||||
audit,
|
||||
};
|
||||
|
||||
if (parsed.json) {
|
||||
console.log(JSON.stringify(summary, null, 2));
|
||||
} else {
|
||||
const would = parsed.dryRun ? ' (dry-run: no cancels performed)' : '';
|
||||
console.log(`[retriage] ${summary.discovered} discovered | ${summary.pass} pass @ threshold ${threshold} | ` +
|
||||
`${summary.below_threshold} below | ${summary.needs_triage} need triage | ` +
|
||||
`${summary.retriaged} judged, ${summary.cache_hits} cached, ${summary.unreliable} unreliable`);
|
||||
if (reconcile) {
|
||||
console.log(`[retriage] queue: ${reconcile.candidates} candidates | ${reconcile.cancelled} cancelled | ` +
|
||||
`${reconcile.converted_for_resubmit} converted for resubmit | ${reconcile.kept_above_threshold} kept | ` +
|
||||
`${reconcile.kept_unscored} unscored kept | ${reconcile.kept_live_queue} live-queue kept | ${reconcile.unmatched} unmatched | ` +
|
||||
`${reconcile.source_mismatch} source-mismatch skipped | ${reconcile.already_terminal} already terminal${would}`);
|
||||
const bySource = Object.entries(reconcile.by_source).map(([s, n]) => `${s}=${n}`).join(', ');
|
||||
if (bySource) console.log(`[retriage] queue by source: ${bySource} | stale dream-inline queues: ${reconcile.by_queue_kind.dream_inline}`);
|
||||
}
|
||||
if (audit) {
|
||||
console.log(`[retriage] reject audit: ${audit.sampled} re-judged by ${config.model}, ` +
|
||||
`${audit.disagreements} disagreements (rate ${audit.disagreement_rate ?? 'n/a'})`);
|
||||
}
|
||||
}
|
||||
}
|
||||
+38
-2
@@ -32,6 +32,7 @@ import {
|
||||
type CycleReport,
|
||||
} from '../core/cycle.ts';
|
||||
import { resolveSourceId } from '../core/source-resolver.ts';
|
||||
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
|
||||
import { fetchSource } from '../core/sources-load.ts';
|
||||
import { existsSync } from 'fs';
|
||||
import { resolve } from 'node:path';
|
||||
@@ -346,6 +347,7 @@ async function resolveBrainDir(
|
||||
|
||||
function printHelp() {
|
||||
console.log(`Usage: gbrain dream [options]
|
||||
gbrain dream retriage [flags] (see: gbrain dream retriage --help)
|
||||
|
||||
Run one brain maintenance cycle. Eight phases:
|
||||
lint -> backlinks -> sync -> synthesize -> extract -> patterns -> embed -> orphans
|
||||
@@ -354,10 +356,17 @@ The synthesize + patterns phases (v0.21) consolidate yesterday's
|
||||
conversation transcripts into reflections, originals, and cross-session
|
||||
pattern pages. Designed for cron (exits when done).
|
||||
|
||||
The synthesize phase (#4152) runs a two-stage cascade: a cheap scored triage
|
||||
(model: models.dream.triage, gate: dream.triage.threshold, default 0.5) gates
|
||||
the expensive per-transcript synthesis subagents (turn budget:
|
||||
dream.synthesize.max_turns, default 16). Retune the threshold any time —
|
||||
scores are cached, so re-gating costs zero new LLM calls. \`dream retriage\`
|
||||
re-scores the corpus and reconciles the queued synthesis backlog.
|
||||
|
||||
Options:
|
||||
--dry-run Preview all fixes without writing. Note: synthesize
|
||||
runs the cheap Haiku significance filter (caches
|
||||
verdicts), but skips the Sonnet synthesis pass.
|
||||
runs the cheap scored triage pass (caches verdicts),
|
||||
but skips the synthesis subagents.
|
||||
"--dry-run" does NOT mean "zero LLM calls."
|
||||
--json Emit the CycleReport as JSON (agent-readable)
|
||||
--phase <name> Run a single phase: ${ALL_PHASES.join(' | ')}
|
||||
@@ -569,6 +578,33 @@ async function runDrain(
|
||||
}
|
||||
|
||||
export async function runDream(engine: BrainEngine | null, args: string[]): Promise<CycleReport | void> {
|
||||
// ─── `dream retriage` subverb (#4152) — dispatched BEFORE parseArgs so its
|
||||
// flag set never collides with the cycle flags. `dream --help` never reaches
|
||||
// here (args[0] is '--help'); `dream retriage --help` prints subcommand help
|
||||
// inside runDreamRetriage without touching the engine (same IRON RULE).
|
||||
if (args[0] === 'retriage') {
|
||||
const { runDreamRetriage } = await import('./dream-retriage.ts');
|
||||
await runDreamRetriage(engine, args.slice(1));
|
||||
return;
|
||||
}
|
||||
// Fail-loud guard (structured-review r3 P1): the CLI flag registry unions
|
||||
// retriage's flags into `dream`, so the pre-dispatch validator accepts
|
||||
// `gbrain dream --reconcile-queue` — but without the `retriage` positional,
|
||||
// parseArgs would ignore the flag and silently run the full (paid, writing)
|
||||
// maintenance cycle instead of the reconciliation the user asked for.
|
||||
{
|
||||
const RETRIAGE_ONLY_FLAGS = ['--reconcile-queue', '--cancel-unmatched', '--audit-rejects'];
|
||||
const stray = args.find(a => RETRIAGE_ONLY_FLAGS.includes(a));
|
||||
if (stray) {
|
||||
console.error(
|
||||
`gbrain dream: ${stray} belongs to the 'retriage' subcommand — ` +
|
||||
`did you mean: gbrain dream retriage ${args.join(' ')}`,
|
||||
);
|
||||
setCliExitVerdict(2);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
const opts = parseArgs(args);
|
||||
|
||||
// ─── IRON RULE: --help short-circuits BEFORE any engine-bearing work ─
|
||||
|
||||
+23
-2
@@ -32,6 +32,7 @@ import {
|
||||
DEFAULT_ALIASES,
|
||||
TIER_DEFAULTS,
|
||||
resolveModel,
|
||||
resolveAlias,
|
||||
type ModelTier,
|
||||
} from '../core/model-config.ts';
|
||||
import { maybeAttachVersionSuffixHint } from '../core/ai/base-url-probe.ts';
|
||||
@@ -45,11 +46,23 @@ interface PerTaskModelRoute {
|
||||
description: string;
|
||||
deprecatedConfigKey?: string;
|
||||
envVar?: string;
|
||||
/**
|
||||
* #4152 (2A): an explicit pre-read key that wins over the whole
|
||||
* resolveModel chain when set — mirrors loadSynthConfig's triage-model
|
||||
* resolution so the dashboard reports the ACTUAL spending route.
|
||||
*/
|
||||
overrideKey?: string;
|
||||
}
|
||||
|
||||
const PER_TASK_KEYS: PerTaskModelRoute[] = [
|
||||
{ key: 'models.dream.synthesize', tier: 'reasoning', description: 'Dream synthesis (conversation → brain pages)' },
|
||||
{ key: 'models.dream.synthesize_verdict', tier: 'utility', description: 'Dream synthesis verdict (Haiku judge)' },
|
||||
{
|
||||
key: 'models.dream.synthesize_verdict',
|
||||
tier: 'utility',
|
||||
description: 'Dream triage judge (scored gate; models.dream.triage preferred)',
|
||||
deprecatedConfigKey: 'dream.synthesize.verdict_model',
|
||||
overrideKey: 'models.dream.triage',
|
||||
},
|
||||
{ key: 'models.dream.patterns', tier: 'reasoning', description: 'Pattern discovery (cross-take themes)' },
|
||||
{ key: 'models.drift', tier: 'reasoning', description: 'Drift LLM judge (v0.29 scaffold)' },
|
||||
{ key: 'models.auto_think', tier: 'deep', description: 'Auto-think question answering' },
|
||||
@@ -128,7 +141,15 @@ async function buildReport(engine: BrainEngine): Promise<ModelsReport> {
|
||||
|
||||
const per_task: ModelsReport['per_task'] = [];
|
||||
for (const route of PER_TASK_KEYS) {
|
||||
const { key, tier, description, deprecatedConfigKey, envVar } = route;
|
||||
const { key, tier, description, deprecatedConfigKey, envVar, overrideKey } = route;
|
||||
// Explicit pre-read override (loadSynthConfig 2A parity): when set, it IS
|
||||
// the effective spending route and must be reported as such.
|
||||
const overrideValue = overrideKey ? await engine.getConfig(overrideKey) : null;
|
||||
if (overrideKey && overrideValue?.trim()) {
|
||||
const resolved = await resolveAlias(engine, overrideValue.trim());
|
||||
per_task.push({ key, tier, resolved, source: `config: ${overrideKey}`, description });
|
||||
continue;
|
||||
}
|
||||
const resolved = await resolveModel(engine, {
|
||||
configKey: key,
|
||||
deprecatedConfigKey,
|
||||
|
||||
@@ -32,7 +32,7 @@ export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
|
||||
'check-backlinks': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dir', '--dry-run', '--explain', '--follow', '--help', '--include-frontmatter', '--json', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--source', '--stale', '--timeout', '--type'],
|
||||
'check-resolvable': ['--brain', '--dry-run', '--fix', '--help', '--json', '--skills-dir', '--source', '--strict', '--verbose'],
|
||||
'check-update': ['--all', '--brain', '--check', '--dim', '--ff-only', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
|
||||
'claw-test': ['--ab', '--agent', '--all', '--auto-update', '--brain', '--break-lock', '--build-index', '--by-mention', '--compile', '--days', '--dir', '--exclusive', '--force-retry', '--force-schema', '--from-meetings', '--help', '--history', '--http', '--json', '--keep-tempdir', '--lang', '--list-agents', '--live', '--local', '--locks', '--markdown', '--max-age', '--message', '--multimodal', '--no-embed', '--no-embedding', '--no-extract', '--output-format', '--path', '--pglite', '--phase', '--priority', '--progress-json', '--refresh-unqualified', '--remediate', '--rollback', '--run-id', '--scenario', '--skip-verify', '--source', '--stale', '--surface', '--transcripts', '--undo-wave', '--use-captured-snapshot', '--version', '--with-calibration', '--yes'],
|
||||
'claw-test': ['--ab', '--agent', '--all', '--auto-update', '--brain', '--break-lock', '--build-index', '--by-mention', '--compile', '--days', '--dir', '--exclusive', '--force', '--force-retry', '--force-schema', '--from-meetings', '--help', '--history', '--http', '--json', '--keep-tempdir', '--lang', '--list-agents', '--live', '--local', '--locks', '--markdown', '--max-age', '--message', '--multimodal', '--no-embed', '--no-embedding', '--no-extract', '--output-format', '--path', '--pglite', '--phase', '--priority', '--progress-json', '--refresh-unqualified', '--remediate', '--rollback', '--run-id', '--scenario', '--skip-verify', '--source', '--stale', '--surface', '--transcripts', '--undo-wave', '--use-captured-snapshot', '--version', '--with-calibration', '--yes'],
|
||||
'code-callees': ['--aliases', '--all', '--all-sources', '--brain', '--chunker-debug', '--clone-dir', '--confirm-destructive', '--federated', '--force', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--no-federated', '--no-json', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--url', '--url-managed', '--yes'],
|
||||
'code-callers': ['--aliases', '--all', '--all-sources', '--brain', '--chunker-debug', '--clone-dir', '--confirm-destructive', '--federated', '--force', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--no-federated', '--no-json', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--url', '--url-managed', '--yes'],
|
||||
'code-def': ['--aliases', '--all', '--brain', '--chunker-debug', '--help', '--include-null-signature', '--json', '--lang', '--limit', '--no-extract', '--no-json', '--pattern', '--pending', '--pretty', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--yes'],
|
||||
@@ -41,7 +41,7 @@ export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
|
||||
'connect': ['--agent', '--bearer-token-env-var', '--bind', '--brain', '--client-id', '--client-secret', '--force', '--grant-types', '--help', '--http', '--install', '--json', '--name', '--oauth', '--public-url', '--register', '--scope', '--scopes', '--show-token', '--source', '--timeout-ms', '--token', '--token-endpoint-auth-method', '--url', '--version', '--yes'],
|
||||
'conversation-parser': ['--aliases', '--all', '--brain', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin'],
|
||||
'doctor': ['--ab', '--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--allow-shell-jobs', '--allow-unverified-remote', '--auto', '--auto-fix', '--auto-update', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--build-index', '--by-mention', '--by-type', '--cached', '--check', '--column', '--compile', '--concurrency', '--confidence', '--confirm', '--content-audit', '--count', '--days', '--delete-brain', '--detach', '--detail', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--exclude-standard', '--exclusive', '--explain', '--fast', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--get', '--git-dir', '--git-path', '--grant-types', '--harness', '--health-interval', '--help', '--history', '--home', '--http', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--include-pseudo', '--index-audit', '--init', '--input', '--is-inside-work-tree', '--job-isolation', '--jq', '--json', '--lang', '--limit', '--local', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-crashes', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--nice', '--no', '--no-cron', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-mutate', '--no-verify', '--oauth-client-secret', '--older-than', '--once', '--others', '--overwrite', '--parallel', '--params', '--pat-file', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--priority', '--probe-pglite', '--progress-interval', '--progress-json', '--project', '--push-only', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--refresh', '--refresh-unqualified', '--regenerate', '--remediate', '--remediation-plan', '--remove', '--repo', '--reset', '--resolve', '--restore-only', '--resume', '--review-lower', '--rollback', '--scope', '--scopes', '--set', '--short', '--show-current', '--show-toplevel', '--since', '--skills-dir', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--skip-verify', '--slugs', '--source', '--source-id', '--stale', '--stats', '--status', '--strategy', '--strict', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--timeout', '--to', '--token', '--token-ttl', '--top-k', '--type', '--undo-wave', '--unsafe-bypass-dream-guard', '--unset-all', '--untracked-files', '--url', '--use-captured-snapshot', '--verbose', '--verify', '--version', '--window', '--with-calibration', '--workers', '--yes'],
|
||||
'dream': ['--against', '--aliases', '--all', '--allow-regression', '--anchor', '--asof', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--by-type', '--by-type-floor', '--code', '--committed-baseline', '--compare', '--compile', '--concurrent', '--ctx-size', '--cycles', '--date', '--detail', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--expansion', '--explain', '--fast', '--federated', '--fix', '--fixtures', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--format', '--from', '--from-db', '--from-pages', '--gold', '--harness', '--help', '--http', '--include-holdout', '--include-null-signature', '--input', '--install', '--json', '--judge-model', '--justification', '--keyword-only', '--lang', '--limit', '--llm', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-tokens', '--max-usd', '--mcp-only', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--name-only', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-llm', '--no-mutate', '--no-trajectory', '--once', '--out', '--output', '--output-dir', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--phase', '--priority', '--progress-interval', '--progress-json', '--pull', '--quiet', '--receipt-dir', '--remediate', '--repo', '--reranking', '--reset', '--resolve', '--restore-only', '--resume-from', '--retrieval-only', '--rounds', '--rubric-version', '--save', '--seed', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--suite', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--take', '--task', '--thin', '--threshold', '--timeout', '--to', '--token-ttl', '--top-k', '--undo', '--unsafe-bypass-dream-guard', '--update-baseline', '--verify', '--version', '--window', '--yes'],
|
||||
'dream': ['--against', '--aliases', '--all', '--allow-regression', '--anchor', '--asof', '--audit-rejects', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--by-type', '--by-type-floor', '--cancel-unmatched', '--code', '--committed-baseline', '--compare', '--compile', '--concurrent', '--ctx-size', '--cycles', '--date', '--detail', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--expansion', '--explain', '--fast', '--federated', '--fix', '--fixtures', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--format', '--from', '--from-db', '--from-pages', '--gold', '--harness', '--help', '--http', '--include-holdout', '--include-null-signature', '--input', '--install', '--json', '--judge-model', '--justification', '--keyword-only', '--lang', '--limit', '--llm', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-tokens', '--max-usd', '--mcp-only', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--name-only', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-llm', '--no-mutate', '--no-trajectory', '--once', '--out', '--output', '--output-dir', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--phase', '--priority', '--progress-interval', '--progress-json', '--pull', '--quiet', '--receipt-dir', '--reconcile-queue', '--remediate', '--repo', '--reranking', '--reset', '--resolve', '--restore-only', '--resume-from', '--retrieval-only', '--rounds', '--rubric-version', '--save', '--seed', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--suite', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--take', '--task', '--thin', '--threshold', '--timeout', '--to', '--token-ttl', '--top-k', '--undo', '--unsafe-bypass-dream-guard', '--update-baseline', '--verify', '--version', '--window', '--yes'],
|
||||
'edges-backfill': ['--aliases', '--all', '--all-sources', '--brain', '--concurrency', '--federated', '--help', '--include-null-signature', '--json', '--max-age', '--max-chunks', '--max-cost-usd', '--no-extract', '--no-federated', '--older-than', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--workers'],
|
||||
'embed': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--catch-up', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--serial', '--slugs', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--token-ttl', '--version'],
|
||||
'enrich': ['--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd-per-day', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--content', '--date', '--days', '--detail', '--dry-run', '--embedding-dimensions', '--embedding-model', '--entities', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--judge-model', '--kind', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-usd', '--min-context', '--mode', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--offset', '--older-than', '--order', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--reenrich-after', '--remediate', '--reset', '--resolve', '--restore-only', '--resume', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--source-id', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--thin-threshold', '--timeout', '--token-ttl', '--trusted-extraction', '--types', '--url', '--url-managed', '--version', '--with-db', '--workers', '--yes'],
|
||||
|
||||
@@ -1048,6 +1048,9 @@ export const KNOWN_CONFIG_KEYS: readonly string[] = [
|
||||
'cycle.extract_atoms.budget_usd',
|
||||
'models.dream.patterns',
|
||||
'models.dream.synthesize_verdict',
|
||||
// #4152: preferred triage-model key (explicit pre-read in loadSynthConfig;
|
||||
// wins over models.dream.synthesize_verdict + dream.synthesize.verdict_model).
|
||||
'models.dream.triage',
|
||||
'models.drift',
|
||||
'models.auto_think',
|
||||
'models.think',
|
||||
@@ -1079,6 +1082,19 @@ export const KNOWN_CONFIG_KEYS: readonly string[] = [
|
||||
'dream.synthesize.output_root',
|
||||
'dream.synthesize.subagent_timeout_ms',
|
||||
'dream.synthesize.subagent_wait_timeout_ms',
|
||||
// #4152 two-stage cascade: subagent turn budget (default 16) + opt-in
|
||||
// per-source daily submission cap (default 0 = disabled; 200 recommended
|
||||
// for busy deployments).
|
||||
'dream.synthesize.max_turns',
|
||||
'dream.synthesize.max_submissions_per_source_per_day',
|
||||
// #4152 triage knobs. The triage model's preferred key is
|
||||
// `models.dream.triage` (models.* prefix, registered via the models.dream.*
|
||||
// family); these tune the gate + sampling + pass budget.
|
||||
'dream.triage.threshold',
|
||||
'dream.triage.max_chars',
|
||||
'dream.triage.max_tokens',
|
||||
'dream.triage.max_ms',
|
||||
'dream.triage.concurrency',
|
||||
'dream.patterns.lookback_days',
|
||||
'dream.patterns.min_evidence',
|
||||
// #2782-family: patterns-phase subagent timeouts (mirror of the
|
||||
|
||||
+918
-189
File diff suppressed because it is too large
Load Diff
+24
-1
@@ -444,17 +444,40 @@ export interface SynthesisEvidenceInput {
|
||||
citation_index: number;
|
||||
}
|
||||
|
||||
/** Dream-cycle Haiku verdict on whether a transcript is worth processing. */
|
||||
/** One candidate segment extracted by triage: a verbatim quote plus why it matters. */
|
||||
export interface TriageSegment {
|
||||
quote: string;
|
||||
note?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Dream-cycle triage verdict on a transcript (#4152 two-stage cascade).
|
||||
* Triage-v1 fields (`score` .. `triage_version`) are null/[] on legacy rows
|
||||
* written by the boolean-era judge — callers treat those rows as cache misses.
|
||||
*/
|
||||
export interface DreamVerdict {
|
||||
worth_processing: boolean;
|
||||
reasons: string[];
|
||||
judged_at: string;
|
||||
/** Ordinal salience score in [0,1]; comparable only within (model, triage_version). */
|
||||
score: number | null;
|
||||
content_type: string | null;
|
||||
segments: TriageSegment[];
|
||||
entities: string[];
|
||||
model: string | null;
|
||||
triage_version: number | null;
|
||||
}
|
||||
|
||||
/** Input shape for putDreamVerdict — judged_at defaults to now() server-side. */
|
||||
export interface DreamVerdictInput {
|
||||
worth_processing: boolean;
|
||||
reasons: string[];
|
||||
score: number;
|
||||
content_type: string | null;
|
||||
segments: TriageSegment[];
|
||||
entities: string[];
|
||||
model: string;
|
||||
triage_version: number;
|
||||
}
|
||||
|
||||
// ============================================================
|
||||
|
||||
@@ -5794,6 +5794,41 @@ export const MIGRATIONS: Migration[] = [
|
||||
);
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 129,
|
||||
name: 'dream_verdicts_triage_v1_columns',
|
||||
// #4152 two-stage cascade: widens the boolean-era verdict cache into a
|
||||
// scored triage record — ordinal salience score in [0,1], content type,
|
||||
// candidate segments (verbatim quotes), entity candidates, plus the
|
||||
// judging model and triage prompt version that make a cached verdict
|
||||
// auditable and version-invalidatable. Legacy rows keep score NULL and
|
||||
// are treated as cache misses by runTriagePass (re-judged once, cheap).
|
||||
// No backfill by design; no index (the table is PK-probed only).
|
||||
//
|
||||
// dream_verdicts is migration-created on PGLite (v30, absent from
|
||||
// PGLITE_SCHEMA_SQL), so these columns take the COLUMN_EXEMPTIONS route
|
||||
// in test/schema-bootstrap-coverage.test.ts rather than bootstrap
|
||||
// probes — there is no schema-blob forward reference to trip on, and
|
||||
// every reader treats NULL as legacy-miss. Keep src/schema.sql (and the
|
||||
// regenerated schema-embedded.ts) in sync for fresh Postgres installs.
|
||||
//
|
||||
// Rollback note: the stored worth_processing boolean derives from the
|
||||
// FIXED 0.5 constant while the new runtime gate uses the configurable
|
||||
// dream.triage.threshold. A binary rollback to pre-#4152 code (which
|
||||
// trusts the boolean as permanent) after retuning the threshold gates
|
||||
// differently until content hashes change — sweep with
|
||||
// `gbrain dream retriage --force` (or clear scored rows) after such a
|
||||
// rollback.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
ALTER TABLE dream_verdicts ADD COLUMN IF NOT EXISTS score DOUBLE PRECISION;
|
||||
ALTER TABLE dream_verdicts ADD COLUMN IF NOT EXISTS content_type TEXT;
|
||||
ALTER TABLE dream_verdicts ADD COLUMN IF NOT EXISTS segments JSONB;
|
||||
ALTER TABLE dream_verdicts ADD COLUMN IF NOT EXISTS entities JSONB;
|
||||
ALTER TABLE dream_verdicts ADD COLUMN IF NOT EXISTS model TEXT;
|
||||
ALTER TABLE dream_verdicts ADD COLUMN IF NOT EXISTS triage_version INT;
|
||||
`,
|
||||
},
|
||||
];
|
||||
|
||||
export const LATEST_VERSION = MIGRATIONS.length > 0
|
||||
|
||||
@@ -4620,14 +4620,21 @@ export class PGLiteEngine implements BrainEngine {
|
||||
return result.rows as FileRow[];
|
||||
}
|
||||
|
||||
// Dream-cycle significance verdict cache (v0.23).
|
||||
// Dream-cycle triage verdict cache (v0.23 boolean era; widened by #4152 triage-v1).
|
||||
async getDreamVerdict(filePath: string, contentHash: string): Promise<DreamVerdict | null> {
|
||||
const result = await this.db.query<{
|
||||
worth_processing: boolean;
|
||||
reasons: string[] | null;
|
||||
judged_at: Date | string;
|
||||
score: number | null;
|
||||
content_type: string | null;
|
||||
segments: Array<{ quote: string; note?: string }> | null;
|
||||
entities: string[] | null;
|
||||
model: string | null;
|
||||
triage_version: number | null;
|
||||
}>(
|
||||
`SELECT worth_processing, reasons, judged_at
|
||||
`SELECT worth_processing, reasons, judged_at,
|
||||
score, content_type, segments, entities, model, triage_version
|
||||
FROM dream_verdicts
|
||||
WHERE file_path = $1 AND content_hash = $2`,
|
||||
[filePath, contentHash]
|
||||
@@ -4638,18 +4645,35 @@ export class PGLiteEngine implements BrainEngine {
|
||||
worth_processing: r.worth_processing,
|
||||
reasons: r.reasons ?? [],
|
||||
judged_at: r.judged_at instanceof Date ? r.judged_at.toISOString() : String(r.judged_at),
|
||||
score: r.score ?? null,
|
||||
content_type: r.content_type ?? null,
|
||||
segments: r.segments ?? [],
|
||||
entities: r.entities ?? [],
|
||||
model: r.model ?? null,
|
||||
triage_version: r.triage_version ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
async putDreamVerdict(filePath: string, contentHash: string, verdict: DreamVerdictInput): Promise<void> {
|
||||
// $N::jsonb + JSON.stringify is legal ONLY on PGLite (its db.query parses
|
||||
// text→jsonb natively); the postgres.js twin must use sql.json().
|
||||
await this.db.query(
|
||||
`INSERT INTO dream_verdicts (file_path, content_hash, worth_processing, reasons)
|
||||
VALUES ($1, $2, $3, $4::jsonb)
|
||||
`INSERT INTO dream_verdicts (file_path, content_hash, worth_processing, reasons,
|
||||
score, content_type, segments, entities, model, triage_version)
|
||||
VALUES ($1, $2, $3, $4::jsonb, $5, $6, $7::jsonb, $8::jsonb, $9, $10)
|
||||
ON CONFLICT (file_path, content_hash) DO UPDATE SET
|
||||
worth_processing = EXCLUDED.worth_processing,
|
||||
reasons = EXCLUDED.reasons,
|
||||
score = EXCLUDED.score,
|
||||
content_type = EXCLUDED.content_type,
|
||||
segments = EXCLUDED.segments,
|
||||
entities = EXCLUDED.entities,
|
||||
model = EXCLUDED.model,
|
||||
triage_version = EXCLUDED.triage_version,
|
||||
judged_at = now()`,
|
||||
[filePath, contentHash, verdict.worth_processing, JSON.stringify(verdict.reasons)]
|
||||
[filePath, contentHash, verdict.worth_processing, JSON.stringify(verdict.reasons),
|
||||
verdict.score, verdict.content_type, JSON.stringify(verdict.segments),
|
||||
JSON.stringify(verdict.entities), verdict.model, verdict.triage_version]
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -4524,15 +4524,22 @@ export class PostgresEngine implements BrainEngine {
|
||||
return rows as FileRow[];
|
||||
}
|
||||
|
||||
// Dream-cycle significance verdict cache (v0.23).
|
||||
// Dream-cycle triage verdict cache (v0.23 boolean era; widened by #4152 triage-v1).
|
||||
async getDreamVerdict(filePath: string, contentHash: string): Promise<DreamVerdict | null> {
|
||||
const sql = this.sql;
|
||||
const rows = await sql<Array<{
|
||||
worth_processing: boolean;
|
||||
reasons: string[] | null;
|
||||
judged_at: Date;
|
||||
score: number | null;
|
||||
content_type: string | null;
|
||||
segments: Array<{ quote: string; note?: string }> | null;
|
||||
entities: string[] | null;
|
||||
model: string | null;
|
||||
triage_version: number | null;
|
||||
}>>`
|
||||
SELECT worth_processing, reasons, judged_at
|
||||
SELECT worth_processing, reasons, judged_at,
|
||||
score, content_type, segments, entities, model, triage_version
|
||||
FROM dream_verdicts
|
||||
WHERE file_path = ${filePath} AND content_hash = ${contentHash}
|
||||
`;
|
||||
@@ -4542,17 +4549,32 @@ export class PostgresEngine implements BrainEngine {
|
||||
worth_processing: r.worth_processing,
|
||||
reasons: r.reasons ?? [],
|
||||
judged_at: r.judged_at instanceof Date ? r.judged_at.toISOString() : String(r.judged_at),
|
||||
score: r.score ?? null,
|
||||
content_type: r.content_type ?? null,
|
||||
segments: r.segments ?? [],
|
||||
entities: r.entities ?? [],
|
||||
model: r.model ?? null,
|
||||
triage_version: r.triage_version ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
async putDreamVerdict(filePath: string, contentHash: string, verdict: DreamVerdictInput): Promise<void> {
|
||||
const sql = this.sql;
|
||||
await sql`
|
||||
INSERT INTO dream_verdicts (file_path, content_hash, worth_processing, reasons)
|
||||
VALUES (${filePath}, ${contentHash}, ${verdict.worth_processing}, ${sql.json(verdict.reasons as Parameters<typeof sql.json>[0])})
|
||||
INSERT INTO dream_verdicts (file_path, content_hash, worth_processing, reasons,
|
||||
score, content_type, segments, entities, model, triage_version)
|
||||
VALUES (${filePath}, ${contentHash}, ${verdict.worth_processing}, ${sql.json(verdict.reasons as Parameters<typeof sql.json>[0])},
|
||||
${verdict.score}, ${verdict.content_type}, ${sql.json(verdict.segments as unknown as Parameters<typeof sql.json>[0])},
|
||||
${sql.json(verdict.entities as Parameters<typeof sql.json>[0])}, ${verdict.model}, ${verdict.triage_version})
|
||||
ON CONFLICT (file_path, content_hash) DO UPDATE SET
|
||||
worth_processing = EXCLUDED.worth_processing,
|
||||
reasons = EXCLUDED.reasons,
|
||||
score = EXCLUDED.score,
|
||||
content_type = EXCLUDED.content_type,
|
||||
segments = EXCLUDED.segments,
|
||||
entities = EXCLUDED.entities,
|
||||
model = EXCLUDED.model,
|
||||
triage_version = EXCLUDED.triage_version,
|
||||
judged_at = now()
|
||||
`;
|
||||
}
|
||||
|
||||
@@ -1118,6 +1118,15 @@ CREATE TABLE IF NOT EXISTS dream_verdicts (
|
||||
worth_processing BOOLEAN NOT NULL,
|
||||
reasons JSONB,
|
||||
judged_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
-- #4152 triage-v1 (migration v129): ordinal salience score in [0,1],
|
||||
-- content type, candidate segments/entities, judging model + prompt
|
||||
-- version. NULL on boolean-era rows — readers treat those as cache misses.
|
||||
score DOUBLE PRECISION,
|
||||
content_type TEXT,
|
||||
segments JSONB,
|
||||
entities JSONB,
|
||||
model TEXT,
|
||||
triage_version INT,
|
||||
PRIMARY KEY (file_path, content_hash)
|
||||
);
|
||||
|
||||
|
||||
@@ -1114,6 +1114,15 @@ CREATE TABLE IF NOT EXISTS dream_verdicts (
|
||||
worth_processing BOOLEAN NOT NULL,
|
||||
reasons JSONB,
|
||||
judged_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
-- #4152 triage-v1 (migration v129): ordinal salience score in [0,1],
|
||||
-- content type, candidate segments/entities, judging model + prompt
|
||||
-- version. NULL on boolean-era rows — readers treat those as cache misses.
|
||||
score DOUBLE PRECISION,
|
||||
content_type TEXT,
|
||||
segments JSONB,
|
||||
entities JSONB,
|
||||
model TEXT,
|
||||
triage_version INT,
|
||||
PRIMARY KEY (file_path, content_hash)
|
||||
);
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# gbrain agent workspace — template
|
||||
|
||||
<!-- gbrain-template-stamp: 0.46.1.0 -->
|
||||
<!-- gbrain-template-stamp: 0.46.2.0 -->
|
||||
|
||||
This repository is the **"Use this template"** distribution artifact for a
|
||||
[gbrain](https://github.com/garrytan/gbrain) personal-agent workspace — the same
|
||||
|
||||
@@ -27,6 +27,8 @@ const HELP_WITHOUT_BRAIN = [
|
||||
'extract-conversation-facts',
|
||||
'transcripts',
|
||||
'jobs',
|
||||
// #4152: dream answers --help (and the retriage subverb help) engine-free.
|
||||
'dream',
|
||||
];
|
||||
|
||||
/**
|
||||
|
||||
@@ -51,3 +51,89 @@ describe('loadSynthConfig honors a configured 0', () => {
|
||||
expect(cfg.cooldownHours).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
// ── #4152 triage knobs: resolution chain, clamps, floors ──────────────────
|
||||
|
||||
describe('loadSynthConfig — triage model resolution chain (#4152 2A)', () => {
|
||||
test('models.dream.triage wins over models.dream.synthesize_verdict and the deprecated key', async () => {
|
||||
const cfg = await __testing.loadSynthConfig(stubEngine({
|
||||
'models.dream.triage': 'anthropic:claude-sonnet-4-6',
|
||||
'models.dream.synthesize_verdict': 'anthropic:claude-haiku-4-5-20251001',
|
||||
'dream.synthesize.verdict_model': 'anthropic:claude-3-5-haiku-20241022',
|
||||
}));
|
||||
expect(cfg.triage.model).toBe('anthropic:claude-sonnet-4-6');
|
||||
});
|
||||
|
||||
test('models.dream.synthesize_verdict wins when models.dream.triage is unset', async () => {
|
||||
const cfg = await __testing.loadSynthConfig(stubEngine({
|
||||
'models.dream.synthesize_verdict': 'anthropic:claude-haiku-4-5-20251001',
|
||||
'dream.synthesize.verdict_model': 'anthropic:claude-3-5-haiku-20241022',
|
||||
}));
|
||||
expect(cfg.triage.model).toBe('anthropic:claude-haiku-4-5-20251001');
|
||||
});
|
||||
|
||||
test('deprecated dream.synthesize.verdict_model still resolves when both new keys are unset', async () => {
|
||||
const cfg = await __testing.loadSynthConfig(stubEngine({
|
||||
'dream.synthesize.verdict_model': 'anthropic:claude-3-5-haiku-20241022',
|
||||
}));
|
||||
expect(cfg.triage.model).toBe('anthropic:claude-3-5-haiku-20241022');
|
||||
});
|
||||
|
||||
test('all keys unset → tier utility default', async () => {
|
||||
const cfg = await __testing.loadSynthConfig(stubEngine({}));
|
||||
expect(cfg.triage.model).toBe('anthropic:claude-haiku-4-5-20251001');
|
||||
});
|
||||
});
|
||||
|
||||
describe('loadSynthConfig — triage knob clamps and floors (#4152)', () => {
|
||||
test('threshold outside [0,1] clamps with a stderr warning; 0 is honored as-is', async () => {
|
||||
const chunks: string[] = [];
|
||||
const orig = process.stderr.write.bind(process.stderr);
|
||||
(process.stderr as unknown as { write: (c: unknown) => boolean }).write = (c: unknown) => {
|
||||
chunks.push(String(c));
|
||||
return true;
|
||||
};
|
||||
try {
|
||||
const over = await __testing.loadSynthConfig(stubEngine({ 'dream.triage.threshold': '1.5' }));
|
||||
expect(over.triage.threshold).toBe(1);
|
||||
const under = await __testing.loadSynthConfig(stubEngine({ 'dream.triage.threshold': '-0.5' }));
|
||||
expect(under.triage.threshold).toBe(0);
|
||||
const zero = await __testing.loadSynthConfig(stubEngine({ 'dream.triage.threshold': '0' }));
|
||||
expect(zero.triage.threshold).toBe(0);
|
||||
} finally {
|
||||
(process.stderr as unknown as { write: typeof orig }).write = orig;
|
||||
}
|
||||
expect(chunks.join('')).toMatch(/dream\.triage\.threshold .* outside \[0,1\]/);
|
||||
});
|
||||
|
||||
test('max_turns floors at 1 (a configured 0 clamps up); default is 16', async () => {
|
||||
const zero = await __testing.loadSynthConfig(stubEngine({ 'dream.synthesize.max_turns': '0' }));
|
||||
expect(zero.maxTurns).toBe(1);
|
||||
const dflt = await __testing.loadSynthConfig(stubEngine({}));
|
||||
expect(dflt.maxTurns).toBe(16);
|
||||
const restored = await __testing.loadSynthConfig(stubEngine({ 'dream.synthesize.max_turns': '30' }));
|
||||
expect(restored.maxTurns).toBe(30);
|
||||
});
|
||||
|
||||
test('triage floors: max_chars >= 1000, max_tokens >= 256, concurrency clamped [1,16], max_ms 0 = unlimited', async () => {
|
||||
const cfg = await __testing.loadSynthConfig(stubEngine({
|
||||
'dream.triage.max_chars': '10',
|
||||
'dream.triage.max_tokens': '5',
|
||||
'dream.triage.concurrency': '99',
|
||||
'dream.triage.max_ms': '0',
|
||||
}));
|
||||
expect(cfg.triage.maxChars).toBe(1000);
|
||||
expect(cfg.triage.maxTokens).toBe(256);
|
||||
expect(cfg.triage.concurrency).toBe(16);
|
||||
expect(cfg.triage.maxMs).toBe(0);
|
||||
const low = await __testing.loadSynthConfig(stubEngine({ 'dream.triage.concurrency': '0' }));
|
||||
expect(low.triage.concurrency).toBe(1);
|
||||
});
|
||||
|
||||
test('daily cap defaults to 0 (disabled, D2D); explicit positive value engages', async () => {
|
||||
const dflt = await __testing.loadSynthConfig(stubEngine({}));
|
||||
expect(dflt.maxSubmissionsPerSourcePerDay).toBe(0);
|
||||
const set = await __testing.loadSynthConfig(stubEngine({ 'dream.synthesize.max_submissions_per_source_per_day': '200' }));
|
||||
expect(set.maxSubmissionsPerSourcePerDay).toBe(200);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,329 @@
|
||||
/**
|
||||
* #4152 — opt-in per-source daily synthesis submission cap
|
||||
* (dream.synthesize.max_submissions_per_source_per_day, default 0 = disabled).
|
||||
*
|
||||
* PGLite, no API key required: verdicts are pre-seeded so the triage pass is
|
||||
* all cache hits, and submitted subagent jobs are auto-cancelled so the
|
||||
* phase's inline wait returns immediately.
|
||||
*
|
||||
* Run: bun test test/cycle-synthesize-daily-cap.test.ts
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { resetPgliteState } from './helpers/reset-pglite.ts';
|
||||
import { withEnv } from './helpers/with-env.ts';
|
||||
import { runPhaseSynthesize, TRIAGE_VERSION } from '../src/core/cycle/synthesize.ts';
|
||||
import { TIER_DEFAULTS } from '../src/core/model-config.ts';
|
||||
|
||||
// Canonical shared-engine block (check-test-isolation R3/R4).
|
||||
let engine: PGLiteEngine;
|
||||
let schemaVersion: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ engine: 'pglite' } as never);
|
||||
await engine.initSchema();
|
||||
// resetPgliteState truncates `config`, wiping the `version` row that
|
||||
// MinionQueue.ensureSchema checks. Capture it so beforeEach can restore.
|
||||
schemaVersion = (await engine.getConfig('version')) ?? '7';
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetPgliteState(engine);
|
||||
await engine.setConfig('version', schemaVersion);
|
||||
});
|
||||
|
||||
interface Rig {
|
||||
engine: PGLiteEngine;
|
||||
brainDir: string;
|
||||
corpusDir: string;
|
||||
cleanup: () => Promise<void>;
|
||||
}
|
||||
|
||||
/** Per-test dirs + dream config on the SHARED engine (reset by beforeEach). */
|
||||
async function setupRig(): Promise<Rig> {
|
||||
const brainDir = mkdtempSync(join(tmpdir(), 'gbrain-cap-brain-'));
|
||||
const corpusDir = mkdtempSync(join(tmpdir(), 'gbrain-cap-corpus-'));
|
||||
await engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await engine.setConfig('dream.synthesize.session_corpus_dir', corpusDir);
|
||||
return {
|
||||
engine,
|
||||
brainDir,
|
||||
corpusDir,
|
||||
cleanup: async () => {
|
||||
try { rmSync(brainDir, { recursive: true, force: true }); } catch { /* */ }
|
||||
try { rmSync(corpusDir, { recursive: true, force: true }); } catch { /* */ }
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function withSubagentAutoCancel<T>(
|
||||
engine: PGLiteEngine,
|
||||
body: () => Promise<T>,
|
||||
opts: { excludeQueue?: string } = {},
|
||||
): Promise<T> {
|
||||
let stopped = false;
|
||||
const loop = (async () => {
|
||||
while (!stopped) {
|
||||
await new Promise(r => setTimeout(r, 50));
|
||||
try {
|
||||
// excludeQueue: deliberately-seeded fixture rows must be handled by
|
||||
// the code under test, not this poller (a poller cancel changes the
|
||||
// row's coalescibility and races the assertion).
|
||||
await engine.executeRaw(
|
||||
`UPDATE minion_jobs
|
||||
SET status = 'cancelled', finished_at = now()
|
||||
WHERE name = 'subagent' AND status IN ('waiting', 'active')
|
||||
AND ($1::text IS NULL OR queue <> $1)`,
|
||||
[opts.excludeQueue ?? null],
|
||||
);
|
||||
} catch { /* race against shutdown is fine */ }
|
||||
}
|
||||
})();
|
||||
try {
|
||||
return await body();
|
||||
} finally {
|
||||
stopped = true;
|
||||
await loop;
|
||||
}
|
||||
}
|
||||
|
||||
/** Write a small transcript + seed a passing triage-v1 verdict for it. */
|
||||
async function seedPassingFile(rig: Rig, name: string): Promise<string> {
|
||||
const content = `conversation in ${name}\n`.repeat(200);
|
||||
const filePath = join(rig.corpusDir, name);
|
||||
writeFileSync(filePath, content);
|
||||
const hash = createHash('sha256').update(content, 'utf8').digest('hex');
|
||||
await rig.engine.putDreamVerdict(filePath, hash, {
|
||||
worth_processing: true,
|
||||
reasons: ['seed'],
|
||||
score: 0.9,
|
||||
content_type: null,
|
||||
segments: [],
|
||||
entities: [],
|
||||
model: TIER_DEFAULTS.utility,
|
||||
triage_version: TRIAGE_VERSION,
|
||||
});
|
||||
return filePath;
|
||||
}
|
||||
|
||||
/** Seed a recent (or old) synth-v2 submission row for the cap counter. */
|
||||
async function seedSubmissionRow(rig: Rig, opts: { ageHours?: number; status?: string; sourceId?: string; tag: string }): Promise<void> {
|
||||
await rig.engine.executeRaw(
|
||||
`INSERT INTO minion_jobs (name, queue, status, data, idempotency_key, created_at)
|
||||
VALUES ('subagent', 'dream-inline-old-run', $1,
|
||||
jsonb_build_object('source_id', $2::text),
|
||||
$3,
|
||||
now() - ($4 || ' hours')::interval)`,
|
||||
[opts.status ?? 'waiting', opts.sourceId ?? 'default',
|
||||
`dream:synth-v2:default:filename:${opts.tag}:0123456789abcdef`,
|
||||
String(opts.ageHours ?? 1)],
|
||||
);
|
||||
}
|
||||
|
||||
interface CapDetails {
|
||||
children_submitted: number;
|
||||
skips: Array<{ filePath: string; reason: string }>;
|
||||
}
|
||||
|
||||
async function runPhase(rig: Rig, opts: { date?: string; excludeQueue?: string } = {}): Promise<CapDetails> {
|
||||
// No key + isolated GBRAIN_HOME: every seeded verdict is a cache hit, so no
|
||||
// judge call happens; the env isolation is belt-and-suspenders against a
|
||||
// dev machine whose config file carries a real key.
|
||||
const { excludeQueue, ...phaseOpts } = opts;
|
||||
const tmpHome = mkdtempSync(join(tmpdir(), 'gbrain-cap-isol-'));
|
||||
try {
|
||||
const result = await withEnv({ ANTHROPIC_API_KEY: undefined, GBRAIN_HOME: tmpHome }, () =>
|
||||
withSubagentAutoCancel(rig.engine, () =>
|
||||
runPhaseSynthesize(rig.engine, { brainDir: rig.brainDir, dryRun: false, ...phaseOpts }),
|
||||
{ excludeQueue }));
|
||||
expect(result.status).toBe('ok');
|
||||
return result.details as unknown as CapDetails;
|
||||
} finally {
|
||||
try { rmSync(tmpHome, { recursive: true, force: true }); } catch { /* */ }
|
||||
}
|
||||
}
|
||||
|
||||
describe('daily cap — default off (D2D)', () => {
|
||||
test('unset cap: every passing file submits, zero daily_cap skips', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await seedPassingFile(rig, '2026-08-01-a.txt');
|
||||
await seedPassingFile(rig, '2026-08-02-b.txt');
|
||||
const details = await runPhase(rig);
|
||||
expect(details.children_submitted).toBe(2);
|
||||
expect(details.skips.filter(s => s.reason.startsWith('daily_cap'))).toHaveLength(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 60_000);
|
||||
});
|
||||
|
||||
describe('daily cap — engaged', () => {
|
||||
test('cap=1 with two passing files: one submits, one skips whole with daily_cap_reached', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.max_submissions_per_source_per_day', '1');
|
||||
await seedPassingFile(rig, '2026-08-01-a.txt');
|
||||
await seedPassingFile(rig, '2026-08-02-b.txt');
|
||||
const details = await runPhase(rig);
|
||||
expect(details.children_submitted).toBe(1);
|
||||
const capSkips = details.skips.filter(s => s.reason.startsWith('daily_cap_reached'));
|
||||
expect(capSkips).toHaveLength(1);
|
||||
expect(capSkips[0].reason).toMatch(/daily_cap_reached: 1\/1/);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test('24h window: a >24h-old submission row does not eat the budget', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.max_submissions_per_source_per_day', '1');
|
||||
await seedSubmissionRow(rig, { ageHours: 30, tag: 'old.txt' });
|
||||
await seedPassingFile(rig, '2026-08-03-c.txt');
|
||||
const details = await runPhase(rig);
|
||||
expect(details.children_submitted).toBe(1);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test('recent row within the window exhausts cap=1 → file skipped, cooldown NOT stamped (CX8)', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.max_submissions_per_source_per_day', '1');
|
||||
await seedSubmissionRow(rig, { ageHours: 1, tag: 'recent.txt' });
|
||||
await seedPassingFile(rig, '2026-08-04-d.txt');
|
||||
const details = await runPhase(rig);
|
||||
expect(details.children_submitted).toBe(0);
|
||||
expect(details.skips[0].reason).toMatch(/daily_cap_reached/);
|
||||
// CX8: a run that submitted nothing must not start the cooldown window —
|
||||
// that would suppress the retry that picks the capped files up.
|
||||
const ts = await rig.engine.getConfig('dream.synthesize.last_completion_ts');
|
||||
expect(ts ?? null).toBeNull();
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test('cancelled rows are excluded from the count (retriage-cancelled jobs return budget)', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.max_submissions_per_source_per_day', '1');
|
||||
await seedSubmissionRow(rig, { ageHours: 1, status: 'cancelled', tag: 'cxl.txt' });
|
||||
await seedPassingFile(rig, '2026-08-05-e.txt');
|
||||
const details = await runPhase(rig);
|
||||
expect(details.children_submitted).toBe(1);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test('per-source scoping: another source’s recent rows do not count', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.max_submissions_per_source_per_day', '1');
|
||||
await seedSubmissionRow(rig, { ageHours: 1, sourceId: 'other-brain', tag: 'other.txt' });
|
||||
await seedPassingFile(rig, '2026-08-06-f.txt');
|
||||
const details = await runPhase(rig); // runs as source 'default'
|
||||
expect(details.children_submitted).toBe(1);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test('explicit-target bypass: --date runs ignore the cap (same rule as cooldown)', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.max_submissions_per_source_per_day', '1');
|
||||
await seedSubmissionRow(rig, { ageHours: 1, tag: 'recent.txt' }); // cap exhausted
|
||||
await seedPassingFile(rig, '2026-08-07-g.txt');
|
||||
const details = await runPhase(rig, { date: '2026-08-07' });
|
||||
expect(details.children_submitted).toBe(1);
|
||||
expect(details.skips.filter(s => s.reason.startsWith('daily_cap'))).toHaveLength(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test('SR-P2: an exhausted cap does NOT strand a file whose keys already exist (idempotent retry)', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.max_submissions_per_source_per_day', '1');
|
||||
// Cap fully consumed by an unrelated recent submission…
|
||||
await seedSubmissionRow(rig, { ageHours: 1, tag: 'consumed.txt' });
|
||||
// …but THIS file's job already exists from a prior run that died
|
||||
// mid-drain (stranded in a dead inline queue). Re-running must reach
|
||||
// queue.add so coalesce/self-heal recovers it — zero NEW spend.
|
||||
const filePath = await seedPassingFile(rig, '2026-08-09-retry.txt');
|
||||
const content = `conversation in 2026-08-09-retry.txt\n`.repeat(200);
|
||||
const hash16 = createHash('sha256').update(content, 'utf8').digest('hex').slice(0, 16);
|
||||
const key = `dream:synth-v2:default:filename:${encodeURIComponent('2026-08-09-retry.txt')}:${hash16}`;
|
||||
await rig.engine.executeRaw(
|
||||
`INSERT INTO minion_jobs (name, queue, status, data, idempotency_key)
|
||||
VALUES ('subagent', 'dream-inline-1700000000000-deadbeef', 'waiting', '{}'::jsonb, $1)`,
|
||||
[key],
|
||||
);
|
||||
const details = await runPhase(rig, { excludeQueue: 'dream-inline-1700000000000-deadbeef' });
|
||||
// The file was NOT daily-cap skipped: self-heal cancelled the stranded
|
||||
// row and re-added into the live run's queue.
|
||||
expect(details.skips.filter(s => s.reason.startsWith('daily_cap'))).toHaveLength(0);
|
||||
expect(details.children_submitted).toBe(1);
|
||||
expect(filePath).toContain('2026-08-09-retry.txt');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test('fail-open: a throwing count query warns to stderr and skips the cap for the run', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.max_submissions_per_source_per_day', '1');
|
||||
await seedPassingFile(rig, '2026-08-08-h.txt');
|
||||
// Monkey-patch executeRaw to throw ONLY for the cap-count query.
|
||||
// MUST be a `function` delegating to the PROTOTYPE method with dynamic
|
||||
// `this`: PGLiteEngine.transaction builds its tx engine via
|
||||
// Object.create(this), which inherits own-property patches — a patch
|
||||
// closed over the OUTER engine would route tx queries to the main
|
||||
// connection and deadlock PGLite.
|
||||
const protoExecuteRaw = PGLiteEngine.prototype.executeRaw;
|
||||
(rig.engine as unknown as { executeRaw: unknown }).executeRaw =
|
||||
async function (this: PGLiteEngine, sql: string, params?: unknown[], opts?: never) {
|
||||
if (sql.includes(`idempotency_key LIKE 'dream:synth-v2:%'`) && sql.includes('COUNT(*)')) {
|
||||
throw new Error('simulated pool reap');
|
||||
}
|
||||
return protoExecuteRaw.call(this, sql, params, opts);
|
||||
};
|
||||
const stderrChunks: string[] = [];
|
||||
const origWrite = process.stderr.write.bind(process.stderr);
|
||||
// Spy with passthrough — swallowing stderr here hides real hang
|
||||
// diagnostics when the phase misbehaves.
|
||||
(process.stderr as unknown as { write: (c: unknown) => boolean }).write = (chunk: unknown) => {
|
||||
stderrChunks.push(String(chunk));
|
||||
return origWrite(chunk as never);
|
||||
};
|
||||
try {
|
||||
const details = await runPhase(rig);
|
||||
expect(details.children_submitted).toBe(1); // cap skipped, phase proceeded
|
||||
} finally {
|
||||
(process.stderr as unknown as { write: typeof origWrite }).write = origWrite;
|
||||
// Remove the own-property patch so the SHARED engine's prototype
|
||||
// method is restored for subsequent tests in this file.
|
||||
delete (rig.engine as unknown as { executeRaw?: unknown }).executeRaw;
|
||||
}
|
||||
expect(stderrChunks.join('')).toMatch(/daily-cap count query failed.*skipping the cap/s);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 60_000);
|
||||
});
|
||||
@@ -9,7 +9,8 @@ import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { resetPgliteState } from './helpers/reset-pglite.ts';
|
||||
import { runPhaseSynthesize } from '../src/core/cycle/synthesize.ts';
|
||||
import { runPhaseSynthesize, TRIAGE_VERSION } from '../src/core/cycle/synthesize.ts';
|
||||
import { TIER_DEFAULTS } from '../src/core/model-config.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
let schemaVersion: string;
|
||||
@@ -37,9 +38,17 @@ async function seedWorthProcessingVerdict(
|
||||
content: string,
|
||||
): Promise<void> {
|
||||
const contentHash = createHash('sha256').update(content, 'utf8').digest('hex');
|
||||
// Triage-v1 cache validity requires score + matching (model, triage_version);
|
||||
// TIER_DEFAULTS.utility is what loadSynthConfig resolves in a bare test env.
|
||||
await engine.putDreamVerdict(filePath, contentHash, {
|
||||
worth_processing: true,
|
||||
reasons: ['seeded for timeout config test'],
|
||||
score: 0.9,
|
||||
content_type: null,
|
||||
segments: [],
|
||||
entities: [],
|
||||
model: TIER_DEFAULTS.utility,
|
||||
triage_version: TRIAGE_VERSION,
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,151 @@
|
||||
/**
|
||||
* #4152 [→EVAL] — triage score-band calibration fixtures.
|
||||
*
|
||||
* Twenty SYNTHETIC transcripts (anonymized mirrors — never real brain
|
||||
* content, per the calibration-corpus rule) with expected score BANDS:
|
||||
* high ≥ 0.70, low ≤ 0.29 — bands, not point values, because LLM scores
|
||||
* jitter. Two layers:
|
||||
*
|
||||
* 1. CI layer (always runs, mock judge): a deterministic scripted judge
|
||||
* derived from the fixture's labeled band pins the parse → gate →
|
||||
* cache plumbing end to end. This is the prompt-REGRESSION guard: if
|
||||
* the judge JSON schema or band semantics drift, this file fails.
|
||||
*
|
||||
* 2. Live layer (env-gated: GBRAIN_TRIAGE_CALIBRATION_LIVE=1 + a real
|
||||
* key): sends the fixtures to the actual resolved utility model and
|
||||
* reports band accuracy — the operator-run calibration loop the
|
||||
* cascade literature calls for. Never runs in CI; costs ~$0.05.
|
||||
*
|
||||
* Run: bun test test/cycle-synthesize-triage-calibration.test.ts
|
||||
* Live: GBRAIN_TRIAGE_CALIBRATION_LIVE=1 bun test test/cycle-synthesize-triage-calibration.test.ts
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import {
|
||||
judgeSignificance,
|
||||
makeJudgeClient,
|
||||
DEFAULT_TRIAGE_THRESHOLD,
|
||||
type JudgeClient,
|
||||
} from '../src/core/cycle/synthesize.ts';
|
||||
import type { DiscoveredTranscript } from '../src/core/cycle/transcript-discovery.ts';
|
||||
|
||||
interface Fixture {
|
||||
name: string;
|
||||
band: 'high' | 'low';
|
||||
content: string;
|
||||
}
|
||||
|
||||
/** Synthetic fixtures — generic placeholders only (alice-example, acme-example, fund-a). */
|
||||
const FIXTURES: Fixture[] = [
|
||||
// ── HIGH band: theses, self-reflection, strategic calls, people depth ──
|
||||
{ name: 'thesis-durability-pricing', band: 'high', content: 'User: I keep coming back to this — we should charge for durability, not storage. Storage is a commodity race to zero, but nobody else can promise your notes survive twenty years of format churn. That is the actual product.\nAssistant: That reframes the pricing page entirely.\nUser: Write it down: durability is the moat. If acme-example copies the feature set, they still cannot copy a decade of trust.' },
|
||||
{ name: 'reflection-conflict-avoidance', band: 'high', content: 'User: I noticed I do this thing where I avoid hard conversations with senior people for weeks, and then it explodes. It happened with alice-example last quarter and again this week. I think the pattern is that I conflate disagreement with disloyalty.\nAssistant: What would breaking the pattern look like?\nUser: Booking the hard conversation within 48 hours, before the story calcifies in my head.' },
|
||||
{ name: 'strategy-fund-allocation', band: 'high', content: 'User: Decision made: fund-a gets the follow-on, fund-b does not. The difference is founder velocity — alice-example shipped four experiments while widget-co polished one deck. I want to remember this heuristic: bet on iteration count, not polish.\nAssistant: Logged. This contradicts your earlier stated preference for polish.\nUser: Right — and the contradiction is the insight. Update my priors.' },
|
||||
{ name: 'idea-agent-memory-market', band: 'high', content: 'User: New frame: agent memory is not a database problem, it is a trust problem. Every agent vendor will ship their own memory, but nobody wants their memories locked to a vendor. The open, portable brain wins the same way open formats beat proprietary ones.\nAssistant: So portability is the wedge.\nUser: Portability plus verifiable recall. That pair is the thesis.' },
|
||||
{ name: 'people-cofounder-dynamics', band: 'high', content: 'User: Long talk with charlie-example today about the cofounder split at acme-example. The root cause was not equity, it was decision-rights ambiguity — they never wrote down who owns product. Charlie said something that stuck: "every unwritten agreement is a loan against the friendship."\nAssistant: Worth capturing for the founder guide?\nUser: Yes, with that quote verbatim.' },
|
||||
{ name: 'mental-model-reversibility', band: 'high', content: 'User: I want to codify the decision rule I have been circling: classify every decision as a one-way or two-way door BEFORE debating it. We spent three weeks on a reversible choice last month. The meta-decision matters more than the decision.\nAssistant: How do you tell them apart quickly?\nUser: If undoing costs less than a week, it is a two-way door. Act, do not deliberate.' },
|
||||
{ name: 'emotional-processing-launch', band: 'high', content: 'User: Honestly the launch flopping hit me harder than I admitted in standup. I notice I perform optimism for the team and then carry the doubt home. Naming it here: the performance is costing me sleep and probably fooling nobody.\nAssistant: What would honest optimism look like?\nUser: "The numbers are bad, here is what we learn, here is the next swing." Truth first, hope second.' },
|
||||
{ name: 'thesis-eval-driven-product', band: 'high', content: 'User: Big realization from this week: evals are not QA, they are product strategy. The features we can measure improve weekly; the ones we cannot measure rot. So the roadmap question is not "what should we build" but "what can we make measurable first."\nAssistant: That inverts the usual planning order.\nUser: Exactly. Measurability is the prerequisite, not the afterthought.' },
|
||||
{ name: 'people-hiring-bar', band: 'high', content: 'User: Debrief on the alice-example interview: technically strong, but I realized mid-loop that I was lowering the bar because we are desperate for the role. Desperation discount is a real failure mode — write this down as a hiring principle: the bar rises when the need is urgent, never falls.\nAssistant: Captured with the context.\nUser: Add: if we would not hire them in a talent surplus, we do not hire them now.' },
|
||||
{ name: 'strategic-call-sunset', band: 'high', content: 'User: Final call: we sunset the widget-co integration in Q3. It serves 2% of users, consumes 20% of on-call, and every incident erodes trust in the core product. I keep relearning the same lesson — kill the long tail before it kills focus.\nAssistant: Communicating it externally?\nUser: Honest deprecation note, 90-day window, migration script. The way we exit says more than the way we launch.' },
|
||||
// ── LOW band: logistics, routine ops, debugging without reflection ──
|
||||
{ name: 'logistics-scheduling', band: 'low', content: 'User: Can you move my 3pm to 4pm tomorrow and send the invite to the team?\nAssistant: Done, moved to 4pm and invited the team.\nUser: Also book a room with a whiteboard.\nAssistant: Room 4B booked.' },
|
||||
{ name: 'routine-email-check', band: 'low', content: 'User: Check my email for anything urgent.\nAssistant: Two newsletters and a receipt, nothing urgent.\nUser: OK archive the newsletters.\nAssistant: Archived.' },
|
||||
{ name: 'debugging-null-pointer', band: 'low', content: 'User: The build is failing with a null pointer in the config loader.\nAssistant: The loader dereferences cfg.paths before the null check on line 42. Swap the guard above the dereference.\nUser: That fixed it, thanks.\nAssistant: Anything else?\nUser: No, we are green now.' },
|
||||
{ name: 'routine-expense', band: 'low', content: 'User: File the receipt from lunch as a team expense, category meals.\nAssistant: Filed, $84.20 under meals.\nUser: And the parking one too.\nAssistant: Filed, $12.00 under travel.' },
|
||||
{ name: 'smalltalk-weather', band: 'low', content: 'User: Nice day out.\nAssistant: Clear skies through the weekend apparently.\nUser: Might bike in tomorrow then.\nAssistant: The forecast says dry until Sunday.' },
|
||||
{ name: 'routine-password-reset', band: 'low', content: 'User: I am locked out of the analytics dashboard again.\nAssistant: Sent a reset link to your inbox.\nUser: Got it, I am in.\nAssistant: Consider adding it to the password manager.' },
|
||||
{ name: 'debugging-css-alignment', band: 'low', content: 'User: The sidebar overlaps the footer on mobile.\nAssistant: The sidebar has position fixed without a bottom offset; add margin-bottom equal to the footer height or switch to sticky.\nUser: Sticky worked.\nAssistant: Shipping it.' },
|
||||
{ name: 'logistics-travel', band: 'low', content: 'User: Book the usual flight to the Tuesday board meeting, aisle seat.\nAssistant: Booked, aisle, arriving 9:40am.\nUser: Car from the airport too.\nAssistant: Scheduled.' },
|
||||
{ name: 'routine-status-update', band: 'low', content: 'User: What shipped yesterday?\nAssistant: Three PRs merged: the retry fix, the logging cleanup, and the dependency bump.\nUser: Any incidents?\nAssistant: None, error rates flat.' },
|
||||
{ name: 'repetitive-reminder', band: 'low', content: 'User: Remind me to water the plants Friday.\nAssistant: Reminder set for Friday 9am.\nUser: Weekly, actually.\nAssistant: Changed to weekly.' },
|
||||
];
|
||||
|
||||
function toTranscript(f: Fixture): DiscoveredTranscript {
|
||||
return {
|
||||
filePath: `/synthetic/${f.name}.txt`,
|
||||
contentHash: `synthetic-${f.name}`,
|
||||
content: f.content,
|
||||
basename: f.name,
|
||||
inferredDate: null,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Deterministic scripted judge: emits a band-consistent triage-v1 JSON for
|
||||
* each fixture. Pins the parse → derive → gate plumbing without any network.
|
||||
*/
|
||||
function scriptedJudge(): JudgeClient {
|
||||
return {
|
||||
create: async (p) => {
|
||||
const userMsg = String((p as { messages: Array<{ content: string }> }).messages[0].content);
|
||||
const fixture = FIXTURES.find(f => userMsg.includes(f.name)) ?? FIXTURES.find(f => userMsg.includes(f.content.slice(0, 40)));
|
||||
const band = fixture?.band ?? 'low';
|
||||
const score = band === 'high' ? 0.85 : 0.1;
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: JSON.stringify({
|
||||
score,
|
||||
content_type: band === 'high' ? 'reflection' : 'routine',
|
||||
segments: band === 'high' ? [{ quote: fixture!.content.slice(0, 80), note: 'fixture' }] : [],
|
||||
entities: [],
|
||||
reasons: [band],
|
||||
}),
|
||||
}],
|
||||
stop_reason: 'end_turn',
|
||||
} as never;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
describe('triage calibration fixtures — CI layer (mock judge, deterministic)', () => {
|
||||
test('fixture set shape: 20 fixtures, 10 per band, all synthetic placeholders', () => {
|
||||
expect(FIXTURES).toHaveLength(20);
|
||||
expect(FIXTURES.filter(f => f.band === 'high')).toHaveLength(10);
|
||||
expect(FIXTURES.filter(f => f.band === 'low')).toHaveLength(10);
|
||||
});
|
||||
|
||||
test('every fixture parses to a band-consistent verdict and gates correctly at the default threshold', async () => {
|
||||
const judge = scriptedJudge();
|
||||
for (const f of FIXTURES) {
|
||||
const r = await judgeSignificance(judge, toTranscript(f));
|
||||
expect(r.unreliable).toBeUndefined();
|
||||
if (f.band === 'high') {
|
||||
expect(r.score).toBeGreaterThanOrEqual(0.7);
|
||||
expect(r.score >= DEFAULT_TRIAGE_THRESHOLD).toBe(true);
|
||||
expect(r.worth_processing).toBe(true);
|
||||
} else {
|
||||
expect(r.score).toBeLessThanOrEqual(0.29);
|
||||
expect(r.worth_processing).toBe(false);
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── Live layer: opt-in real-model calibration (never in CI) ──
|
||||
const LIVE = process.env.GBRAIN_TRIAGE_CALIBRATION_LIVE === '1';
|
||||
const describeLive = LIVE ? describe : describe.skip;
|
||||
|
||||
describeLive('triage calibration — LIVE utility model (opt-in, ~$0.05)', () => {
|
||||
test('band accuracy ≥ 80% on the synthetic corpus', async () => {
|
||||
const model = 'anthropic:claude-haiku-4-5-20251001';
|
||||
const judge = makeJudgeClient(model);
|
||||
if (!judge) {
|
||||
throw new Error('GBRAIN_TRIAGE_CALIBRATION_LIVE=1 but no reachable provider for the utility model');
|
||||
}
|
||||
let correct = 0;
|
||||
const misses: string[] = [];
|
||||
for (const f of FIXTURES) {
|
||||
const r = await judgeSignificance(judge, toTranscript(f), model);
|
||||
if (r.unreliable) { misses.push(`${f.name}: unreliable(${r.unreliable})`); continue; }
|
||||
const passed = r.score >= DEFAULT_TRIAGE_THRESHOLD;
|
||||
const expected = f.band === 'high';
|
||||
if (passed === expected) correct++;
|
||||
else misses.push(`${f.name}: band=${f.band} score=${r.score}`);
|
||||
}
|
||||
const accuracy = correct / FIXTURES.length;
|
||||
console.error(`[calibration] band accuracy: ${(accuracy * 100).toFixed(0)}% | misses: ${misses.join('; ') || 'none'}`);
|
||||
expect(accuracy).toBeGreaterThanOrEqual(0.8);
|
||||
}, 300_000);
|
||||
});
|
||||
@@ -0,0 +1,457 @@
|
||||
/**
|
||||
* #4152 two-stage cascade — runTriagePass + buildTriageMapBlock +
|
||||
* parseSynthV2Key unit tests.
|
||||
*
|
||||
* runTriagePass touches the engine only through get/putDreamVerdict, so a
|
||||
* Map-backed fake engine keeps these tests fast and deterministic (no PGLite
|
||||
* startup). The judge is injected via cfg.judge; the clock via cfg.now — no
|
||||
* real sleeps anywhere.
|
||||
*
|
||||
* Run: bun test test/cycle-synthesize-triage.test.ts
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import {
|
||||
runTriagePass,
|
||||
buildTriageMapBlock,
|
||||
parseSynthV2Key,
|
||||
dreamInlineQueueAgeMs,
|
||||
DREAM_INLINE_LIVE_GRACE_MS,
|
||||
TRIAGE_VERSION,
|
||||
type JudgeClient,
|
||||
type TriagePassCfg,
|
||||
} from '../src/core/cycle/synthesize.ts';
|
||||
import { AIConfigError } from '../src/core/ai/errors.ts';
|
||||
import type { BrainEngine, DreamVerdict, DreamVerdictInput } from '../src/core/engine.ts';
|
||||
import type { DiscoveredTranscript } from '../src/core/cycle/transcript-discovery.ts';
|
||||
|
||||
const MODEL = 'anthropic:claude-haiku-4-5-20251001';
|
||||
|
||||
function makeTranscript(name: string, content = `content of ${name} `.repeat(50)): DiscoveredTranscript {
|
||||
return {
|
||||
filePath: `/corpus/${name}.txt`,
|
||||
contentHash: `hash-${name}`.padEnd(20, '0'),
|
||||
content,
|
||||
basename: name,
|
||||
inferredDate: null,
|
||||
};
|
||||
}
|
||||
|
||||
/** Map-backed fake engine exposing only the two verdict methods runTriagePass uses. */
|
||||
function makeFakeEngine(): { engine: BrainEngine; rows: Map<string, DreamVerdict>; putCalls: number } {
|
||||
const rows = new Map<string, DreamVerdict>();
|
||||
const state = { putCalls: 0 };
|
||||
const engine = {
|
||||
async getDreamVerdict(filePath: string, contentHash: string): Promise<DreamVerdict | null> {
|
||||
return rows.get(`${filePath}|${contentHash}`) ?? null;
|
||||
},
|
||||
async putDreamVerdict(filePath: string, contentHash: string, v: DreamVerdictInput): Promise<void> {
|
||||
state.putCalls++;
|
||||
rows.set(`${filePath}|${contentHash}`, { ...v, judged_at: new Date().toISOString() });
|
||||
},
|
||||
} as unknown as BrainEngine;
|
||||
return {
|
||||
engine,
|
||||
rows,
|
||||
get putCalls() { return state.putCalls; },
|
||||
};
|
||||
}
|
||||
|
||||
function scoredJudge(score: number, extra: Record<string, unknown> = {}): JudgeClient {
|
||||
return {
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: JSON.stringify({ score, reasons: ['mock'], ...extra }) }],
|
||||
stop_reason: 'end_turn',
|
||||
} as never),
|
||||
};
|
||||
}
|
||||
|
||||
function baseCfg(judge: JudgeClient | null, over: Partial<TriagePassCfg> = {}): TriagePassCfg {
|
||||
return {
|
||||
model: MODEL,
|
||||
maxChars: 24_000,
|
||||
maxTokens: 2048,
|
||||
threshold: 0.5,
|
||||
concurrency: 4,
|
||||
maxMs: 0,
|
||||
judge,
|
||||
...over,
|
||||
};
|
||||
}
|
||||
|
||||
function seedVerdict(rows: Map<string, DreamVerdict>, t: DiscoveredTranscript, v: Partial<DreamVerdict>): void {
|
||||
rows.set(`${t.filePath}|${t.contentHash}`, {
|
||||
worth_processing: true,
|
||||
reasons: ['seed'],
|
||||
judged_at: new Date().toISOString(),
|
||||
score: 0.9,
|
||||
content_type: null,
|
||||
segments: [],
|
||||
entities: [],
|
||||
model: MODEL,
|
||||
triage_version: TRIAGE_VERSION,
|
||||
...v,
|
||||
});
|
||||
}
|
||||
|
||||
describe('runTriagePass — cache validity (C8)', () => {
|
||||
test('valid cached row is a HIT: no judge call, no cache write, report cached=true, byPath populated', async () => {
|
||||
const fake = makeFakeEngine();
|
||||
const t = makeTranscript('cached');
|
||||
seedVerdict(fake.rows, t, { score: 0.8 });
|
||||
let judgeCalls = 0;
|
||||
const judge: JudgeClient = { create: async () => { judgeCalls++; throw new Error('should not be called'); } };
|
||||
const r = await runTriagePass(fake.engine, [t], baseCfg(judge));
|
||||
expect(judgeCalls).toBe(0);
|
||||
expect(fake.putCalls).toBe(0); // hit path never re-writes the row
|
||||
expect(r.cacheHits).toBe(1);
|
||||
expect(r.reports[0].cached).toBe(true);
|
||||
expect(r.reports[0].worth).toBe(true);
|
||||
expect(r.byPath.get(t.filePath)?.score).toBe(0.8);
|
||||
});
|
||||
|
||||
test('legacy boolean-era row (score null) is a MISS — re-judged and overwritten', async () => {
|
||||
const { engine, rows } = makeFakeEngine();
|
||||
const t = makeTranscript('legacy');
|
||||
seedVerdict(rows, t, { score: null, triage_version: null, model: null });
|
||||
const r = await runTriagePass(engine, [t], baseCfg(scoredJudge(0.7)));
|
||||
expect(r.judged).toBe(1);
|
||||
expect(r.cacheHits).toBe(0);
|
||||
expect(rows.get(`${t.filePath}|${t.contentHash}`)?.score).toBe(0.7);
|
||||
expect(rows.get(`${t.filePath}|${t.contentHash}`)?.triage_version).toBe(TRIAGE_VERSION);
|
||||
});
|
||||
|
||||
test('model mismatch is a MISS — switching models re-judges (C8)', async () => {
|
||||
const { engine, rows } = makeFakeEngine();
|
||||
const t = makeTranscript('model-switch');
|
||||
seedVerdict(rows, t, { score: 0.9, model: 'anthropic:some-other-model' });
|
||||
const r = await runTriagePass(engine, [t], baseCfg(scoredJudge(0.3)));
|
||||
expect(r.judged).toBe(1);
|
||||
expect(rows.get(`${t.filePath}|${t.contentHash}`)?.model).toBe(MODEL);
|
||||
expect(rows.get(`${t.filePath}|${t.contentHash}`)?.score).toBe(0.3);
|
||||
});
|
||||
|
||||
test('triage_version mismatch is a MISS', async () => {
|
||||
const { engine, rows } = makeFakeEngine();
|
||||
const t = makeTranscript('version-bump');
|
||||
seedVerdict(rows, t, { score: 0.9, triage_version: TRIAGE_VERSION + 1 });
|
||||
const r = await runTriagePass(engine, [t], baseCfg(scoredJudge(0.6)));
|
||||
expect(r.judged).toBe(1);
|
||||
});
|
||||
|
||||
test('staleBefore treats older rows as misses; force ignores the cache entirely', async () => {
|
||||
const { engine, rows } = makeFakeEngine();
|
||||
const t = makeTranscript('stale');
|
||||
seedVerdict(rows, t, { score: 0.9, judged_at: '2020-01-01T00:00:00.000Z' });
|
||||
const stale = await runTriagePass(engine, [t], baseCfg(scoredJudge(0.6), { staleBefore: new Date('2025-01-01') }));
|
||||
expect(stale.judged).toBe(1);
|
||||
// Fresh again now; force still re-judges.
|
||||
const forced = await runTriagePass(engine, [t], baseCfg(scoredJudge(0.2), { force: true }));
|
||||
expect(forced.judged).toBe(1);
|
||||
expect(rows.get(`${t.filePath}|${t.contentHash}`)?.score).toBe(0.2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('runTriagePass — gate + threshold', () => {
|
||||
test('>= threshold boundary: exactly-at passes, just-below does not', async () => {
|
||||
const { engine, rows } = makeFakeEngine();
|
||||
const at = makeTranscript('at');
|
||||
const below = makeTranscript('below');
|
||||
seedVerdict(rows, at, { score: 0.5 });
|
||||
seedVerdict(rows, below, { score: 0.4999 });
|
||||
const r = await runTriagePass(engine, [at, below], baseCfg(null, { threshold: 0.5 }));
|
||||
expect(r.reports.find(x => x.filePath === at.filePath)?.worth).toBe(true);
|
||||
expect(r.reports.find(x => x.filePath === below.filePath)?.worth).toBe(false);
|
||||
});
|
||||
|
||||
test('threshold 0 gates everything judged in (a low score still passes)', async () => {
|
||||
const { engine, rows } = makeFakeEngine();
|
||||
const t = makeTranscript('zero');
|
||||
seedVerdict(rows, t, { score: 0 });
|
||||
const r = await runTriagePass(engine, [t], baseCfg(null, { threshold: 0 }));
|
||||
expect(r.reports[0].worth).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('runTriagePass — time budget (1C) + shouldStop', () => {
|
||||
test('maxMs expiry defers remaining MISSES (uncached) while cache hits stay free', async () => {
|
||||
const { engine, rows } = makeFakeEngine();
|
||||
const miss1 = makeTranscript('m1');
|
||||
const miss2 = makeTranscript('m2');
|
||||
const miss3 = makeTranscript('m3');
|
||||
const hit = makeTranscript('hit');
|
||||
seedVerdict(rows, hit, { score: 0.9 });
|
||||
// Fake clock: each now() call advances 40ms; budget 100ms → the first
|
||||
// miss judges, later misses defer. concurrency 1 for determinism.
|
||||
let clock = 0;
|
||||
const now = (): number => { clock += 40; return clock; };
|
||||
const r = await runTriagePass(engine, [miss1, miss2, miss3, hit], baseCfg(scoredJudge(0.8), {
|
||||
concurrency: 1,
|
||||
maxMs: 100,
|
||||
now,
|
||||
}));
|
||||
expect(r.judged).toBeGreaterThanOrEqual(1);
|
||||
expect(r.deferred).toBeGreaterThanOrEqual(1);
|
||||
expect(r.cacheHits).toBe(1); // the hit is NEVER deferred
|
||||
const deferredReports = r.reports.filter(x => x.deferred);
|
||||
expect(deferredReports.length).toBe(r.deferred);
|
||||
for (const d of deferredReports) {
|
||||
expect(d.worth).toBe(false);
|
||||
expect(d.score).toBeNull();
|
||||
// Deferred files are NOT cached — next pass continues.
|
||||
expect([...rows.keys()].some(k => k.startsWith(d.filePath))).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
test('in-flight semantics: a judge call started inside the budget completes and IS cached', async () => {
|
||||
const { engine, rows } = makeFakeEngine();
|
||||
const t = makeTranscript('inflight');
|
||||
// Clock: 0 at start; stays 0 until the judge call BEGINS, then jumps past
|
||||
// the budget while the call is in flight. The pull happened inside the
|
||||
// budget, so the verdict must complete and be cached (no torn judgments).
|
||||
let clock = 0;
|
||||
const now = (): number => clock;
|
||||
const judge: JudgeClient = {
|
||||
create: async (p) => {
|
||||
clock = 10_000; // budget (500ms) expires mid-call
|
||||
return scoredJudge(0.7).create(p);
|
||||
},
|
||||
};
|
||||
const r = await runTriagePass(engine, [t], baseCfg(judge, { maxMs: 500, now }));
|
||||
expect(r.judged).toBe(1);
|
||||
expect(r.deferred).toBe(0);
|
||||
expect(rows.size).toBe(1); // cached despite expiry mid-flight
|
||||
});
|
||||
|
||||
test('CX3: shouldStop is ticked on UNRELIABLE judge attempts too (paid calls count)', async () => {
|
||||
const { engine } = makeFakeEngine();
|
||||
const ts = [makeTranscript('u1'), makeTranscript('u2'), makeTranscript('u3')];
|
||||
let ticks = 0;
|
||||
// Judge always truncates — every attempt is unreliable but still paid.
|
||||
const judge: JudgeClient = {
|
||||
create: async () => ({ content: [{ type: 'text', text: '{"scor' }], stop_reason: 'max_tokens' } as never),
|
||||
};
|
||||
const r = await runTriagePass(engine, ts, baseCfg(judge, {
|
||||
concurrency: 1,
|
||||
shouldStop: () => { ticks++; return ticks >= 2; },
|
||||
}));
|
||||
expect(ticks).toBe(2); // budget consumed by unreliable attempts
|
||||
expect(r.unreliable).toBe(2);
|
||||
expect(r.deferred).toBe(1); // third file never pulled
|
||||
});
|
||||
|
||||
test('shouldStop stops pulling new misses (retriage --max-usd seam)', async () => {
|
||||
const { engine } = makeFakeEngine();
|
||||
const ts = [makeTranscript('s1'), makeTranscript('s2'), makeTranscript('s3'), makeTranscript('s4')];
|
||||
let judged = 0;
|
||||
const judge = scoredJudge(0.6);
|
||||
const counting: JudgeClient = { create: async (p) => { judged++; return judge.create(p); } };
|
||||
const r = await runTriagePass(engine, ts, baseCfg(counting, {
|
||||
concurrency: 1,
|
||||
shouldStop: () => judged >= 2,
|
||||
}));
|
||||
expect(r.judged).toBe(2);
|
||||
expect(r.deferred).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('runTriagePass — degrade + failure contracts', () => {
|
||||
test('null judge (no provider) degrades per transcript; nothing cached; cache hits still served', async () => {
|
||||
const { engine, rows } = makeFakeEngine();
|
||||
const hit = makeTranscript('hit');
|
||||
const miss = makeTranscript('miss');
|
||||
seedVerdict(rows, hit, { score: 0.9 });
|
||||
const r = await runTriagePass(engine, [hit, miss], baseCfg(null));
|
||||
expect(r.cacheHits).toBe(1);
|
||||
const missReport = r.reports.find(x => x.filePath === miss.filePath)!;
|
||||
expect(missReport.worth).toBe(false);
|
||||
expect(missReport.reasons[0]).toContain('no configured provider');
|
||||
expect(rows.size).toBe(1); // only the seed
|
||||
});
|
||||
|
||||
test('AIConfigError degrades per transcript (gateway error reason), pass continues', async () => {
|
||||
const { engine } = makeFakeEngine();
|
||||
const bad = makeTranscript('bad');
|
||||
const good = makeTranscript('good');
|
||||
let call = 0;
|
||||
const judge: JudgeClient = {
|
||||
create: async (p) => {
|
||||
call++;
|
||||
if (call === 1) throw new AIConfigError('simulated revoked key');
|
||||
return scoredJudge(0.8).create(p);
|
||||
},
|
||||
};
|
||||
const r = await runTriagePass(engine, [bad, good], baseCfg(judge, { concurrency: 1 }));
|
||||
expect(r.reports[0].reasons[0]).toContain('gateway error');
|
||||
expect(r.reports[1].score).toBe(0.8);
|
||||
});
|
||||
|
||||
test('hard (non-AIConfig) error aborts new pulls and rethrows — phase fails', async () => {
|
||||
const { engine } = makeFakeEngine();
|
||||
const ts = [makeTranscript('h1'), makeTranscript('h2'), makeTranscript('h3')];
|
||||
const judge: JudgeClient = { create: async () => { throw new Error('database on fire'); } };
|
||||
await expect(runTriagePass(engine, ts, baseCfg(judge, { concurrency: 1 }))).rejects.toThrow('database on fire');
|
||||
});
|
||||
|
||||
test('unreliable judgments are reported but never cached', async () => {
|
||||
const { engine, rows } = makeFakeEngine();
|
||||
const t = makeTranscript('trunc');
|
||||
const judge: JudgeClient = {
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: '{"scor' }],
|
||||
stop_reason: 'max_tokens',
|
||||
} as never),
|
||||
};
|
||||
const r = await runTriagePass(engine, [t], baseCfg(judge));
|
||||
expect(r.unreliable).toBe(1);
|
||||
expect(r.reports[0].unreliable).toBe('truncated');
|
||||
expect(rows.size).toBe(0);
|
||||
expect(r.byPath.has(t.filePath)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('runTriagePass — pool + reports + lock tick', () => {
|
||||
test('reports are index-stable in discovery order regardless of concurrency', async () => {
|
||||
const { engine } = makeFakeEngine();
|
||||
const ts = Array.from({ length: 9 }, (_, i) => makeTranscript(`ord${i}`));
|
||||
const r = await runTriagePass(engine, ts, baseCfg(scoredJudge(0.6), { concurrency: 4 }));
|
||||
expect(r.reports.map(x => x.filePath)).toEqual(ts.map(t => t.filePath));
|
||||
});
|
||||
|
||||
test('yieldDuringPhase ticks are coarse: at most one per 30s window (C11)', async () => {
|
||||
const { engine } = makeFakeEngine();
|
||||
const ts = Array.from({ length: 6 }, (_, i) => makeTranscript(`tick${i}`));
|
||||
let ticks = 0;
|
||||
// Fake clock advances 10s per now() call — several items settle inside
|
||||
// each 30s window, so ticks must be well below the item count.
|
||||
let clock = 0;
|
||||
const now = (): number => { clock += 10_000; return clock; };
|
||||
const r = await runTriagePass(engine, ts, baseCfg(scoredJudge(0.6), { concurrency: 1, now }), async () => { ticks++; });
|
||||
expect(r.judged).toBe(6);
|
||||
expect(ticks).toBeGreaterThan(0);
|
||||
expect(ticks).toBeLessThan(6);
|
||||
});
|
||||
|
||||
test('yieldDuringPhase throwing is swallowed (best-effort)', async () => {
|
||||
const { engine } = makeFakeEngine();
|
||||
const ts = Array.from({ length: 3 }, (_, i) => makeTranscript(`yt${i}`));
|
||||
let clock = 0;
|
||||
const now = (): number => { clock += 40_000; return clock; };
|
||||
const r = await runTriagePass(engine, ts, baseCfg(scoredJudge(0.6), { concurrency: 1, now }),
|
||||
async () => { throw new Error('tick boom'); });
|
||||
expect(r.judged).toBe(3);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildTriageMapBlock', () => {
|
||||
const verdict = {
|
||||
score: 0.82,
|
||||
content_type: 'reflection',
|
||||
segments: [
|
||||
{ quote: 'the future of memory is a database that dreams', note: 'thesis' },
|
||||
{ quote: 'we should charge for durability not storage', note: 'pricing frame' },
|
||||
],
|
||||
entities: ['acme-example', 'fund-a'],
|
||||
};
|
||||
|
||||
test('empty for undefined verdict and for legacy (score null) — prompt stays byte-identical', () => {
|
||||
expect(buildTriageMapBlock(undefined, 'text', 1)).toBe('');
|
||||
expect(buildTriageMapBlock({ score: null, content_type: null, segments: [], entities: [] }, 'text', 1)).toBe('');
|
||||
});
|
||||
|
||||
test('single-chunk block carries score, type, entities, and verbatim-verified segments', () => {
|
||||
// The presence filter applies to EVERY chunk count (fabricated quotes are
|
||||
// dropped), so the chunk text must actually contain the quotes.
|
||||
const fullText = 'intro… the future of memory is a database that dreams. later: '
|
||||
+ 'we should charge for durability not storage. outro.';
|
||||
const block = buildTriageMapBlock(verdict, fullText, 1);
|
||||
expect(block).toContain('TRIAGE MAP');
|
||||
expect(block).toContain('signal score: 0.82');
|
||||
expect(block).toContain('content type: reflection');
|
||||
expect(block).toContain('acme-example, fund-a');
|
||||
expect(block).toContain('database that dreams');
|
||||
expect(block).toContain('charge for durability');
|
||||
expect(block).not.toContain('bounded sample');
|
||||
expect(block).toContain('Work from the candidate segments first');
|
||||
});
|
||||
|
||||
test('fabricated quotes are dropped even for single-chunk transcripts (security)', () => {
|
||||
const block = buildTriageMapBlock(verdict, 'text that contains neither quote', 1);
|
||||
expect(block).not.toContain('database that dreams');
|
||||
expect(block).not.toContain('charge for durability');
|
||||
// Score/type/entities still ride (they are advisory labels, not quotes).
|
||||
expect(block).toContain('signal score: 0.82');
|
||||
});
|
||||
|
||||
test('chunked: segments filter to those whose quote prefix appears in THIS chunk + caveat line', () => {
|
||||
const chunkWithFirst = 'blah blah the future of memory is a database that dreams blah';
|
||||
const block = buildTriageMapBlock(verdict, chunkWithFirst, 3);
|
||||
expect(block).toContain('database that dreams');
|
||||
expect(block).not.toContain('charge for durability');
|
||||
expect(block).toContain('bounded sample of the full transcript');
|
||||
});
|
||||
|
||||
test('whitespace-normalized matching: quote with collapsed spacing still matches', () => {
|
||||
const chunk = 'x the future\nof memory is a database that dreams y';
|
||||
const block = buildTriageMapBlock(verdict, chunk, 2);
|
||||
expect(block).toContain('database that dreams');
|
||||
});
|
||||
|
||||
test('size bound: worst-case block stays bounded', () => {
|
||||
const segments = Array.from({ length: 8 }, (_, i) => ({ quote: `q${i} ` + 'x'.repeat(300), note: 'n'.repeat(200) }));
|
||||
const fat = {
|
||||
score: 0.99,
|
||||
content_type: 'mixed',
|
||||
segments,
|
||||
entities: Array.from({ length: 12 }, (_, i) => `entity-${i}-` + 'e'.repeat(70)),
|
||||
};
|
||||
// Chunk text contains every quote so the presence filter keeps all 8.
|
||||
const chunkText = segments.map(s => s.quote).join(' ');
|
||||
const block = buildTriageMapBlock(fat, chunkText, 1);
|
||||
expect(block).toContain('q7 ');
|
||||
expect(block.length).toBeLessThan(6000); // clipped upstream at judge-parse time; this is the structural bound
|
||||
});
|
||||
});
|
||||
|
||||
describe('dreamInlineQueueAgeMs (CX1 liveness)', () => {
|
||||
test('parses the embedded timestamp; null outside the grammar', () => {
|
||||
expect(dreamInlineQueueAgeMs('dream-inline-1700000000000-deadbeef', 1700000001000)).toBe(1000);
|
||||
expect(dreamInlineQueueAgeMs('dream-inline-not-a-ts-deadbeef', 1)).toBeNull();
|
||||
expect(dreamInlineQueueAgeMs('default', 1)).toBeNull();
|
||||
expect(dreamInlineQueueAgeMs('dream-inline-1700000000000-NOTHEX!', 1)).toBeNull();
|
||||
});
|
||||
|
||||
test('grace boundary: a fresh queue is within the liveness grace; an old one is past it', () => {
|
||||
const nowMs = 1_800_000_000_000;
|
||||
const young = `dream-inline-${nowMs - 60_000}-abcd1234`;
|
||||
const old = `dream-inline-${nowMs - DREAM_INLINE_LIVE_GRACE_MS - 60_000}-abcd1234`;
|
||||
expect(dreamInlineQueueAgeMs(young, nowMs)! <= DREAM_INLINE_LIVE_GRACE_MS).toBe(true);
|
||||
expect(dreamInlineQueueAgeMs(old, nowMs)! > DREAM_INLINE_LIVE_GRACE_MS).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseSynthV2Key', () => {
|
||||
test('single-chunk key round-trips source + basename + hash16', () => {
|
||||
const k = parseSynthV2Key('dream:synth-v2:my-source:filename:2026-05-01-session.txt:0123456789abcdef');
|
||||
expect(k).toEqual({ source: 'my-source', basename: '2026-05-01-session.txt', hash16: '0123456789abcdef' });
|
||||
});
|
||||
|
||||
test('chunked key carries chunk index + total', () => {
|
||||
const k = parseSynthV2Key('dream:synth-v2:default:filename:fat.txt:0123456789abcdef:c2of5');
|
||||
expect(k?.chunk).toEqual({ i: 2, n: 5 });
|
||||
});
|
||||
|
||||
test('percent-encoded basename decodes (spaces, unicode)', () => {
|
||||
const enc = encodeURIComponent('весёлый файл (1).txt');
|
||||
const k = parseSynthV2Key(`dream:synth-v2:default:filename:${enc}:0123456789abcdef`);
|
||||
expect(k?.basename).toBe('весёлый файл (1).txt');
|
||||
});
|
||||
|
||||
test('null on: legacy v1 keys, malformed hash, malformed encoding, foreign keys', () => {
|
||||
expect(parseSynthV2Key('dream:synth:/abs/path.txt:0123456789abcdef')).toBeNull();
|
||||
expect(parseSynthV2Key('dream:synth-v2:default:filename:x.txt:SHORT')).toBeNull();
|
||||
expect(parseSynthV2Key('dream:synth-v2:default:filename:%E0%A4%A:0123456789abcdef')).toBeNull();
|
||||
expect(parseSynthV2Key('embed-backfill:source:x')).toBeNull();
|
||||
expect(parseSynthV2Key('dream:synth-v2:default:filename:x.txt:0123456789abcdef:c1of')).toBeNull();
|
||||
});
|
||||
});
|
||||
+140
-32
@@ -305,11 +305,14 @@ describe('judgeSignificance', () => {
|
||||
};
|
||||
}
|
||||
|
||||
/** Triage-v1 mock output — the judge now emits a scored verdict. */
|
||||
const TRIAGE_JSON = '{"score": 0.9, "content_type": "reflection", "segments": [{"quote": "a memorable line", "note": "names a pattern"}], "entities": ["acme-example"], "reasons": ["test"]}';
|
||||
|
||||
function mockClient(captured: { model?: string }): JudgeClient {
|
||||
return {
|
||||
create: async (p: any) => {
|
||||
captured.model = p.model;
|
||||
return { content: [{ type: 'text', text: '{"worth_processing": true, "reasons": ["test"]}' }] } as any;
|
||||
return { content: [{ type: 'text', text: TRIAGE_JSON }] } as any;
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -326,24 +329,39 @@ describe('judgeSignificance', () => {
|
||||
expect(captured.model).toBe('claude-haiku-4-5-20251001');
|
||||
});
|
||||
|
||||
test('returns worth_processing=false when judge returns unparseable text', async () => {
|
||||
test('parses a scored triage verdict: score, content_type, segments, entities', async () => {
|
||||
const client: JudgeClient = {
|
||||
create: async () => ({ content: [{ type: 'text', text: TRIAGE_JSON }], stop_reason: 'end_turn' } as any),
|
||||
};
|
||||
const r = await judgeSignificance(client, makeTranscript());
|
||||
expect(r.score).toBe(0.9);
|
||||
expect(r.content_type).toBe('reflection');
|
||||
expect(r.segments).toEqual([{ quote: 'a memorable line', note: 'names a pattern' }]);
|
||||
expect(r.entities).toEqual(['acme-example']);
|
||||
expect(r.worth_processing).toBe(true); // derived: score >= DEFAULT_TRIAGE_THRESHOLD
|
||||
expect(r.unreliable).toBeUndefined();
|
||||
});
|
||||
|
||||
test('derived worth_processing uses the fixed constant: 0.5 passes, 0.49 does not', async () => {
|
||||
const at = await judgeSignificance({
|
||||
create: async () => ({ content: [{ type: 'text', text: '{"score": 0.5, "reasons": []}' }], stop_reason: 'end_turn' } as any),
|
||||
}, makeTranscript());
|
||||
expect(at.worth_processing).toBe(true);
|
||||
const below = await judgeSignificance({
|
||||
create: async () => ({ content: [{ type: 'text', text: '{"score": 0.49, "reasons": []}' }], stop_reason: 'end_turn' } as any),
|
||||
}, makeTranscript());
|
||||
expect(below.worth_processing).toBe(false);
|
||||
expect(below.unreliable).toBeUndefined(); // a low score is a VALID verdict, not degenerate
|
||||
});
|
||||
|
||||
test('returns score 0 + unparseable when judge returns non-JSON text', async () => {
|
||||
const client: JudgeClient = {
|
||||
create: async () => ({ content: [{ type: 'text', text: 'no json here' }] } as any),
|
||||
};
|
||||
const r = await judgeSignificance(client, makeTranscript());
|
||||
expect(r.worth_processing).toBe(false);
|
||||
expect(r.score).toBe(0);
|
||||
expect(r.reasons[0]).toContain('unparseable');
|
||||
});
|
||||
|
||||
test('marks unparseable output unreliable so the caller never caches it', async () => {
|
||||
const client: JudgeClient = {
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: 'no json here' }],
|
||||
stop_reason: 'end_turn',
|
||||
} as any),
|
||||
};
|
||||
const r = await judgeSignificance(client, makeTranscript());
|
||||
expect(r.worth_processing).toBe(false);
|
||||
expect(r.unreliable).toBe('unparseable');
|
||||
});
|
||||
|
||||
@@ -353,7 +371,7 @@ describe('judgeSignificance', () => {
|
||||
// stop_reason is 'max_tokens'.
|
||||
const client: JudgeClient = {
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: '{"worth_process' }],
|
||||
content: [{ type: 'text', text: '{"scor' }],
|
||||
stop_reason: 'max_tokens',
|
||||
} as any),
|
||||
};
|
||||
@@ -366,12 +384,13 @@ describe('judgeSignificance', () => {
|
||||
test('marks truncated response unreliable even when a parseable JSON object survives the cut', async () => {
|
||||
const client: JudgeClient = {
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: '{"worth_processing": true, "reasons": ["r1"]}' }],
|
||||
content: [{ type: 'text', text: '{"score": 0.8, "reasons": ["r1"]}' }],
|
||||
stop_reason: 'max_tokens',
|
||||
} as any),
|
||||
};
|
||||
const r = await judgeSignificance(client, makeTranscript());
|
||||
// The parsed values still drive THIS cycle...
|
||||
expect(r.score).toBe(0.8);
|
||||
expect(r.worth_processing).toBe(true);
|
||||
expect(r.reasons).toEqual(['r1']);
|
||||
// ...but the verdict must not be banked as permanent.
|
||||
@@ -381,7 +400,7 @@ describe('judgeSignificance', () => {
|
||||
test('clean parse with stop_reason=end_turn stays cacheable (no unreliable marker)', async () => {
|
||||
const client: JudgeClient = {
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: '{"worth_processing": true, "reasons": ["r1"]}' }],
|
||||
content: [{ type: 'text', text: '{"score": 0.8, "reasons": ["r1"]}' }],
|
||||
stop_reason: 'end_turn',
|
||||
} as any),
|
||||
};
|
||||
@@ -393,7 +412,7 @@ describe('judgeSignificance', () => {
|
||||
test('marks refused/content-filtered response (stop_reason=refusal) unreliable', async () => {
|
||||
const client: JudgeClient = {
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: '{"worth_processing": false, "reasons": ["blocked"]}' }],
|
||||
content: [{ type: 'text', text: '{"score": 0.1, "reasons": ["blocked"]}' }],
|
||||
stop_reason: 'refusal',
|
||||
} as any),
|
||||
};
|
||||
@@ -401,7 +420,7 @@ describe('judgeSignificance', () => {
|
||||
expect(r.unreliable).toBe('refusal');
|
||||
});
|
||||
|
||||
test('JSON object without worth_processing key is unparseable, not a cacheable false', async () => {
|
||||
test('JSON object without a score is unparseable, not a cacheable rejection', async () => {
|
||||
const client: JudgeClient = {
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: '{}' }],
|
||||
@@ -413,10 +432,10 @@ describe('judgeSignificance', () => {
|
||||
expect(r.unreliable).toBe('unparseable');
|
||||
});
|
||||
|
||||
test('non-boolean worth_processing ("true") is unparseable, not a cacheable false', async () => {
|
||||
test('non-numeric score ("0.7") is unparseable, not a cacheable rejection', async () => {
|
||||
const client: JudgeClient = {
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: '{"worth_processing": "true", "reasons": ["r1"]}' }],
|
||||
content: [{ type: 'text', text: '{"score": "0.7", "reasons": ["r1"]}' }],
|
||||
stop_reason: 'end_turn',
|
||||
} as any),
|
||||
};
|
||||
@@ -425,19 +444,101 @@ describe('judgeSignificance', () => {
|
||||
expect(r.unreliable).toBe('unparseable');
|
||||
});
|
||||
|
||||
test('judge max_tokens budget is exactly 1024 (reasoning-model headroom, cost-capped)', async () => {
|
||||
test('score outside [0,1] is unparseable — never clamped into a cacheable verdict', async () => {
|
||||
for (const bad of ['1.5', '-0.2', '42']) {
|
||||
const r = await judgeSignificance({
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: `{"score": ${bad}, "reasons": []}` }],
|
||||
stop_reason: 'end_turn',
|
||||
} as any),
|
||||
}, makeTranscript());
|
||||
expect(r.unreliable).toBe('unparseable');
|
||||
expect(r.reasons[0]).toContain('score out of range');
|
||||
}
|
||||
});
|
||||
|
||||
test('non-finite score (Infinity via 1e999) is unparseable', async () => {
|
||||
const r = await judgeSignificance({
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: '{"score": 1e999, "reasons": []}' }],
|
||||
stop_reason: 'end_turn',
|
||||
} as any),
|
||||
}, makeTranscript());
|
||||
expect(r.unreliable).toBe('unparseable');
|
||||
});
|
||||
|
||||
test('lenient optional fields: bad segments/entities/content_type drop, never degenerate', async () => {
|
||||
const r = await judgeSignificance({
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: '{"score": 0.6, "content_type": 42, "segments": [{"note": "no quote"}, "junk"], "entities": [1, "", " real-entity "], "reasons": ["ok"]}' }],
|
||||
stop_reason: 'end_turn',
|
||||
} as any),
|
||||
}, makeTranscript());
|
||||
expect(r.unreliable).toBeUndefined();
|
||||
expect(r.content_type).toBeNull();
|
||||
expect(r.segments).toEqual([]);
|
||||
expect(r.entities).toEqual(['real-entity']);
|
||||
});
|
||||
|
||||
test('segment/entity clipping: ≤8 segments, quotes ≤300 chars; ≤12 entities, each ≤80 chars', async () => {
|
||||
const segments = Array.from({ length: 12 }, (_, i) => ({ quote: `q${i}-` + 'x'.repeat(400), note: 'n'.repeat(300) }));
|
||||
const entities = Array.from({ length: 20 }, (_, i) => `e${i}-` + 'y'.repeat(100));
|
||||
const r = await judgeSignificance({
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: JSON.stringify({ score: 0.7, segments, entities, reasons: [] }) }],
|
||||
stop_reason: 'end_turn',
|
||||
} as any),
|
||||
}, makeTranscript());
|
||||
expect(r.segments).toHaveLength(8);
|
||||
expect(r.segments.every(s => s.quote.length <= 300)).toBe(true);
|
||||
expect(r.segments.every(s => (s.note ?? '').length <= 200)).toBe(true);
|
||||
expect(r.entities).toHaveLength(12);
|
||||
expect(r.entities.every(e => e.length <= 80)).toBe(true);
|
||||
});
|
||||
|
||||
test('judge max_tokens budget defaults to 2048 and honors the opts override', async () => {
|
||||
let captured: number | undefined;
|
||||
const client: JudgeClient = {
|
||||
create: async (p: any) => {
|
||||
captured = p.max_tokens;
|
||||
return {
|
||||
content: [{ type: 'text', text: '{"worth_processing": false, "reasons": []}' }],
|
||||
content: [{ type: 'text', text: '{"score": 0.1, "reasons": []}' }],
|
||||
stop_reason: 'end_turn',
|
||||
} as any;
|
||||
},
|
||||
};
|
||||
await judgeSignificance(client, makeTranscript());
|
||||
expect(captured).toBe(1024);
|
||||
expect(captured).toBe(2048);
|
||||
await judgeSignificance(client, makeTranscript(), 'claude-haiku-4-5-20251001', { maxTokens: 512 });
|
||||
expect(captured).toBe(512);
|
||||
});
|
||||
|
||||
test('sampled marker appended to reasons when the transcript exceeds maxChars', async () => {
|
||||
const longTranscript = { ...makeTranscript(), content: 'z'.repeat(30_000) };
|
||||
const r = await judgeSignificance({
|
||||
create: async () => ({
|
||||
content: [{ type: 'text', text: '{"score": 0.6, "reasons": ["r1"]}' }],
|
||||
stop_reason: 'end_turn',
|
||||
} as any),
|
||||
}, longTranscript, 'claude-haiku-4-5-20251001', { maxChars: 24_000 });
|
||||
expect(r.reasons.some(x => x.startsWith('sampled:'))).toBe(true);
|
||||
});
|
||||
|
||||
test('three-window sampling: mid-transcript content reaches the judge prompt', async () => {
|
||||
// 100K-char transcript with a unique marker dead-center — a head+tail
|
||||
// sample would miss it; the head/middle/tail sample must include it.
|
||||
const half = 'a'.repeat(50_000);
|
||||
const content = half + 'MIDDLE-SIGNAL-MARKER' + half;
|
||||
let prompt = '';
|
||||
await judgeSignificance({
|
||||
create: async (p: any) => {
|
||||
prompt = p.messages[0].content;
|
||||
return { content: [{ type: 'text', text: '{"score": 0.5, "reasons": []}' }], stop_reason: 'end_turn' } as any;
|
||||
},
|
||||
}, { ...makeTranscript(), content }, 'claude-haiku-4-5-20251001', { maxChars: 24_000 });
|
||||
expect(prompt).toContain('MIDDLE-SIGNAL-MARKER');
|
||||
// And the sample is bounded: prompt stays well under the raw 100K.
|
||||
expect(prompt.length).toBeLessThan(30_000);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -523,41 +624,48 @@ describe('judgeSignificance — UTF-16 safety (v0.41.13)', () => {
|
||||
};
|
||||
}
|
||||
|
||||
// ─── Head-boundary cases (offset around 4000) ──────────────────────
|
||||
// Triage-v1 note: the default sample window moved to 24K with three
|
||||
// windows (head 50% / middle 20% / tail 30%). These fixtures pass
|
||||
// `{ maxChars: 8000 }` to keep the 8001-char content on the slicing path;
|
||||
// at 8000 the window boundaries are head end = 4000, middle = [4000, 5600),
|
||||
// tail start = 5601. Every boundary routes through safeSplitIndex, so the
|
||||
// unpaired-surrogate scan is the invariant regardless of exact offsets.
|
||||
const SAMPLE_OPTS = { maxChars: 8000 };
|
||||
|
||||
// ─── Head-boundary cases (offset around 4000, also the middle-window start) ──
|
||||
|
||||
test.each([3998, 3999, 4000, 4001])(
|
||||
'emoji at head offset %i: captured prompt has zero unpaired surrogates',
|
||||
async (offset) => {
|
||||
const content = buildContentWithEmojiAt(8001, offset);
|
||||
const { client, captured } = makeCapturingClient();
|
||||
await judgeSignificance(client, makeLongTranscript(content));
|
||||
await judgeSignificance(client, makeLongTranscript(content), 'claude-haiku-4-5-20251001', SAMPLE_OPTS);
|
||||
expect(captured.userMessage).not.toBeNull();
|
||||
const result = scanForUnpairedSurrogates(captured.userMessage!);
|
||||
expect(result).toBeNull();
|
||||
},
|
||||
);
|
||||
|
||||
// ─── Tail-boundary cases (offset around length-4000 = 4001) ────────
|
||||
// ─── Middle-end + tail-boundary cases (5600 = middle end, 5601 = tail start) ──
|
||||
|
||||
test.each([3999, 4000, 4001, 4002])(
|
||||
test.each([5599, 5600, 5601, 5602])(
|
||||
'emoji at tail offset %i: captured prompt has zero unpaired surrogates',
|
||||
async (offset) => {
|
||||
// 8001 - 4000 = 4001 is the tail boundary; we test around it.
|
||||
const content = buildContentWithEmojiAt(8001, offset);
|
||||
const { client, captured } = makeCapturingClient();
|
||||
await judgeSignificance(client, makeLongTranscript(content));
|
||||
await judgeSignificance(client, makeLongTranscript(content), 'claude-haiku-4-5-20251001', SAMPLE_OPTS);
|
||||
expect(captured.userMessage).not.toBeNull();
|
||||
const result = scanForUnpairedSurrogates(captured.userMessage!);
|
||||
expect(result).toBeNull();
|
||||
},
|
||||
);
|
||||
|
||||
// ─── Sub-8000 short-content branch: no slicing, no risk ────────────
|
||||
// ─── Sub-window short-content branch: no slicing, no risk ────────────
|
||||
|
||||
test('content <= 8000 chars: no slicing applied, emoji passes through unchanged', async () => {
|
||||
test('content <= maxChars: no slicing applied, emoji passes through unchanged', async () => {
|
||||
const content = 'a'.repeat(100) + ROBOT + 'b'.repeat(100); // 202 chars total
|
||||
const { client, captured } = makeCapturingClient();
|
||||
await judgeSignificance(client, makeLongTranscript(content));
|
||||
await judgeSignificance(client, makeLongTranscript(content), 'claude-haiku-4-5-20251001', SAMPLE_OPTS);
|
||||
expect(captured.userMessage).not.toBeNull();
|
||||
expect(scanForUnpairedSurrogates(captured.userMessage!)).toBeNull();
|
||||
// Emoji's full pair must appear at least once.
|
||||
|
||||
@@ -32,8 +32,13 @@ afterEach(() => {
|
||||
|
||||
// Canned "worth processing" LLM text used by the parsed-verdict parity tests.
|
||||
// Mirrors what a well-tuned Haiku would emit for a substantive transcript.
|
||||
// Triage-v1 (#4152): the judge emits a scored verdict; worth_processing is
|
||||
// derived from `score >= DEFAULT_TRIAGE_THRESHOLD`.
|
||||
const WORTH_PROCESSING_JSON = JSON.stringify({
|
||||
worth_processing: true,
|
||||
score: 0.85,
|
||||
content_type: 'strategy',
|
||||
segments: [],
|
||||
entities: [],
|
||||
reasons: ['user reflects on portfolio framework', 'concrete strategic call'],
|
||||
});
|
||||
|
||||
|
||||
@@ -55,7 +55,7 @@ describe('dream CLI flag wiring', () => {
|
||||
});
|
||||
|
||||
test('help text documents dry-run synthesis semantics (Codex finding #8)', () => {
|
||||
expect(dreamSrc).toContain('skips the Sonnet');
|
||||
expect(dreamSrc).toContain('skips the synthesis subagents');
|
||||
expect(dreamSrc.toLowerCase()).toContain('zero llm calls');
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,658 @@
|
||||
/**
|
||||
* `gbrain dream retriage` (#4152) — PGLite-backed tests for the backlog
|
||||
* reconciliation matrix, the spend gate, and dispatch/usage errors.
|
||||
*
|
||||
* No LLM calls anywhere: every discovered transcript gets a pre-seeded
|
||||
* triage-v1 verdict (cache hits), or the test runs --dry-run (zero judge
|
||||
* calls by contract), or the spend gate aborts before judging.
|
||||
*
|
||||
* Run: bun test test/dream-retriage.test.ts
|
||||
*/
|
||||
|
||||
import { describe, test, expect, afterEach, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join, basename } from 'path';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { resetPgliteState } from './helpers/reset-pglite.ts';
|
||||
import { withEnv } from './helpers/with-env.ts';
|
||||
import { runDreamRetriage } from '../src/commands/dream-retriage.ts';
|
||||
import { runDream } from '../src/commands/dream.ts';
|
||||
import { TRIAGE_VERSION, CHARS_PER_TOKEN } from '../src/core/cycle/synthesize.ts';
|
||||
import { canonicalLookup } from '../src/core/model-pricing.ts';
|
||||
import { SPEND_CONFIRM_USD } from '../src/commands/dream-retriage-constants.ts';
|
||||
import { TIER_DEFAULTS } from '../src/core/model-config.ts';
|
||||
import { MinionQueue } from '../src/core/minions/queue.ts';
|
||||
|
||||
// Canonical shared-engine block (check-test-isolation R3/R4).
|
||||
let sharedEngine: PGLiteEngine;
|
||||
let schemaVersion: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
sharedEngine = new PGLiteEngine();
|
||||
await sharedEngine.connect({ engine: 'pglite' } as never);
|
||||
await sharedEngine.initSchema();
|
||||
// resetPgliteState truncates `config`, wiping the `version` row that
|
||||
// MinionQueue.ensureSchema checks. Capture it so beforeEach can restore.
|
||||
schemaVersion = (await sharedEngine.getConfig('version')) ?? '7';
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await sharedEngine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetPgliteState(sharedEngine);
|
||||
await sharedEngine.setConfig('version', schemaVersion);
|
||||
});
|
||||
|
||||
interface Rig {
|
||||
engine: PGLiteEngine;
|
||||
corpusDir: string;
|
||||
cleanup: () => Promise<void>;
|
||||
}
|
||||
|
||||
/** Per-test corpus dir + dream config on the SHARED engine (reset by beforeEach). */
|
||||
async function setupRig(): Promise<Rig> {
|
||||
const corpusDir = mkdtempSync(join(tmpdir(), 'gbrain-retriage-corpus-'));
|
||||
await sharedEngine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await sharedEngine.setConfig('dream.synthesize.session_corpus_dir', corpusDir);
|
||||
return {
|
||||
engine: sharedEngine,
|
||||
corpusDir,
|
||||
cleanup: async () => {
|
||||
try { rmSync(corpusDir, { recursive: true, force: true }); } catch { /* */ }
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function writeTranscript(corpusDir: string, name: string): { filePath: string; content: string; hash: string; hash16: string } {
|
||||
const content = `conversation in ${name}\n`.repeat(200);
|
||||
const filePath = join(corpusDir, name);
|
||||
writeFileSync(filePath, content);
|
||||
const hash = createHash('sha256').update(content, 'utf8').digest('hex');
|
||||
return { filePath, content, hash, hash16: hash.slice(0, 16) };
|
||||
}
|
||||
|
||||
async function seedScore(rig: Rig, filePath: string, hash: string, score: number): Promise<void> {
|
||||
await rig.engine.putDreamVerdict(filePath, hash, {
|
||||
worth_processing: score >= 0.5,
|
||||
reasons: ['seed'],
|
||||
score,
|
||||
content_type: null,
|
||||
segments: [],
|
||||
entities: [],
|
||||
model: TIER_DEFAULTS.utility,
|
||||
triage_version: TRIAGE_VERSION,
|
||||
});
|
||||
}
|
||||
|
||||
/** Seed a raw minion_jobs row (bypasses queue.add validators for full control). */
|
||||
async function seedJob(rig: Rig, opts: {
|
||||
key: string;
|
||||
queue?: string;
|
||||
status?: string;
|
||||
sourceId?: string;
|
||||
}): Promise<number> {
|
||||
const rows = await rig.engine.executeRaw<{ id: number }>(
|
||||
`INSERT INTO minion_jobs (name, queue, status, data, idempotency_key)
|
||||
VALUES ('subagent', $1, $2, jsonb_build_object('source_id', $3::text), $4)
|
||||
RETURNING id`,
|
||||
[opts.queue ?? 'dream-inline-1700000000000-deadbeef', opts.status ?? 'waiting', opts.sourceId ?? 'default', opts.key],
|
||||
);
|
||||
return rows[0].id;
|
||||
}
|
||||
|
||||
function synthKey(sourceId: string, fileBasename: string, hash16: string, chunk?: string): string {
|
||||
const base = `dream:synth-v2:${encodeURIComponent(sourceId)}:filename:${encodeURIComponent(fileBasename)}:${hash16}`;
|
||||
return chunk ? `${base}:${chunk}` : base;
|
||||
}
|
||||
|
||||
async function jobStatus(rig: Rig, id: number): Promise<string> {
|
||||
const rows = await rig.engine.executeRaw<{ status: string }>(`SELECT status FROM minion_jobs WHERE id = $1`, [id]);
|
||||
return rows[0].status;
|
||||
}
|
||||
|
||||
async function captureStdout<T>(body: () => Promise<T>): Promise<{ result: T; out: string }> {
|
||||
const chunks: string[] = [];
|
||||
const orig = console.log;
|
||||
console.log = (...args: unknown[]) => { chunks.push(args.map(String).join(' ')); };
|
||||
try {
|
||||
const result = await body();
|
||||
return { result, out: chunks.join('\n') };
|
||||
} finally {
|
||||
console.log = orig;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Hermeticity: run with ANTHROPIC_API_KEY cleared AND GBRAIN_HOME pointed at a
|
||||
* fresh tmpdir (hasAnthropicKey reads BOTH). Any cache MISS then degrades
|
||||
* per-file instead of hitting a live provider — a dev machine with a real key
|
||||
* must never spend money running this suite.
|
||||
*/
|
||||
async function withoutAnthropicKey<T>(body: () => Promise<T>): Promise<T> {
|
||||
const tmpHome = mkdtempSync(join(tmpdir(), 'gbrain-retriage-isol-'));
|
||||
try {
|
||||
return await withEnv({ ANTHROPIC_API_KEY: undefined, GBRAIN_HOME: tmpHome }, body);
|
||||
} finally {
|
||||
try { rmSync(tmpHome, { recursive: true, force: true }); } catch { /* */ }
|
||||
}
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
process.exitCode = 0;
|
||||
});
|
||||
|
||||
describe('dream retriage — dispatch + usage', () => {
|
||||
test('runDream dispatches args[0]===retriage; --help prints without engine', async () => {
|
||||
const { out } = await captureStdout(() => runDream(null, ['retriage', '--help']));
|
||||
expect(out).toContain('reconcile the synth backlog');
|
||||
expect(process.exitCode ?? 0).toBe(0);
|
||||
});
|
||||
|
||||
test('unknown flag → exit code 2', async () => {
|
||||
await runDreamRetriage(null, ['--bogus']);
|
||||
expect(process.exitCode).toBe(2);
|
||||
});
|
||||
|
||||
test('--threshold out of range → exit code 2; --cancel-unmatched without --reconcile-queue → exit code 2', async () => {
|
||||
await runDreamRetriage(null, ['--threshold', '1.5']);
|
||||
expect(process.exitCode).toBe(2);
|
||||
process.exitCode = 0;
|
||||
await runDreamRetriage(null, ['--cancel-unmatched']);
|
||||
expect(process.exitCode).toBe(2);
|
||||
});
|
||||
|
||||
test('CX2: --cancel-unmatched cannot combine with --limit → exit code 2', async () => {
|
||||
await runDreamRetriage(null, ['--reconcile-queue', '--cancel-unmatched', '--limit', '5']);
|
||||
expect(process.exitCode).toBe(2);
|
||||
});
|
||||
|
||||
test('CX3: --max-usd with an unpriced model → exit code 2, nothing judged', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('models.dream.triage', 'ollama:totally-unpriced-model');
|
||||
writeTranscript(rig.corpusDir, '2026-08-20-unpriced.txt');
|
||||
await withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--max-usd', '1', '--yes']));
|
||||
expect(process.exitCode).toBe(2);
|
||||
const rows = await rig.engine.executeRaw<{ n: number }>(`SELECT COUNT(*)::int AS n FROM dream_verdicts`);
|
||||
expect(rows[0].n).toBe(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('no engine (non-help) → exit code 1; no corpus config → exit code 1', async () => {
|
||||
await runDreamRetriage(null, ['--dry-run']);
|
||||
expect(process.exitCode).toBe(1);
|
||||
process.exitCode = 0;
|
||||
// beforeEach reset the shared engine's config, so no corpus dir is set.
|
||||
await runDreamRetriage(sharedEngine, ['--dry-run']);
|
||||
expect(process.exitCode).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('dream retriage — reconcile matrix', () => {
|
||||
test('full matrix: below-threshold cancelled; above in stale queue converted; unmatched kept; legacy untouched; completed untouched', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const below = writeTranscript(rig.corpusDir, '2026-08-01-logistics.txt');
|
||||
const above = writeTranscript(rig.corpusDir, '2026-08-02-thesis.txt');
|
||||
const unscored = writeTranscript(rig.corpusDir, '2026-08-03-new.txt');
|
||||
await seedScore(rig, below.filePath, below.hash, 0.2);
|
||||
await seedScore(rig, above.filePath, above.hash, 0.9);
|
||||
// `unscored` gets NO verdict → dry-run-style needs_triage; matched-but-unscored is kept.
|
||||
|
||||
const belowJob = await seedJob(rig, { key: synthKey('default', basename(below.filePath), below.hash16) });
|
||||
const aboveStale = await seedJob(rig, { key: synthKey('default', basename(above.filePath), above.hash16) });
|
||||
const unscoredJob = await seedJob(rig, { key: synthKey('default', basename(unscored.filePath), unscored.hash16) });
|
||||
const unmatchedJob = await seedJob(rig, { key: synthKey('default', 'deleted-file.txt', 'aaaaaaaaaaaaaaaa') });
|
||||
const legacyJob = await seedJob(rig, { key: 'dream:synth:/old/path.txt:0123456789abcdef' });
|
||||
const completedJob = await seedJob(rig, { key: synthKey('default', basename(below.filePath), below.hash16, 'c0of2'), status: 'completed' });
|
||||
|
||||
// All verdicts are cache hits → zero judge calls even without --dry-run.
|
||||
const { out } = await captureStdout(() => withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--reconcile-queue', '--json'])));
|
||||
const summary = JSON.parse(out) as { queue: Record<string, unknown>; retriaged: number };
|
||||
|
||||
expect(summary.retriaged).toBe(0); // hits only — the third file degrades (no provider), never judged
|
||||
expect(await jobStatus(rig, belowJob)).toBe('cancelled');
|
||||
expect(await jobStatus(rig, aboveStale)).toBe('cancelled'); // converted_for_resubmit
|
||||
expect(await jobStatus(rig, unscoredJob)).toBe('waiting');
|
||||
expect(await jobStatus(rig, unmatchedJob)).toBe('waiting');
|
||||
expect(await jobStatus(rig, legacyJob)).toBe('waiting'); // legacy grammar → unmatched, untouched
|
||||
expect(await jobStatus(rig, completedJob)).toBe('completed'); // terminal rows never selected
|
||||
|
||||
expect(summary.queue.cancelled).toBe(1);
|
||||
expect(summary.queue.converted_for_resubmit).toBe(1);
|
||||
expect(summary.queue.kept_unscored).toBe(1);
|
||||
// deleted-file only: the legacy dream:synth: key is excluded at the SQL
|
||||
// LIKE 'dream:synth-v2:%' filter — it never even becomes a candidate.
|
||||
expect(summary.queue.unmatched).toBe(1);
|
||||
expect(summary.queue.candidates).toBe(4); // completed row is status-excluded as well
|
||||
expect(process.exitCode ?? 0).toBe(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('above-threshold in a LIVE (non dream-inline) queue is kept', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const above = writeTranscript(rig.corpusDir, '2026-08-04-keeper.txt');
|
||||
await seedScore(rig, above.filePath, above.hash, 0.8);
|
||||
const liveJob = await seedJob(rig, {
|
||||
key: synthKey('default', basename(above.filePath), above.hash16),
|
||||
queue: 'default',
|
||||
});
|
||||
const { out } = await captureStdout(() => withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--reconcile-queue', '--json'])));
|
||||
const summary = JSON.parse(out) as { queue: { kept_above_threshold: number } };
|
||||
expect(await jobStatus(rig, liveJob)).toBe('waiting');
|
||||
expect(summary.queue.kept_above_threshold).toBe(1);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('C9: key-source vs data.source_id mismatch is skipped, never cancelled', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const below = writeTranscript(rig.corpusDir, '2026-08-05-mismatch.txt');
|
||||
await seedScore(rig, below.filePath, below.hash, 0.1);
|
||||
const mismatched = await seedJob(rig, {
|
||||
key: synthKey('other-source', basename(below.filePath), below.hash16),
|
||||
sourceId: 'default', // payload disagrees with the key's encoded source
|
||||
});
|
||||
const { out } = await captureStdout(() => withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--reconcile-queue', '--json'])));
|
||||
const summary = JSON.parse(out) as { queue: { source_mismatch: number } };
|
||||
expect(await jobStatus(rig, mismatched)).toBe('waiting');
|
||||
expect(summary.queue.source_mismatch).toBe(1);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('--source scopes cancels: other sources counted as other_source, untouched', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const below = writeTranscript(rig.corpusDir, '2026-08-06-scoped.txt');
|
||||
await seedScore(rig, below.filePath, below.hash, 0.1);
|
||||
const mine = await seedJob(rig, { key: synthKey('work', basename(below.filePath), below.hash16), sourceId: 'work' });
|
||||
const theirs = await seedJob(rig, { key: synthKey('personal', basename(below.filePath), below.hash16), sourceId: 'personal' });
|
||||
const { out } = await captureStdout(() =>
|
||||
withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--reconcile-queue', '--source', 'work', '--json'])));
|
||||
const summary = JSON.parse(out) as { queue: { cancelled: number; other_source: number } };
|
||||
expect(await jobStatus(rig, mine)).toBe('cancelled');
|
||||
expect(await jobStatus(rig, theirs)).toBe('waiting');
|
||||
expect(summary.queue.cancelled).toBe(1);
|
||||
expect(summary.queue.other_source).toBe(1);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('--cancel-unmatched cancels jobs whose file no longer exists (corpus still reachable)', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
// CX2 guard requires a non-empty discovery result — seed one real file
|
||||
// so "the corpus is reachable" and only the ghost job is unmatched.
|
||||
const real = writeTranscript(rig.corpusDir, '2026-08-25-still-here.txt');
|
||||
await seedScore(rig, real.filePath, real.hash, 0.9);
|
||||
const gone = await seedJob(rig, { key: synthKey('default', 'vanished.txt', 'bbbbbbbbbbbbbbbb') });
|
||||
const { out } = await captureStdout(() =>
|
||||
withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--reconcile-queue', '--cancel-unmatched', '--json'])));
|
||||
const summary = JSON.parse(out) as { queue: { unmatched_cancelled: number } };
|
||||
expect(await jobStatus(rig, gone)).toBe('cancelled');
|
||||
expect(summary.queue.unmatched_cancelled).toBe(1);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('--dry-run: reports would-cancel counts, performs ZERO cancels and zero judge calls', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const below = writeTranscript(rig.corpusDir, '2026-08-07-dry.txt');
|
||||
const fresh = writeTranscript(rig.corpusDir, '2026-08-08-fresh.txt'); // no verdict → needs_triage
|
||||
await seedScore(rig, below.filePath, below.hash, 0.1);
|
||||
const belowJob = await seedJob(rig, { key: synthKey('default', basename(below.filePath), below.hash16) });
|
||||
const freshJob = await seedJob(rig, { key: synthKey('default', basename(fresh.filePath), fresh.hash16) });
|
||||
const { out } = await captureStdout(() =>
|
||||
withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--reconcile-queue', '--dry-run', '--json'])));
|
||||
const summary = JSON.parse(out) as { retriaged: number; needs_triage: number; queue: { cancelled: number } };
|
||||
expect(summary.retriaged).toBe(0); // dry-run judges nothing
|
||||
expect(summary.needs_triage).toBe(1);
|
||||
expect(summary.queue.cancelled).toBe(1); // would-cancel
|
||||
expect(await jobStatus(rig, belowJob)).toBe('waiting'); // ...but nothing actually cancelled
|
||||
expect(await jobStatus(rig, freshJob)).toBe('waiting');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('cancelled row releases its idempotency slot: a later add with the same key creates a fresh row', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const below = writeTranscript(rig.corpusDir, '2026-08-09-resubmit.txt');
|
||||
await seedScore(rig, below.filePath, below.hash, 0.1);
|
||||
const key = synthKey('default', basename(below.filePath), below.hash16);
|
||||
const oldId = await seedJob(rig, { key });
|
||||
await captureStdout(() => withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--reconcile-queue', '--json'])));
|
||||
expect(await jobStatus(rig, oldId)).toBe('cancelled');
|
||||
// Threshold lowered later → the same key must be re-submittable.
|
||||
const queue = new MinionQueue(rig.engine);
|
||||
const fresh = await queue.add('subagent', {
|
||||
prompt: 'x',
|
||||
model: 'anthropic:claude-sonnet-4-6',
|
||||
max_turns: 1,
|
||||
}, { idempotency_key: key }, { allowProtectedSubmit: true });
|
||||
expect(fresh.id).not.toBe(oldId);
|
||||
expect(fresh.status).toBe('waiting');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('SR-P1: --max-usd + --audit-rejects with an unpriced SYNTHESIS model → exit 2', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
// Triage model priced; synthesis (audit) model unpriced — the audit's
|
||||
// budget share would be silently un-metered without the guard.
|
||||
await rig.engine.setConfig('models.dream.synthesize', 'ollama:unpriced-frontier');
|
||||
writeTranscript(rig.corpusDir, '2026-08-26-audit-unpriced.txt');
|
||||
await withoutAnthropicKey(() =>
|
||||
runDreamRetriage(rig.engine, ['--max-usd', '1', '--audit-rejects', '2', '--yes']));
|
||||
expect(process.exitCode).toBe(2);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('SR-P2: a DELAYED job in a provably-dead inline queue converts for resubmit', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const above = writeTranscript(rig.corpusDir, '2026-08-27-delayed.txt');
|
||||
await seedScore(rig, above.filePath, above.hash, 0.9);
|
||||
const delayedJob = await seedJob(rig, {
|
||||
key: synthKey('default', basename(above.filePath), above.hash16),
|
||||
status: 'delayed',
|
||||
});
|
||||
const { out } = await captureStdout(() => withoutAnthropicKey(() =>
|
||||
runDreamRetriage(rig.engine, ['--reconcile-queue', '--json'])));
|
||||
const summary = JSON.parse(out) as { queue: { converted_for_resubmit: number } };
|
||||
expect(await jobStatus(rig, delayedJob)).toBe('cancelled');
|
||||
expect(summary.queue.converted_for_resubmit).toBe(1);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('SR3-P1: dream --reconcile-queue without the retriage positional fails loud (exit code 2)', async () => {
|
||||
// The flag registry unions retriage flags into `dream`, so the strict
|
||||
// pre-dispatch validator accepts them — the guard in runDream must reject
|
||||
// instead of silently running the full paid maintenance cycle.
|
||||
for (const stray of ['--reconcile-queue', '--cancel-unmatched', '--audit-rejects']) {
|
||||
process.exitCode = 0;
|
||||
const result = await runDream(null, [stray]);
|
||||
expect(result).toBeUndefined(); // no cycle ran
|
||||
expect(process.exitCode).toBe(2);
|
||||
}
|
||||
});
|
||||
|
||||
test('SR3-P2: gbrain models reports models.dream.triage as the effective triage route', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('models.dream.triage', 'anthropic:claude-sonnet-4-6');
|
||||
const { runModels } = await import('../src/commands/models.ts');
|
||||
// runModels writes through process.stdout.write; capture that stream.
|
||||
const chunks: string[] = [];
|
||||
const orig = process.stdout.write.bind(process.stdout);
|
||||
(process.stdout as unknown as { write: (c: unknown) => boolean }).write = (c: unknown) => {
|
||||
chunks.push(String(c));
|
||||
return true;
|
||||
};
|
||||
try {
|
||||
await captureStdout(() => runModels(rig.engine as never, ['--json']));
|
||||
} finally {
|
||||
(process.stdout as unknown as { write: typeof orig }).write = orig;
|
||||
}
|
||||
const raw = chunks.join('');
|
||||
const jsonStart = raw.indexOf('{');
|
||||
const report = JSON.parse(raw.slice(jsonStart)) as { per_task: Array<{ key: string; resolved: string; source: string }> };
|
||||
const row = report.per_task.find(r => r.key === 'models.dream.synthesize_verdict')!;
|
||||
expect(row.resolved).toBe('anthropic:claude-sonnet-4-6');
|
||||
expect(row.source).toBe('config: models.dream.triage');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('SR2-P1: unpriced TRIAGE model + priced audit still confirms on the audit dollars', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
// Triage unpriced (file-count gate alone would NOT trigger for 1 file),
|
||||
// synthesis model priced by default — a large --audit-rejects must gate
|
||||
// on its own estimated dollars.
|
||||
await rig.engine.setConfig('models.dream.triage', 'ollama:totally-unpriced-model');
|
||||
const reject = writeTranscript(rig.corpusDir, '2026-08-30-audit-gate.txt');
|
||||
// Seed a below-threshold verdict UNDER THE UNPRICED MODEL so it is a
|
||||
// cache hit (missCount 0) and the audit is the only spend.
|
||||
await rig.engine.putDreamVerdict(reject.filePath, reject.hash, {
|
||||
worth_processing: false, reasons: ['seed'], score: 0.1, content_type: null,
|
||||
segments: [], entities: [], model: 'ollama:totally-unpriced-model', triage_version: TRIAGE_VERSION,
|
||||
});
|
||||
await withoutAnthropicKey(() =>
|
||||
runDreamRetriage(rig.engine, ['--audit-rejects', '100000', '--json']));
|
||||
expect(process.exitCode).toBe(2); // gate fired without --yes
|
||||
const rows = await rig.engine.executeRaw<{ n: number }>(
|
||||
`SELECT COUNT(*)::int AS n FROM dream_verdicts WHERE model <> 'ollama:totally-unpriced-model'`);
|
||||
expect(rows[0].n).toBe(0); // nothing audited
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('SR2-P2: a live lock for ANOTHER source does not suppress this source\'s conversions', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const above = writeTranscript(rig.corpusDir, '2026-08-31-other-lock.txt');
|
||||
await seedScore(rig, above.filePath, above.hash, 0.9);
|
||||
const job = await seedJob(rig, { key: synthKey('default', basename(above.filePath), above.hash16) });
|
||||
await rig.engine.executeRaw(
|
||||
`INSERT INTO gbrain_cycle_locks (id, holder_pid, ttl_expires_at)
|
||||
VALUES ('gbrain-cycle:some-other-source', 12345, NOW() + interval '10 minutes')`,
|
||||
);
|
||||
const { out } = await captureStdout(() => withoutAnthropicKey(() =>
|
||||
runDreamRetriage(rig.engine, ['--reconcile-queue', '--json'])));
|
||||
const summary = JSON.parse(out) as { queue: { converted_for_resubmit: number; kept_live_queue: number } };
|
||||
expect(await jobStatus(rig, job)).toBe('cancelled'); // converted despite the foreign lock
|
||||
expect(summary.queue.converted_for_resubmit).toBe(1);
|
||||
expect(summary.queue.kept_live_queue).toBe(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('SR-P2: a live cycle lock marks even OLD inline queues possibly-live (no conversions)', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const above = writeTranscript(rig.corpusDir, '2026-08-28-locked.txt');
|
||||
await seedScore(rig, above.filePath, above.hash, 0.9);
|
||||
const job = await seedJob(rig, { key: synthKey('default', basename(above.filePath), above.hash16) });
|
||||
// Simulate a running cycle: an unexpired cycle-lock row.
|
||||
await rig.engine.executeRaw(
|
||||
`INSERT INTO gbrain_cycle_locks (id, holder_pid, ttl_expires_at)
|
||||
VALUES ('gbrain-cycle:default', 12345, NOW() + interval '10 minutes')`,
|
||||
);
|
||||
const { out } = await captureStdout(() => withoutAnthropicKey(() =>
|
||||
runDreamRetriage(rig.engine, ['--reconcile-queue', '--json'])));
|
||||
const summary = JSON.parse(out) as { queue: { kept_live_queue: number; converted_for_resubmit: number } };
|
||||
expect(await jobStatus(rig, job)).toBe('waiting');
|
||||
expect(summary.queue.kept_live_queue).toBe(1);
|
||||
expect(summary.queue.converted_for_resubmit).toBe(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('SR-P2: dry-run counts would-cancel unmatched rows instead of understating the preview', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const real = writeTranscript(rig.corpusDir, '2026-08-29-anchor.txt');
|
||||
await seedScore(rig, real.filePath, real.hash, 0.9);
|
||||
const ghost = await seedJob(rig, { key: synthKey('default', 'ghost-preview.txt', 'dddddddddddddddd') });
|
||||
const { out } = await captureStdout(() => withoutAnthropicKey(() =>
|
||||
runDreamRetriage(rig.engine, ['--reconcile-queue', '--cancel-unmatched', '--dry-run', '--json'])));
|
||||
const summary = JSON.parse(out) as { queue: { unmatched_cancelled: number } };
|
||||
expect(summary.queue.unmatched_cancelled).toBe(1); // would cancel
|
||||
expect(await jobStatus(rig, ghost)).toBe('waiting'); // but did not
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('CX1: rows in a YOUNG dream-inline-* queue are kept as possibly-live, never cancelled', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const below = writeTranscript(rig.corpusDir, '2026-08-21-live.txt');
|
||||
await seedScore(rig, below.filePath, below.hash, 0.1); // below threshold — would cancel in a dead queue
|
||||
const liveQueue = `dream-inline-${Date.now()}-abcdef01`; // younger than the 1h grace
|
||||
const liveJob = await seedJob(rig, {
|
||||
key: synthKey('default', basename(below.filePath), below.hash16),
|
||||
queue: liveQueue,
|
||||
});
|
||||
const { out } = await captureStdout(() => withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--reconcile-queue', '--json'])));
|
||||
const summary = JSON.parse(out) as { queue: { kept_live_queue: number; cancelled: number } };
|
||||
expect(await jobStatus(rig, liveJob)).toBe('waiting');
|
||||
expect(summary.queue.kept_live_queue).toBe(1);
|
||||
expect(summary.queue.cancelled).toBe(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('CX2: zero discovered transcripts + queued jobs → --cancel-unmatched refused with exit 2', async () => {
|
||||
const rig = await setupRig(); // corpus dir exists but is EMPTY
|
||||
try {
|
||||
const job = await seedJob(rig, { key: synthKey('default', 'ghost.txt', 'cccccccccccccccc') });
|
||||
await withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--reconcile-queue', '--cancel-unmatched', '--json', '--yes']));
|
||||
expect(process.exitCode).toBe(2);
|
||||
expect(await jobStatus(rig, job)).toBe('waiting'); // retry frontier preserved
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('--audit-rejects: frontier second opinion via stubbed transport, disagreement rate reported', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const { __setChatTransportForTests, resetGateway } = await import('../src/core/ai/gateway.ts');
|
||||
const rejectA = writeTranscript(rig.corpusDir, '2026-08-22-reject-a.txt');
|
||||
const rejectB = writeTranscript(rig.corpusDir, '2026-08-23-reject-b.txt');
|
||||
await seedScore(rig, rejectA.filePath, rejectA.hash, 0.1);
|
||||
await seedScore(rig, rejectB.filePath, rejectB.hash, 0.2);
|
||||
// Frontier stub: always scores HIGH → disagrees with both rejects.
|
||||
__setChatTransportForTests(async () => ({
|
||||
text: '{"score": 0.9, "content_type": "idea", "segments": [], "entities": [], "reasons": ["frontier disagrees"]}',
|
||||
blocks: [],
|
||||
stopReason: 'end',
|
||||
usage: { input_tokens: 10, output_tokens: 50, cache_read_tokens: 0, cache_creation_tokens: 0 },
|
||||
model: 'test:stub',
|
||||
providerId: 'test',
|
||||
}));
|
||||
try {
|
||||
// Fake key so makeJudgeClient constructs; the transport stub prevents network.
|
||||
const { out } = await captureStdout(() => withEnv({ ANTHROPIC_API_KEY: 'sk-test-audit' }, () =>
|
||||
runDreamRetriage(rig.engine, ['--audit-rejects', '2', '--yes', '--json'])));
|
||||
const summary = JSON.parse(out) as { audit: { sampled: number; disagreements: number; disagreement_rate: number | null } };
|
||||
expect(summary.audit.sampled).toBe(2);
|
||||
expect(summary.audit.disagreements).toBe(2);
|
||||
expect(summary.audit.disagreement_rate).toBe(1);
|
||||
} finally {
|
||||
__setChatTransportForTests(null);
|
||||
resetGateway();
|
||||
}
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('--audit-rejects under --dry-run is a loud no-op (audit null, notice printed)', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const below = writeTranscript(rig.corpusDir, '2026-08-24-noop.txt');
|
||||
await seedScore(rig, below.filePath, below.hash, 0.1);
|
||||
const { out } = await captureStdout(() => withoutAnthropicKey(() =>
|
||||
runDreamRetriage(rig.engine, ['--audit-rejects', '1', '--dry-run', '--json'])));
|
||||
const summary = JSON.parse(out) as { audit: unknown };
|
||||
expect(summary.audit).toBeNull();
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('--threshold override wins over config for the gate', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const mid = writeTranscript(rig.corpusDir, '2026-08-10-mid.txt');
|
||||
await seedScore(rig, mid.filePath, mid.hash, 0.6);
|
||||
const job = await seedJob(rig, { key: synthKey('default', basename(mid.filePath), mid.hash16) });
|
||||
// 0.6 passes the default 0.5 gate — but a 0.7 override cancels it.
|
||||
await captureStdout(() => withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--reconcile-queue', '--threshold', '0.7', '--json'])));
|
||||
expect(await jobStatus(rig, job)).toBe('cancelled');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
describe('dream retriage — spend gate (C12)', () => {
|
||||
/**
|
||||
* Force the confirmation gate robustly: derive the sample window from LIVE
|
||||
* canonical pricing so a single un-cached file's estimate exceeds
|
||||
* SPEND_CONFIRM_USD regardless of future price updates in model-pricing.ts
|
||||
* (a hardcoded char count silently flips the gate when prices drift).
|
||||
*/
|
||||
async function forceSpendGate(rig: Rig): Promise<void> {
|
||||
const model = 'anthropic:claude-fable-5';
|
||||
const price = canonicalLookup(model)!;
|
||||
const charsForGate = Math.ceil(((SPEND_CONFIRM_USD * 1.5) / price.input) * 1_000_000 * CHARS_PER_TOKEN);
|
||||
await rig.engine.setConfig('models.dream.triage', model);
|
||||
await rig.engine.setConfig('dream.triage.max_chars', String(charsForGate));
|
||||
}
|
||||
|
||||
test('non-interactive gate: un-triaged files + --json without --yes → exit 2, nothing judged', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
// Two un-cached files × a priced frontier model × a pricing-derived
|
||||
// sample window pushes the estimate past the gate.
|
||||
await forceSpendGate(rig);
|
||||
writeTranscript(rig.corpusDir, '2026-08-11-a.txt');
|
||||
writeTranscript(rig.corpusDir, '2026-08-12-b.txt');
|
||||
await withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--json']));
|
||||
expect(process.exitCode).toBe(2);
|
||||
// Nothing was judged or cached.
|
||||
const rows = await rig.engine.executeRaw<{ n: number }>(`SELECT COUNT(*)::int AS n FROM dream_verdicts`);
|
||||
expect(rows[0].n).toBe(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('--yes skips the gate (files degrade with no provider rather than aborting)', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await forceSpendGate(rig);
|
||||
writeTranscript(rig.corpusDir, '2026-08-13-c.txt');
|
||||
// withoutAnthropicKey → judge client null → per-file degrade, no spend.
|
||||
const { out } = await captureStdout(() => withoutAnthropicKey(() => runDreamRetriage(rig.engine, ['--json', '--yes'])));
|
||||
const summary = JSON.parse(out) as { discovered: number; retriaged: number };
|
||||
expect(summary.discovered).toBe(1);
|
||||
expect(summary.retriaged).toBe(0); // degraded, not judged
|
||||
expect(process.exitCode ?? 0).toBe(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
@@ -23,7 +23,8 @@ import { mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { runPhaseSynthesize } from '../../src/core/cycle/synthesize.ts';
|
||||
import { runPhaseSynthesize, TRIAGE_VERSION } from '../../src/core/cycle/synthesize.ts';
|
||||
import { TIER_DEFAULTS } from '../../src/core/model-config.ts';
|
||||
|
||||
interface TestRig {
|
||||
engine: PGLiteEngine;
|
||||
@@ -68,16 +69,25 @@ async function withoutAnthropicKey<T>(body: () => Promise<T>): Promise<T> {
|
||||
* 35 minutes. Cancelling moves them to a terminal state so the phase
|
||||
* returns and we can inspect submission shape.
|
||||
*/
|
||||
async function withSubagentAutoCancel<T>(engine: PGLiteEngine, body: () => Promise<T>): Promise<T> {
|
||||
async function withSubagentAutoCancel<T>(
|
||||
engine: PGLiteEngine,
|
||||
body: () => Promise<T>,
|
||||
opts: { excludeQueue?: string } = {},
|
||||
): Promise<T> {
|
||||
let stopped = false;
|
||||
const loop = (async () => {
|
||||
while (!stopped) {
|
||||
await new Promise(r => setTimeout(r, 50));
|
||||
try {
|
||||
// excludeQueue: rows a test seeded deliberately (e.g. the C1
|
||||
// stranded-row fixture) must be cancelled by the CODE UNDER TEST,
|
||||
// not this poller — otherwise the assertion is vacuous/racy.
|
||||
await engine.executeRaw(
|
||||
`UPDATE minion_jobs
|
||||
SET status = 'cancelled', finished_at = now()
|
||||
WHERE name = 'subagent' AND status IN ('waiting', 'active')`,
|
||||
WHERE name = 'subagent' AND status IN ('waiting', 'active')
|
||||
AND ($1::text IS NULL OR queue <> $1)`,
|
||||
[opts.excludeQueue ?? null],
|
||||
);
|
||||
} catch {
|
||||
// Race against shutdown is fine; ignore.
|
||||
@@ -100,9 +110,17 @@ async function withSubagentAutoCancel<T>(engine: PGLiteEngine, body: () => Promi
|
||||
async function seedVerdict(engine: PGLiteEngine, filePath: string, content: string): Promise<string> {
|
||||
const { createHash } = await import('node:crypto');
|
||||
const contentHash = createHash('sha256').update(content, 'utf8').digest('hex');
|
||||
// Triage-v1 cache validity requires score + matching (model, triage_version);
|
||||
// TIER_DEFAULTS.utility is what loadSynthConfig resolves in a bare test env.
|
||||
await engine.putDreamVerdict(filePath, contentHash, {
|
||||
worth_processing: true,
|
||||
reasons: ['seeded for chunking E2E test'],
|
||||
score: 0.9,
|
||||
content_type: null,
|
||||
segments: [],
|
||||
entities: [],
|
||||
model: TIER_DEFAULTS.utility,
|
||||
triage_version: TRIAGE_VERSION,
|
||||
});
|
||||
return contentHash;
|
||||
}
|
||||
@@ -436,3 +454,189 @@ describe('E2E synthesize chunking — fan-out shape', () => {
|
||||
function escapeRe(s: string): string {
|
||||
return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
}
|
||||
|
||||
describe('E2E synthesize — max_turns (#4152 REGRESSION pin) + triage map injection', () => {
|
||||
// IRON-RULE REGRESSION TEST: the default turn budget dropped 30 → 16 with
|
||||
// the two-stage cascade. Pin BOTH the new default AND the config path that
|
||||
// restores the old behavior.
|
||||
test('submitted subagent jobs carry max_turns=16 by default; dream.synthesize.max_turns=30 restores 30', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
// Two back-to-back runs in this test — disable the cooldown so the
|
||||
// second run isn't skipped (configured 0 is honored).
|
||||
await rig.engine.setConfig('dream.synthesize.cooldown_hours', '0');
|
||||
const basename = '2026-08-14-turns.txt';
|
||||
const filePath = corpusPath(rig.corpusDir, basename);
|
||||
const content = 'a substantive conversation line\n'.repeat(200);
|
||||
writeFileSync(filePath, content);
|
||||
await seedVerdict(rig.engine, filePath, content);
|
||||
|
||||
await withoutAnthropicKey(async () => {
|
||||
await withSubagentAutoCancel(rig.engine, async () => {
|
||||
await runPhaseSynthesize(rig.engine, { brainDir: rig.brainDir, dryRun: false });
|
||||
});
|
||||
});
|
||||
let rows = await rig.engine.executeRaw<{ data: { max_turns?: number; prompt?: string } }>(
|
||||
`SELECT data FROM minion_jobs WHERE name = 'subagent' ORDER BY id DESC LIMIT 1`,
|
||||
);
|
||||
expect(rows[0].data.max_turns).toBe(16);
|
||||
|
||||
// Restore path: config override back to the pre-#4152 value. Cancelled
|
||||
// rows release the idempotency key, so a re-run resubmits fresh.
|
||||
await rig.engine.setConfig('dream.synthesize.max_turns', '30');
|
||||
await withoutAnthropicKey(async () => {
|
||||
await withSubagentAutoCancel(rig.engine, async () => {
|
||||
await runPhaseSynthesize(rig.engine, { brainDir: rig.brainDir, dryRun: false });
|
||||
});
|
||||
});
|
||||
rows = await rig.engine.executeRaw<{ data: { max_turns?: number } }>(
|
||||
`SELECT data FROM minion_jobs WHERE name = 'subagent' ORDER BY id DESC LIMIT 1`,
|
||||
);
|
||||
expect(rows[0].data.max_turns).toBe(30);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test('TRIAGE MAP block rides in the synthesis prompt when the verdict carries segments', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
const basename = '2026-08-15-mapped.txt';
|
||||
const filePath = corpusPath(rig.corpusDir, basename);
|
||||
const content = 'the future of memory is a database that dreams\n'.repeat(100);
|
||||
writeFileSync(filePath, content);
|
||||
const { createHash } = await import('node:crypto');
|
||||
const contentHash = createHash('sha256').update(content, 'utf8').digest('hex');
|
||||
await rig.engine.putDreamVerdict(filePath, contentHash, {
|
||||
worth_processing: true,
|
||||
reasons: ['seeded'],
|
||||
score: 0.91,
|
||||
content_type: 'idea',
|
||||
segments: [{ quote: 'the future of memory is a database that dreams', note: 'thesis' }],
|
||||
entities: ['acme-example'],
|
||||
model: TIER_DEFAULTS.utility,
|
||||
triage_version: TRIAGE_VERSION,
|
||||
});
|
||||
await withoutAnthropicKey(async () => {
|
||||
await withSubagentAutoCancel(rig.engine, async () => {
|
||||
await runPhaseSynthesize(rig.engine, { brainDir: rig.brainDir, dryRun: false });
|
||||
});
|
||||
});
|
||||
const rows = await rig.engine.executeRaw<{ data: { prompt?: string } }>(
|
||||
`SELECT data FROM minion_jobs WHERE name = 'subagent' ORDER BY id DESC LIMIT 1`,
|
||||
);
|
||||
const prompt = rows[0].data.prompt ?? '';
|
||||
expect(prompt).toContain('TRIAGE MAP');
|
||||
expect(prompt).toContain('signal score: 0.91');
|
||||
expect(prompt).toContain('content type: idea');
|
||||
expect(prompt).toContain('acme-example');
|
||||
expect(prompt).toContain('database that dreams');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
describe('E2E synthesize — fan-out self-heal for stranded coalesced rows (#4152 C1)', () => {
|
||||
test('a waiting row in a FOREIGN dream-inline-* queue is cancelled + re-added into the live run', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
const basename = '2026-08-16-stranded.txt';
|
||||
const filePath = corpusPath(rig.corpusDir, basename);
|
||||
const content = 'stranded conversation line\n'.repeat(200);
|
||||
writeFileSync(filePath, content);
|
||||
const contentHash = await seedVerdict(rig.engine, filePath, content);
|
||||
const key = `dream:synth-v2:default:filename:${encodeURIComponent(basename)}:${contentHash.slice(0, 16)}`;
|
||||
|
||||
// Simulate a previously-killed run: its child sits waiting in a dead
|
||||
// per-run private queue no worker will ever claim. The queue timestamp
|
||||
// (Nov 2023) is far past the CX1 liveness grace, so the self-heal may
|
||||
// legally cancel it.
|
||||
const stranded = await rig.engine.executeRaw<{ id: number }>(
|
||||
`INSERT INTO minion_jobs (name, queue, status, data, idempotency_key)
|
||||
VALUES ('subagent', 'dream-inline-1700000000000-deadbeef', 'waiting', '{}'::jsonb, $1)
|
||||
RETURNING id`,
|
||||
[key],
|
||||
);
|
||||
const strandedId = stranded[0].id;
|
||||
|
||||
await withoutAnthropicKey(async () => {
|
||||
// Testing-specialist race fix: the auto-cancel poller must NOT touch
|
||||
// the seeded stranded row — if it cancels it first, queue.add's own
|
||||
// dead/cancelled key-release path produces the asserted end-state
|
||||
// WITHOUT the self-heal branch ever running (vacuous pass), and a
|
||||
// poller firing between coalesce and cancelJob hard-fails the test.
|
||||
await withSubagentAutoCancel(rig.engine, async () => {
|
||||
const result = await runPhaseSynthesize(rig.engine, { brainDir: rig.brainDir, dryRun: false });
|
||||
expect(result.status).toBe('ok');
|
||||
const details = result.details as { children_submitted: number };
|
||||
expect(details.children_submitted).toBe(1);
|
||||
}, { excludeQueue: 'dream-inline-1700000000000-deadbeef' });
|
||||
});
|
||||
|
||||
// The stranded row was cancelled (key released) and a FRESH row with the
|
||||
// same key was created in the live run's queue.
|
||||
const rows = await rig.engine.executeRaw<{ id: number; status: string; queue: string; idempotency_key: string | null }>(
|
||||
`SELECT id, status, queue, idempotency_key FROM minion_jobs WHERE name = 'subagent' ORDER BY id`,
|
||||
);
|
||||
const old = rows.find(r => r.id === strandedId)!;
|
||||
expect(old.status).toBe('cancelled');
|
||||
expect(old.idempotency_key).toBeNull(); // slot released on re-add
|
||||
const fresh = rows.find(r => r.id !== strandedId)!;
|
||||
expect(fresh.idempotency_key).toBe(key);
|
||||
expect(fresh.queue).not.toBe('dream-inline-1700000000000-deadbeef');
|
||||
expect(fresh.queue.startsWith('dream-inline-')).toBe(true);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('CX1 guard: a coalesced row in a YOUNG (possibly-live) dream-inline queue is NOT healed', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
// Keep the phase's wait short: the un-healed foreign row never goes
|
||||
// terminal (the poller excludes it), so waitForCompletion must time out
|
||||
// fast instead of the 35-min default.
|
||||
await rig.engine.setConfig('dream.synthesize.subagent_wait_timeout_ms', '2000');
|
||||
const basename = '2026-08-17-live-queue.txt';
|
||||
const filePath = corpusPath(rig.corpusDir, basename);
|
||||
const content = 'possibly live conversation line\n'.repeat(200);
|
||||
writeFileSync(filePath, content);
|
||||
const contentHash = await seedVerdict(rig.engine, filePath, content);
|
||||
const key = `dream:synth-v2:default:filename:${encodeURIComponent(basename)}:${contentHash.slice(0, 16)}`;
|
||||
// A FRESH foreign queue — inside the liveness grace, may belong to a
|
||||
// concurrently running cycle. The self-heal must leave it alone.
|
||||
const liveQueue = `dream-inline-${Date.now()}-0abc1234`;
|
||||
const seeded = await rig.engine.executeRaw<{ id: number }>(
|
||||
`INSERT INTO minion_jobs (name, queue, status, data, idempotency_key)
|
||||
VALUES ('subagent', $2, 'waiting', '{}'::jsonb, $1)
|
||||
RETURNING id`,
|
||||
[key, liveQueue],
|
||||
);
|
||||
await withoutAnthropicKey(async () => {
|
||||
await withSubagentAutoCancel(rig.engine, async () => {
|
||||
const result = await runPhaseSynthesize(rig.engine, { brainDir: rig.brainDir, dryRun: false });
|
||||
expect(result.status).toBe('ok'); // child outcome is 'timeout', phase still completes
|
||||
}, { excludeQueue: liveQueue });
|
||||
});
|
||||
const rows = await rig.engine.executeRaw<{ id: number; status: string; queue: string }>(
|
||||
`SELECT id, status, queue FROM minion_jobs WHERE name = 'subagent'`,
|
||||
);
|
||||
// Exactly the seeded row exists, untouched: no cancel, no re-add.
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].id).toBe(seeded[0].id);
|
||||
expect(rows[0].status).toBe('waiting');
|
||||
expect(rows[0].queue).toBe(liveQueue);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
@@ -18,7 +18,8 @@ import { mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { runPhaseSynthesize, renderPageToMarkdown, __testing as synthTesting } from '../../src/core/cycle/synthesize.ts';
|
||||
import { runPhaseSynthesize, renderPageToMarkdown, TRIAGE_VERSION, __testing as synthTesting } from '../../src/core/cycle/synthesize.ts';
|
||||
import { TIER_DEFAULTS } from '../../src/core/model-config.ts';
|
||||
|
||||
interface TestRig {
|
||||
engine: PGLiteEngine;
|
||||
@@ -201,6 +202,46 @@ describe('E2E synthesize — no API key skip path', () => {
|
||||
// string was 'no ANTHROPIC_API_KEY for significance judge'; post-
|
||||
// rework is 'no configured provider for verdict model: <model>'.
|
||||
expect(verdicts[0].reasons[0]).toMatch(/no configured provider for verdict model/);
|
||||
// CX7: a provider outage is DEGRADED, never "below threshold" — an
|
||||
// all-outage run must say so in both the counters and the headline.
|
||||
const triage = (result.details as { triage: { degraded: number; below_threshold: number } }).triage;
|
||||
expect(triage.degraded).toBe(1);
|
||||
expect(triage.below_threshold).toBe(0);
|
||||
expect(result.summary).toContain('triage degraded');
|
||||
expect(result.summary).not.toContain('below triage threshold');
|
||||
});
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('3A: a time-boxed cold pass labels deferred files "not yet triaged" in the headline', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
// A 1ms budget + 25 uncached files: the budget check runs after each
|
||||
// per-file cache lookup, and 25 PGLite roundtrips take well over 1ms,
|
||||
// so at least the tail of the corpus is guaranteed to defer (exact
|
||||
// count depends on wall-clock — assert >= 1, not equality).
|
||||
await rig.engine.setConfig('dream.triage.max_ms', '1');
|
||||
for (let i = 0; i < 25; i++) {
|
||||
writeFileSync(
|
||||
join(rig.corpusDir, `2026-04-26-cold-${String(i).padStart(2, '0')}.txt`),
|
||||
`an untriaged conversation ${i}\n`.repeat(200),
|
||||
);
|
||||
}
|
||||
await withoutAnthropicKey(async () => {
|
||||
const result = await runPhaseSynthesize(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: true,
|
||||
});
|
||||
expect(result.status).toBe('ok');
|
||||
const triage = (result.details as { triage: { deferred: number; degraded: number } }).triage;
|
||||
expect(triage.deferred).toBeGreaterThanOrEqual(1);
|
||||
expect(triage.deferred + triage.degraded).toBe(25);
|
||||
expect(result.summary).toContain('not yet triaged');
|
||||
expect(result.summary).toContain('dream retriage');
|
||||
});
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
@@ -499,9 +540,17 @@ describe('E2E synthesize — verdict cache (Q-2)', () => {
|
||||
await runPhaseSynthesize(rig.engine, { brainDir: rig.brainDir, dryRun: false });
|
||||
const { createHash } = await import('node:crypto');
|
||||
const hash = createHash('sha256').update(body, 'utf8').digest('hex');
|
||||
// Triage-v1 cache validity: a below-threshold score with matching
|
||||
// (model, triage_version) is a HIT that gates the file out.
|
||||
await rig.engine.putDreamVerdict(filePath, hash, {
|
||||
worth_processing: false,
|
||||
reasons: ['cached test verdict'],
|
||||
score: 0.1,
|
||||
content_type: null,
|
||||
segments: [],
|
||||
entities: [],
|
||||
model: TIER_DEFAULTS.utility,
|
||||
triage_version: TRIAGE_VERSION,
|
||||
});
|
||||
const result = await runPhaseSynthesize(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
@@ -600,11 +649,11 @@ describe('E2E synthesize — degenerate verdicts are NOT cached in dream_verdict
|
||||
|
||||
test('truncated judge response (stop_reason=length) → no dream_verdicts row + warning', async () => {
|
||||
const { verdictRow, stderr } = await runWithStubbedJudge({
|
||||
text: '{"worth_process', // reasoning ate the budget; partial JSON
|
||||
text: '{"scor', // reasoning ate the budget; partial JSON
|
||||
stopReason: 'length',
|
||||
});
|
||||
expect(verdictRow).toBeNull();
|
||||
expect(stderr).toMatch(/\[dream\] verdict for 2026-05-01-session was truncated/);
|
||||
expect(stderr).toMatch(/\[dream\] triage for 2026-05-01-session was truncated/);
|
||||
expect(stderr).toMatch(/not caching in dream_verdicts/);
|
||||
}, 30_000);
|
||||
|
||||
@@ -614,19 +663,74 @@ describe('E2E synthesize — degenerate verdicts are NOT cached in dream_verdict
|
||||
stopReason: 'end',
|
||||
});
|
||||
expect(verdictRow).toBeNull();
|
||||
expect(stderr).toMatch(/\[dream\] verdict for 2026-05-01-session was unparseable/);
|
||||
expect(stderr).toMatch(/\[dream\] triage for 2026-05-01-session was unparseable/);
|
||||
expect(stderr).toMatch(/not caching in dream_verdicts/);
|
||||
}, 30_000);
|
||||
|
||||
test('control: clean parseable verdict is still cached', async () => {
|
||||
test('boolean-era judge output (no score) is unparseable — never cached', async () => {
|
||||
// The old `{"worth_processing": ...}` shape has no score; under triage-v1
|
||||
// it must NOT become a cacheable rejection.
|
||||
const { verdictRow, stderr } = await runWithStubbedJudge({
|
||||
text: '{"worth_processing": false, "reasons": ["routine ops"]}',
|
||||
stopReason: 'end',
|
||||
});
|
||||
expect(verdictRow).toBeNull();
|
||||
expect(stderr).toMatch(/\[dream\] triage for 2026-05-01-session was unparseable/);
|
||||
}, 30_000);
|
||||
|
||||
test('control: clean scored verdict is cached with score + model + triage_version', async () => {
|
||||
const { verdictRow, stderr } = await runWithStubbedJudge({
|
||||
text: '{"score": 0.1, "content_type": "routine", "segments": [], "entities": [], "reasons": ["routine ops"]}',
|
||||
stopReason: 'end',
|
||||
});
|
||||
expect(verdictRow).not.toBeNull();
|
||||
expect((verdictRow as { worth_processing: boolean }).worth_processing).toBe(false);
|
||||
const row = verdictRow as { worth_processing: boolean; score: number | null; content_type: string | null; model: string | null; triage_version: number | null };
|
||||
expect(row.worth_processing).toBe(false); // derived: 0.1 < DEFAULT_TRIAGE_THRESHOLD
|
||||
expect(row.score).toBe(0.1);
|
||||
expect(row.content_type).toBe('routine');
|
||||
expect(row.model).toBe(TIER_DEFAULTS.utility);
|
||||
expect(row.triage_version).toBe(TRIAGE_VERSION);
|
||||
expect(stderr).not.toMatch(/not caching in dream_verdicts/);
|
||||
}, 30_000);
|
||||
|
||||
test('legacy boolean-era cached row (score NULL) is a MISS — re-judged and overwritten', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
const filePath = join(rig.corpusDir, '2026-05-02-legacy.txt');
|
||||
const body = 'a meaningful conversation\n'.repeat(200);
|
||||
writeFileSync(filePath, body);
|
||||
const { createHash } = await import('node:crypto');
|
||||
const hash = createHash('sha256').update(body, 'utf8').digest('hex');
|
||||
// Seed a boolean-era row directly (score NULL — pre-v129 shape).
|
||||
await rig.engine.executeRaw(
|
||||
`INSERT INTO dream_verdicts (file_path, content_hash, worth_processing, reasons)
|
||||
VALUES ($1, $2, true, '["legacy row"]'::jsonb)`,
|
||||
[filePath, hash],
|
||||
);
|
||||
__setChatTransportForTests(async () => ({
|
||||
text: '{"score": 0.9, "content_type": "reflection", "segments": [], "entities": [], "reasons": ["re-judged"]}',
|
||||
blocks: [],
|
||||
stopReason: 'end',
|
||||
usage: { input_tokens: 10, output_tokens: 200, cache_read_tokens: 0, cache_creation_tokens: 0 },
|
||||
model: 'test:stub',
|
||||
providerId: 'test',
|
||||
}));
|
||||
await withFakeAnthropicKey(() =>
|
||||
runPhaseSynthesize(rig.engine, { brainDir: rig.brainDir, dryRun: true }),
|
||||
);
|
||||
const row = await rig.engine.getDreamVerdict(filePath, hash);
|
||||
expect(row).not.toBeNull();
|
||||
expect(row!.score).toBe(0.9);
|
||||
expect(row!.triage_version).toBe(TRIAGE_VERSION);
|
||||
expect(row!.reasons).toEqual(['re-judged']);
|
||||
} finally {
|
||||
__setChatTransportForTests(null);
|
||||
resetGateway();
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
describe('E2E synthesize — PGLite inline subagent drain (takeover of #2699)', () => {
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
/**
|
||||
* #4152 — Postgres-only checks for the triage-v1 dream_verdicts widening.
|
||||
*
|
||||
* DATABASE_URL-gated: self-skips without a real Postgres (same placement
|
||||
* pattern as test/e2e/op-checkpoint-jsonb-parity.test.ts — picked up by the
|
||||
* existing Postgres-service CI job; deliberately NOT a .serial.test.ts,
|
||||
* which would need its own workflow job and silently never run).
|
||||
*
|
||||
* Covers what PGLite structurally cannot:
|
||||
* 1. Migration-129 upgrade path: a brain whose dream_verdicts predates the
|
||||
* six triage columns gains them from the idempotent ALTERs.
|
||||
* 2. jsonb_typeof on the postgres.js write path: segments/entities land as
|
||||
* real jsonb ARRAYS, not double-encoded string scalars (#2339 class).
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { hasDatabase, setupDB, teardownDB, getEngine } from './helpers.ts';
|
||||
import { runMigrations } from '../../src/core/migrate.ts';
|
||||
|
||||
const describePg = hasDatabase() ? describe : describe.skip;
|
||||
|
||||
describePg('#4152 dream_verdicts triage-v1 — Postgres', () => {
|
||||
beforeAll(async () => {
|
||||
await setupDB();
|
||||
}, 90_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await teardownDB();
|
||||
}, 30_000);
|
||||
|
||||
test('migration 129 upgrade path: pre-129 table shape gains the six columns idempotently', async () => {
|
||||
const engine = getEngine();
|
||||
// Simulate a pre-129 brain: drop the triage columns, then re-run
|
||||
// migrations. The idempotent ALTER ... IF NOT EXISTS set must restore
|
||||
// them without erroring on any other (already-applied) migration.
|
||||
await engine.executeRaw(`
|
||||
ALTER TABLE dream_verdicts
|
||||
DROP COLUMN IF EXISTS score,
|
||||
DROP COLUMN IF EXISTS content_type,
|
||||
DROP COLUMN IF EXISTS segments,
|
||||
DROP COLUMN IF EXISTS entities,
|
||||
DROP COLUMN IF EXISTS model,
|
||||
DROP COLUMN IF EXISTS triage_version
|
||||
`);
|
||||
await engine.setConfig('version', '128'); // force the v129 step to re-run
|
||||
await runMigrations(engine);
|
||||
const cols = await engine.executeRaw<{ column_name: string }>(
|
||||
`SELECT column_name FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'dream_verdicts'`,
|
||||
);
|
||||
const names = new Set(cols.map(c => c.column_name));
|
||||
for (const c of ['score', 'content_type', 'segments', 'entities', 'model', 'triage_version']) {
|
||||
expect(names.has(c)).toBe(true);
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test('jsonb_typeof: segments/entities are jsonb arrays on disk, never string scalars', async () => {
|
||||
const engine = getEngine();
|
||||
await engine.putDreamVerdict('/corpus/typeof.txt', 'typeof-hash-0001', {
|
||||
worth_processing: true,
|
||||
reasons: ['r1'],
|
||||
score: 0.7,
|
||||
content_type: 'idea',
|
||||
segments: [{ quote: 'verbatim', note: 'n' }],
|
||||
entities: ['acme-example'],
|
||||
model: 'anthropic:claude-haiku-4-5-20251001',
|
||||
triage_version: 1,
|
||||
});
|
||||
const rows = await engine.executeRaw<{ seg_t: string; ent_t: string; reasons_t: string }>(
|
||||
`SELECT jsonb_typeof(segments) AS seg_t, jsonb_typeof(entities) AS ent_t,
|
||||
jsonb_typeof(reasons) AS reasons_t
|
||||
FROM dream_verdicts
|
||||
WHERE file_path = '/corpus/typeof.txt' AND content_hash = 'typeof-hash-0001'`,
|
||||
);
|
||||
// 'string' here = the #2339 double-encode bug PGLite can't surface.
|
||||
expect(rows[0].seg_t).toBe('array');
|
||||
expect(rows[0].ent_t).toBe('array');
|
||||
expect(rows[0].reasons_t).toBe('array');
|
||||
}, 30_000);
|
||||
});
|
||||
@@ -150,6 +150,57 @@ describeBoth('Engine parity — Postgres vs PGLite', () => {
|
||||
expect(pgResults[0]?.slug).toBe(pgliteResults[0]?.slug);
|
||||
});
|
||||
|
||||
test('#4152 dream verdict triage-v1 round-trip: identical shape on both engines (jsonb path)', async () => {
|
||||
// The postgres path binds segments/entities via sql.json(); PGLite via
|
||||
// $N::jsonb + JSON.stringify. A double-encode regression on the postgres
|
||||
// side would come back as a jsonb STRING scalar — the parity assert on
|
||||
// the parsed arrays catches exactly that class (#2339).
|
||||
const input = {
|
||||
worth_processing: true,
|
||||
reasons: ['thesis articulated', 'names a pattern'],
|
||||
score: 0.83,
|
||||
content_type: 'reflection',
|
||||
segments: [
|
||||
{ quote: 'a verbatim line with "quotes" and unicode — 🤖', note: 'why it matters' },
|
||||
{ quote: 'second segment' },
|
||||
],
|
||||
entities: ['acme-example', 'fund-a'],
|
||||
model: 'anthropic:claude-haiku-4-5-20251001',
|
||||
triage_version: 1,
|
||||
};
|
||||
for (const eng of [pgEngine, pgliteEngine]) {
|
||||
await eng.putDreamVerdict('/corpus/parity.txt', 'parity-hash-0001', input);
|
||||
// Upsert path: overwrite with a new score, same PK.
|
||||
await eng.putDreamVerdict('/corpus/parity.txt', 'parity-hash-0001', { ...input, score: 0.31 });
|
||||
}
|
||||
const pg = await pgEngine.getDreamVerdict('/corpus/parity.txt', 'parity-hash-0001');
|
||||
const lite = await pgliteEngine.getDreamVerdict('/corpus/parity.txt', 'parity-hash-0001');
|
||||
expect(pg).not.toBeNull();
|
||||
expect(lite).not.toBeNull();
|
||||
for (const v of [pg!, lite!]) {
|
||||
expect(v.score).toBe(0.31);
|
||||
expect(v.content_type).toBe('reflection');
|
||||
expect(Array.isArray(v.segments)).toBe(true); // NOT a double-encoded string scalar
|
||||
expect(v.segments).toEqual(input.segments);
|
||||
expect(v.entities).toEqual(input.entities);
|
||||
expect(v.model).toBe(input.model);
|
||||
expect(v.triage_version).toBe(1);
|
||||
}
|
||||
// Legacy-row semantics: a boolean-era row reads back with null triage fields.
|
||||
for (const eng of [pgEngine, pgliteEngine]) {
|
||||
await eng.executeRaw(
|
||||
`INSERT INTO dream_verdicts (file_path, content_hash, worth_processing, reasons)
|
||||
VALUES ('/corpus/legacy.txt', 'legacy-hash-0001', true, '["old"]'::jsonb)
|
||||
ON CONFLICT (file_path, content_hash) DO NOTHING`,
|
||||
);
|
||||
const legacy = await eng.getDreamVerdict('/corpus/legacy.txt', 'legacy-hash-0001');
|
||||
expect(legacy!.score).toBeNull();
|
||||
expect(legacy!.triage_version).toBeNull();
|
||||
expect(legacy!.segments).toEqual([]);
|
||||
expect(legacy!.entities).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
test('email citation metadata projects identically across engines', async () => {
|
||||
const slug = 'mail/example-citation';
|
||||
const page = {
|
||||
|
||||
@@ -881,6 +881,19 @@ const COLUMN_EXEMPTIONS = new Set<string>([
|
||||
'minion_jobs.budget_remaining_cents',
|
||||
'minion_jobs.budget_owner_job_id',
|
||||
'minion_jobs.budget_root_owner_id',
|
||||
// #4152 (migration v129) — triage-v1 columns on dream_verdicts. Same
|
||||
// precedent as facts.dimension et al: dream_verdicts is migration-created
|
||||
// on PGLite (v30, absent from PGLITE_SCHEMA_SQL), so no schema-blob
|
||||
// forward reference can exist; no index references these columns; and
|
||||
// every reader (runTriagePass cache-validity check) treats NULL as a
|
||||
// legacy-era cache miss, so pre-v129 rows are invisible-by-design until
|
||||
// re-judged. Column-only, no bootstrap probe needed.
|
||||
'dream_verdicts.score',
|
||||
'dream_verdicts.content_type',
|
||||
'dream_verdicts.segments',
|
||||
'dream_verdicts.entities',
|
||||
'dream_verdicts.model',
|
||||
'dream_verdicts.triage_version',
|
||||
]);
|
||||
|
||||
test('every ALTER TABLE ADD COLUMN in MIGRATIONS is covered by applyForwardReferenceBootstrap (column-only class)', async () => {
|
||||
|
||||
Reference in New Issue
Block a user