diff --git a/docs/architecture/KEY_FILES.md b/docs/architecture/KEY_FILES.md index 28e141d3c..80d63cf87 100644 --- a/docs/architecture/KEY_FILES.md +++ b/docs/architecture/KEY_FILES.md @@ -132,7 +132,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this - `src/core/ai/recipes/voyage.ts` — Voyage AI openai-compatible recipe. Declares `chars_per_token=1` + `safety_factor=0.5` so the gateway pre-splits Voyage batches at a 60K-character budget (50% of 120K-token cap with the dense-tokenizer ratio), avoiding the backfill loop where tiktoken-grounded budgeting undercounted Voyage's actual token usage. Declares `multimodal_models: ['voyage-multimodal-3']` so the gateway rejects text-only Voyage models pointed at the multimodal endpoint with a clear `AIConfigError` instead of waiting for Voyage's HTTP 400. Recipe docstring at `:7-16` names the seven hosted flexible-dim models that accept `output_dimension` (`voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-code-3`) and notes `voyage-4-nano` is the open-weight variant fixed at 1024-dim that does NOT accept the parameter (negative regression assertion in `test/ai/gateway.test.ts`: `dimsProviderOptions` returns `undefined` for `voyage-4-nano`). `voyage-code-3` is the recommended embedding model for gstack per-worktree code brains (Topology 3 in `docs/architecture/topologies.md`); discoverability surfaces: decision-tree branch in `docs/integrations/embedding-providers.md`, Topology 3 "Recommended embedding model" subsection, runtime nudge from `gbrain reindex --code` against non-code-tuned models. Recipe-shape regression pinned by `test/ai/voyage-code-3-recipe.test.ts`. - `src/core/ai/recipes/anthropic.ts` — Anthropic recipe (chat + expansion touchpoints). Canonical id is `claude-sonnet-4-6` (no date suffix); a reverse alias `claude-sonnet-4-6-20250929 → claude-sonnet-4-6` keeps stale user configs working (rescues `facts.extraction_model` and `models.dream.synthesize`). Recipe-shape regression pinned by `test/anthropic-model-ids.test.ts`. - `src/core/model-pricing.ts` — single source of truth for paid-cloud CHAT/completion model pricing (USD per 1M tokens, input | output). `CANONICAL_PRICING` is a `provider:model`-keyed table (Anthropic Opus 5/4.8/4.7/4.6 `$5/$25`, Sonnet 4.6 `$3/$15`, Haiku 4.5 `$1/$5` both dateless + dated, plus OpenAI / Google / Together / DeepSeek panel models). `canonicalLookup(modelId)` resolves bare (`claude-opus-4-8`), colon (`anthropic:claude-opus-4-8`), and slash (`anthropic/...`) forms — bare ids default to the `anthropic:` provider; nested OpenRouter ids (`openrouter:anthropic/...`) intentionally MISS so OpenRouter markup isn't repriced as the inner vendor. Every other chat-pricing table is a DERIVED view of this one (NOT a hand-copied duplicate), so cross-table price drift is structurally impossible. Embeddings live separately in `embedding-pricing.ts` (different unit). Pinned by `test/model-pricing.test.ts` whose drift guard asserts each derived view equals canonical and that the cross-modal panel models are all present. -- `src/core/anthropic-pricing.ts` — bare-keyed Anthropic VIEW of `model-pricing.ts` (the `anthropic:` canonical entries with the prefix stripped). Kept distinct because many callers look up by bare Claude id and because `estimateMaxCostUsd(modelId, inTokens, maxOutTokens)` carries the null-on-miss contract the dream-cycle budget gate depends on (non-Anthropic ids return null → caller warns `BUDGET_METER_NO_PRICING` once and runs unbounded). `estimateMaxCostUsd` routes bare/colon/slash ids through `splitProviderModelId`. Do NOT hand-edit prices here — the map is derived from canonical, so it cannot drift. `ANTHROPIC_PRICING` is consumed by `budget/budget-tracker.ts`, `minions/batch-projection.ts`, and `cycle/budget-meter.ts`. +- `src/core/anthropic-pricing.ts` — bare-keyed Anthropic VIEW of `model-pricing.ts` (the `anthropic:` canonical entries with the prefix stripped). Kept distinct because many callers look up by bare Claude id and because `estimateMaxCostUsd(modelId, inTokens, maxOutTokens)` carries the null-on-miss contract the dream-cycle budget gate depends on (non-Anthropic ids return null; `BudgetMeter` tries `canonicalLookup` first and only falls back here, so it warns `BUDGET_METER_NO_PRICING` and runs unbounded only when canonical has no rates either). `estimateMaxCostUsd` routes bare/colon/slash ids through `splitProviderModelId`. Do NOT hand-edit prices here — the map is derived from canonical, so it cannot drift. `ANTHROPIC_PRICING` is consumed by `budget/budget-tracker.ts`, `minions/batch-projection.ts`, and `cycle/budget-meter.ts`. - `src/core/takes-quality-eval/pricing.ts` — fail-closed budget pricing for `eval takes-quality run --budget-usd N`. `MODEL_PRICING` is a curated `provider:model` allowlist (default panel + likely overrides) whose VALUES are derived from `model-pricing.ts` via `canonicalLookup`; an allowlisted id missing from canonical throws at module load. Schema is `{input_per_1m, output_per_1m}`. A model NOT on the allowlist aborts the run with an actionable error rather than guessing (distinct from `cross-modal-eval/runner.ts`, which silently estimates zero on unknown models — both now source numbers from canonical). - `src/core/budget/budget-tracker.ts` — keystone primitive for the brainstorm cost-cathedral wave. One typed error (`BudgetExhausted` with `reason: 'cost' | 'runtime' | 'no_pricing'`), one schema-stable audit JSONL at `~/.gbrain/audit/budget-YYYY-Www.jsonl`. Contracts: `record()` throws when cumulative spend exceeds cap (the cap is a real ceiling, not a suggestion); `reserve()` hard-fails with `reason: 'no_pricing'` when `maxCostUsd` is set AND the model is missing from pricing maps (warn-once preserved when cap is unset); `extractUsageFromError(err, fallback)` returns `err.usage` when the SDK provides it, else the pessimistic fallback (caller passes `maxOutputTokens`, not the optimistic pre-call estimate). `onExhausted(cb)` fires once synchronously BEFORE the throw propagates so callers can persist checkpoints. Replaces three parallel copies (inline brainstorm class, cycle/budget-meter, eval-contradictions). Adapts the old `BudgetMeter` (public shape preserved + `schema_version: 1` stamped on every dream-budget audit line). Pinned by 18 unit cases. - `src/core/audit-week-file.ts` — single source of truth for ISO-week audit JSONL filename math. Exports `isoWeek(d)`, `isoWeekFilename(prefix, now?)`, `resolveAuditDir()` (honors `GBRAIN_AUDIT_DIR`). Year-boundary correctness pinned by tests at 2020-W53 (the 53-week year), 2025-W01 rolling in from 2024-12-30 (Monday), 2026-W01. Four call sites migrated: `src/core/minions/handlers/shell-audit.ts`, `src/core/facts/phantom-audit.ts`, `src/core/audit-slug-fallback.ts`, `src/core/cycle/budget-meter.ts`. Each keeps its `computeAuditFilename` thin wrapper for back-compat with existing tests. diff --git a/src/core/anthropic-pricing.ts b/src/core/anthropic-pricing.ts index 668444ed3..aa90641b7 100644 --- a/src/core/anthropic-pricing.ts +++ b/src/core/anthropic-pricing.ts @@ -11,9 +11,12 @@ * takes-quality-eval/pricing.ts duplicated the numbers and drifted: Opus 4.7 * read $15/$75 in one and $5/$25 in the other.) * - * Codex P1 #10 fold: non-Anthropic models (gemini, gpt, anything not in this - * map) bypass the budget gate with a `BUDGET_METER_NO_PRICING` warn once per - * process. The cycle still runs unbounded for those models. + * Codex P1 #10 fold: `estimateMaxCostUsd` keeps its null-on-miss contract, and + * a null estimate makes the dream-cycle budget gate wave the submit through. + * BudgetMeter therefore prices through `canonicalLookup` first and only falls + * back here, so a model is unbounded ONLY when the canonical table has no + * rates for it either — that case still warns `BUDGET_METER_NO_PRICING` once + * per process. */ import { CANONICAL_PRICING, type ModelPricing } from './model-pricing.ts'; diff --git a/src/core/cycle/budget-meter.ts b/src/core/cycle/budget-meter.ts index 446eb3ecb..c0b61ebfb 100644 --- a/src/core/cycle/budget-meter.ts +++ b/src/core/cycle/budget-meter.ts @@ -11,7 +11,9 @@ * * Per Codex P1 #10: each subagent submit estimates max-cost from * `model + max_output_tokens`, accumulates per-cycle, refuses next submit - * if cumulative > budget. Non-Anthropic models bypass the gate with a + * if cumulative > budget. Pricing resolves through the canonical chat table + * (`canonicalLookup`), so any provider carried there is gated. Only a model + * absent from canonical too bypasses the gate, with a * `BUDGET_METER_NO_PRICING` warn (once per process). * * Ledger lives at `~/.gbrain/audit/dream-budget-YYYY-Www.jsonl` (ISO-week @@ -24,6 +26,7 @@ import { mkdirSync, appendFileSync } from 'node:fs'; import { dirname, join } from 'node:path'; import { isoWeekFilename, resolveAuditDir } from '../audit-week-file.ts'; import { estimateMaxCostUsd, ANTHROPIC_PRICING } from '../anthropic-pricing.ts'; +import { canonicalLookup } from '../model-pricing.ts'; export interface BudgetMeterOpts { /** USD cap for the whole cycle. 0 or negative disables the gate. */ @@ -81,21 +84,58 @@ export class BudgetMeter { this.auditPath = auditFilePath(opts.auditPath); } + /** + * Max-cost estimate for a planned submit. + * + * Prices through `canonicalLookup` first. `estimateMaxCostUsd` reads + * ANTHROPIC_PRICING, which CLAUDE.md defines as a DERIVED view of the one + * canonical chat-pricing table — so reaching for it directly made every + * non-Anthropic model unpriceable here even when the canonical table has + * its rates, and an unpriceable model disables the gate entirely (see + * `check`). Anthropic ids resolve identically either way, since the derived + * view is generated from canonical. + * + * Returns null only for models absent from the canonical table too; the + * caller keeps the existing warn-and-allow behaviour for those. + */ + private estimateCost(estimate: SubmitEstimate): number | null { + const p = canonicalLookup(estimate.modelId); + const raw = p + ? (estimate.estimatedInputTokens / 1_000_000) * p.input + + (estimate.maxOutputTokens / 1_000_000) * p.output + : estimateMaxCostUsd( + estimate.modelId, + estimate.estimatedInputTokens, + estimate.maxOutputTokens, + ); + // A non-finite estimate must not reach the accumulator. Both tables are + // plain object literals, so a model id colliding with an inherited key + // ('constructor', 'toString', '__proto__') resolves to a truthy + // Object.prototype value whose .input/.output are undefined, and the + // arithmetic yields NaN. `cumulative + NaN` is NaN, `NaN > budget` is + // false, so a single such submit would silently disable the gate for the + // rest of the cycle. Treated as unpriceable instead, which routes into + // the documented warn-and-allow branch for that one submit and leaves + // the running total intact. (The same shape exists on the + // estimateMaxCostUsd path today; not changed here.) + return raw !== null && Number.isFinite(raw) ? raw : null; + } + /** * Check whether a planned submit fits within the remaining budget. * Records the attempt to the ledger regardless of allow/deny. * Caller is responsible for skipping the actual LLM call when allowed=false. */ check(estimate: SubmitEstimate): BudgetCheckResult { - const cost = estimateMaxCostUsd(estimate.modelId, estimate.estimatedInputTokens, estimate.maxOutputTokens); + const cost = this.estimateCost(estimate); - // Codex P1 #10: non-Anthropic / unpriced models bypass the gate. + // Codex P1 #10: models absent from the canonical table bypass the gate. if (cost === null) { this.unpricedSubmitsThisCycle++; if (!_unpricedWarnings.has(estimate.modelId)) { _unpricedWarnings.add(estimate.modelId); process.stderr.write( - `[budget] BUDGET_METER_NO_PRICING: model "${estimate.modelId}" not in ANTHROPIC_PRICING. ` + + `[budget] BUDGET_METER_NO_PRICING: model "${estimate.modelId}" has no canonical pricing. ` + `Budget gate disabled for this submit. (Per-provider pricing modules: TODO v0.29.)\n`, ); } diff --git a/test/budget-meter.test.ts b/test/budget-meter.test.ts index 79234a601..cf520d882 100644 --- a/test/budget-meter.test.ts +++ b/test/budget-meter.test.ts @@ -3,6 +3,7 @@ import { mkdtempSync, rmSync, readFileSync, existsSync } from 'node:fs'; import { join } from 'node:path'; import { tmpdir } from 'node:os'; import { BudgetMeter, _resetBudgetMeterWarningsForTest, ANTHROPIC_PRICING } from '../src/core/cycle/budget-meter.ts'; +import { estimateMaxCostUsd } from '../src/core/anthropic-pricing.ts'; let tmpDir: string; let auditPath: string; @@ -60,6 +61,65 @@ describe('BudgetMeter', () => { expect(meter.unpricedSubmits).toBe(2); }); + test('a canonical-priced non-Anthropic model is gated, not waved through', () => { + // openai:gpt-5.2 is in CANONICAL_PRICING but not in the derived + // ANTHROPIC_PRICING view, so the pre-canonical lookup returned null and + // check() took the unpriced bypass: allowed, cost 0, gate disabled. + const meter = new BudgetMeter({ budgetUsd: 0.001, phase: 'auto_think', auditPath }); + const r = meter.check({ + modelId: 'openai:gpt-5.2', + estimatedInputTokens: 1_000_000, + maxOutputTokens: 1_000_000, + label: 'canonical-priced', + }); + expect(r.unpriced).toBeFalsy(); + expect(r.estimatedCostUsd).toBeGreaterThan(0); + expect(r.allowed).toBe(false); // 1M+1M tokens cannot fit $0.001 + expect(meter.unpricedSubmits).toBe(0); + expect(readLedger().at(-1)!.event).toBe('submit_denied'); + }); + + test('a model absent from the canonical table keeps the documented bypass', () => { + // Unchanged behaviour: gemini-3-pro is in neither table. Whether these + // should also be gated (via a conservative fallback rate, as + // synthesize-concepts and skillopt/preflight do) is a policy call and is + // deliberately not decided here. + const meter = new BudgetMeter({ budgetUsd: 0.001, phase: 'auto_think', auditPath }); + const r = meter.check({ modelId: 'gemini-3-pro', estimatedInputTokens: 1000, maxOutputTokens: 1000, label: 'absent' }); + expect(r.unpriced).toBe(true); + expect(r.allowed).toBe(true); + expect(readLedger().at(-1)!.event).toBe('submit_unpriced'); + }); + + test('Anthropic ids price identically through canonical and the derived view', () => { + // The derived view is generated from canonical, so routing through + // canonicalLookup must not move any Anthropic number. + const meter = new BudgetMeter({ budgetUsd: 1000, phase: 'auto_think', auditPath }); + const r = meter.check({ modelId: 'claude-opus-4-7', estimatedInputTokens: 5000, maxOutputTokens: 4000, label: 'parity' }); + const viaView = estimateMaxCostUsd('claude-opus-4-7', 5000, 4000); + expect(viaView).not.toBeNull(); + expect(r.estimatedCostUsd).toBeCloseTo(viaView!, 10); + }); + + test('an inherited-key model id cannot poison the running total', () => { + // Both pricing tables are object literals, so 'constructor' resolves to a + // truthy Object.prototype value and the rate arithmetic yields NaN. If + // that reached cumulativeUsd, every later submit would pass the gate. + const meter = new BudgetMeter({ budgetUsd: 0.001, phase: 'auto_think', auditPath }); + const poison = meter.check({ modelId: 'constructor', estimatedInputTokens: 1000, maxOutputTokens: 1000, label: 'poison' }); + expect(poison.unpriced).toBe(true); // treated as unpriceable + expect(Number.isFinite(poison.cumulativeCostUsd)).toBe(true); + + const after = meter.check({ + modelId: 'claude-opus-4-7', + estimatedInputTokens: 1_000_000, + maxOutputTokens: 1_000_000, + label: 'after-poison', + }); + expect(after.allowed).toBe(false); // gate still enforcing + expect(Number.isFinite(after.cumulativeCostUsd)).toBe(true); + }); + test('ledger captures every submit (allowed + denied + unpriced)', () => { const meter = new BudgetMeter({ budgetUsd: 0.001, phase: 'auto_think', auditPath }); meter.check({ modelId: 'claude-opus-4-7', estimatedInputTokens: 5000, maxOutputTokens: 4000, label: 'a' });