mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-14 08:53:22 +00:00
* feat(core): execution-environment detection — local | cloud-sandbox | ephemeral-container detectExecutionEnvironment() + isCredentialInjectingProxy() with injected signals (CLAUDE_CODE_REMOTE, cse_ session-id prefix, proxy-injected token placeholder, anthropic-egress proxy JWT, container markers). binaryOnPath moves here as the canonical PATH probe. autopilot's detectInstallTarget ephemeral branch now routes through the shared detector. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(bootstrap): repo-visibility ladder + per-turn Stop push + same-session failure banner The durability lane. One repo-visibility verdict for every consumer (REST first — never GraphQL, which sandbox proxies pin; authed ls-remote + an attributed anonymous probe as the git-protocol fallback), replacing three drifted probes. A 200 counts as public only with advertisement proof; a 401/404 counts as private-signal only with an auth challenge — fail-closed in BOTH directions. Private verdicts cache 1h (private-only, per origin). gbrain hook stop now spawns a debounced detached push per turn (per-root state; cloud-sandbox defaults to every turn, elsewhere 5 min; a failing status bypasses the debounce), closing the /exit and VM-reclaim gaps. Push status is per workspace root, read through one shared reader by the user-prompt banner (additionalContext + systemMessage — visible to the human, not just the model), the SessionStart note, and doctor. Escape hatches for self-hosted git: flag > env > file-plane config key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(bootstrap): environment-aware install — honest cron skip, cloud repo-create guard, execution_env surfaces installDurabilityCron probes for crontab before writing anything (containers and cloud sandboxes ship without one — expected, reported as an honest skip naming the event-driven pushes that still cover persistence). The repo phase installs the container-friendly harden half (post-commit hook, no scheduler) outside local machines. createPrivateRepo fails fast in cloud sandboxes with the flow that works (create outside, open the session ON the repo, attach). bootstrap verify gains a never-gating execution_env check; bootstrap status --json carries execution_environment for installing agents to branch on. Also fixes a live-PATH resolution class: binaryOnPath and the crontab execs now pass the current env explicitly (Bun resolves against the startup snapshot otherwise). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bootstrap): lifecycle hygiene — uninstall teardown, doctor job liveness, .mcp.json out of the repo, honest persistence copy uninstall now tears down the durability wiring it installed (launchd/cron job, untracked post-commit hook, credential wiring — the committed helper and AGENTS rules stay). doctor gains bootstrap_durability_job: presence + LIVENESS (launchctl load-state, crontab line, pull-log freshness) — a plist on disk with a dead job no longer reads as healthy. Rendered .gitignore now covers .mcp.json (absolute machine paths must not land in the private repo); verify warns on pre-fix installs that committed it; the never-built state/mcp.json promise is gone from GITHUB.md. Persistence copy tells the truth everywhere: event-driven pushes do the durability work, the 30-min job is a multi-machine pull freshener. Post-commit hook install/removal is worktree-safe (git-path resolution; the git marker is a FILE there) and the cron wrapper's self-disable tests the repo dir, not its git marker. Two follow-up TODOs filed (plugin hook distribution; Channels push lane). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(bootstrap): committed hook carrier for cloud + cloud-setup-script emitter + cloud runbook Cloud sessions clone fresh and snapshot hook config at session start — the gitignored settings.local.json never exists there, so hooks never fired in cloud at all. Cloud installs now write the repo-COMMITTED .claude/settings.json with PATH-resolved, fail-open commands (no machine paths; a host without the binary no-ops); local installs keep settings.local.json; the writers enforce that one event never fires from both carriers, and removal cleans both. New: gbrain bootstrap cloud-setup-script prints the paste-ready environment setup script (npm transport — bun fetching is proxy-incompatible in cloud; never the unrelated npm-registry package). Runbook gains a NEVER FABRICATE TOOLING hard rule, a cloud-sandbox section (expected degradations as facts to relay, the attach-first flow), and failure-table rows for the proxy-403 and missing-crontab signatures. Codex lane stated honestly: AGENTS.md Gate 2 now has the pull-side push-health check. Guide documents the new knobs and the cloud contract; llms bundles regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(bootstrap): per-turn push e2e chain, cloud-sandbox sim, per-root status reader migration + registry regen bootstrap-persistence e2e gains the Stop-hook chain (one stop banks the turn to the real bare remote; the debounce holds across stops; debounce-0 lands consecutive turns) and fixes a pre-existing post-#4024 break: repoPhaseComplete required a github-parseable origin, so session-end pushes deferred FOREVER for self-hosted/file-transport origins — non-github repo_urls now bind by exact URL equality (redirect protection preserved). Degraded-modes e2e gains the cloud-sandbox simulation (status reports the environment; repo creation refuses with the attach flow). workspace-push tests read per-root status through the shared reader. Flag registry regenerated (new git argv literals from the git-path/ls-files calls — the accepted argv-bleed class). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(doctor): categorize bootstrap_durability_job (categories drift guard) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bootstrap): adversarial-review hardening — fail-closed privacy, no push-storm, no exfil paths Both Claude and Codex adversarial passes ran; 11 findings fixed (union): - Non-github 401+WWW-Authenticate no longer reads as private (RFC 7235 makes the header mandatory on every 401, so a middlebox 401s identically) — the top exfil path both models flagged; now unverifiable/fail-closed, operator confirms via the escape hatch. github.com still needs x-github-request-id. - Escape hatches downgrade ONLY 'unverifiable' — a PROVEN-public origin still refuses (hatches never authorize a public push). - treeNeedsPush measures against origin/<branch> (the push's own ref), not @{u}: a no-upstream branch no longer reports a committed-but-unpushed tree as push_clean and silently strands it. - [D20] failing-retry uses a fixed 60s floor, not min(debounce,60s) — cloud debounce=0 no longer re-runs the network ladder every turn. - Committed hook carrier: dedupe/suppress only on the EXACT portable-command shape, not a 'gbrain hook' substring (blocks the suppress-local-run-evil supply-chain vector); GBRAIN_HOME refused in the committed carrier. - push-status reason sanitized (charset+length) at every surface (banner, doctor, status blob) so remote git stderr can't inject via the remediation. - Per-root state: ghost-root records (deleted workspaces) filtered so a dead failing record can't re-fire the banner forever; uninstall removes them. - statusReport support blob reads push status through the shared per-root reader; visibility cache strips URL userinfo (no PAT persisted). - anonProbe strips userinfo + redirect:manual + SSRF flags + --end-of-options on ls-remote; cron self-disable uses git rev-parse (worktree-safe both ways); cloud-setup-script fails loud on a broken update; durability liveness won't certify a never-run crontab as live; config get/unset resolve the dotted file-plane keys; .bak/.broken gitignored; typed config fields. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * v0.45.8.0 feat(bootstrap): first-class cloud-sandbox install + per-turn persistence + fail-closed privacy ladder VERSION + package.json + CHANGELOG + TODOS + llms bundles. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: sync KEY_FILES + README for the cloud-DX wave (v0.45.8.0) New KEY_FILES entries for execution-env.ts and repo-visibility.ts; updated the bootstrap repo/hooks, workspace-push, brain-repo-durability, and hook.ts entries to current behavior (ladder verification, committed cloud hook carrier, per-root push status, crontab probe + liveness, per-turn stop push + banner). README's Claude Code line now states per-turn + cloud persistence honestly. llms bundles regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bootstrap): doc-review pass — attach records repo_url (enables cloud persistence), verify reads per-root push status, honest docs Cross-model doc review (Codex) caught that the headline cloud-persistence claim was hollow and two docs over-claimed: - attach now records repo_url from the adopted origin, so the no-daemon push gate (repoPhaseComplete) recognizes the repo phase as done — WITHOUT this, the per-turn/session-end pushes deferred forever after an attach, which is the ONLY install path in a cloud sandbox (repo is refused there). Privacy is still enforced at push time by the ladder. - bootstrap verify's push_probe reads the shared per-root reader [D8], not the legacy single file — a fresh v0.45.8 install no longer reports 'no push recorded' when per-root status exists. - Docs corrected to match code: runbook stamp → 0.45.8.0; README states the per-turn cadence honestly (debounced local, next-turn failure notice); GITHUB.md qualifies auto-push as Claude Code (Codex is pull); KEY_FILES says repo/status use REST (not the full ladder) and drops the stale --push; bootstrap.md hooks-location covers both carriers; README verbs surface says seven. llms bundles regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bootstrap): regenerate vendored template-repo tree + isolate env-mutating tests (verify gate) The verify CI gate caught two things the piecemeal local runs missed: - templates/bootstrap/template-repo/ is a GENERATED tree; the source-template edits (AGENTS.md Gate 2, CLAUDE.md cloud note) plus the version stamp had drifted. Regenerated from source (check:bootstrap-templates green). - test/bootstrap-repo.test.ts + test/durability-cron.test.ts newly mutated process.env in non-serial files; converted to withEnv() (check:test-isolation green) — no .serial rename needed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(ambient-recall): pin a keyless gateway so delta/context_pack writes never fire a real embed Root cause of the shard-2 CI red after the 0.45.9.0 re-bump: adding this wave's test files reshuffled the weight-packed shards, moving ambient-recall.test.ts next to a neighbor that leaks CI's dummy OPENAI_API_KEY (sk-test-*) into the gateway singleton (the bunfig preload configures with env:{...process.env}, and a present-but-invalid key turns remember's keyless-degrade embed into a hard 401). The delta/context_pack tests exercise cursor + budget logic, not embedding quality, so this pins a keyless gateway (env:{}) in beforeAll — isAvailable('embedding') is false, writeSingleFact degrades (degraded_dedup) with no HTTP call, and the file is deterministic regardless of shard bin-packing. Verified: passes with OPENAI_API_KEY=sk-test-* set (the CI condition). Master's own comment in legacy-embedding-preload.ts already warns 'adding any test file reshuffles the mines'; this makes the file immune. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
255 lines
11 KiB
TypeScript
255 lines
11 KiB
TypeScript
/**
|
|
* GAP 7 — session-end persistence chain, UNMOCKED end-to-end.
|
|
*
|
|
* `test/hook-command.serial.test.ts` proves session-end up to the mocked
|
|
* `io.spawnPush` seam (it only records that a push WOULD fire). This file
|
|
* closes the remaining gap: it drives the session-end hook against a REAL
|
|
* local bare git remote with the REAL push implementation — the same
|
|
* `workspacePush` the detached `gbrain sources push` child runs — so the full
|
|
* hook → scan → commit → push chain is exercised and its outcome verified on
|
|
* the remote.
|
|
*
|
|
* The only concession to a hermetic test is `allowUnverifiedRemote: true`:
|
|
* `workspacePush`'s privacy gate can only confirm PRIVATE visibility for a
|
|
* real GitHub origin via `gh`, which a `file://` bare remote isn't — every
|
|
* sibling `workspace-push.serial.test.ts` case makes the same concession. The
|
|
* scan/commit/push machinery under test is otherwise identical to production.
|
|
*
|
|
* Serial: mutates HOME/GBRAIN_HOME + spawns git subprocesses. All secrets are
|
|
* synthetic fixtures; GBRAIN_GIT_ALLOW_FILE_TRANSPORT=1 permits file transport.
|
|
*/
|
|
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
|
import { mkdtempSync, mkdirSync, rmSync, writeFileSync, existsSync, readFileSync } from 'node:fs';
|
|
import { join } from 'node:path';
|
|
import { tmpdir } from 'node:os';
|
|
import { execFileSync } from 'node:child_process';
|
|
|
|
import { runHook } from '../../src/commands/hook.ts';
|
|
import { workspacePush, type WorkspacePushResult } from '../../src/core/workspace-push.ts';
|
|
import { writeManifest, writeReceipt } from '../../src/core/bootstrap/format.ts';
|
|
|
|
const OPENAI_KEY = 'sk-' + 'A1b2C3d4E5f6G7h8I9j0K1l2M3n4';
|
|
const ENV_KEYS = ['HOME', 'GBRAIN_HOME', 'DATABASE_URL', 'GBRAIN_DATABASE_URL', 'GBRAIN_HOOKS', 'GBRAIN_GIT_ALLOW_FILE_TRANSPORT'] as const;
|
|
|
|
// #2943: env: process.env is REQUIRED — Bun snapshots env at startup, so a
|
|
// spawned git would otherwise be blind to beforeEach's HOME/GBRAIN_HOME swap.
|
|
function git(cwd: string, ...args: string[]): string {
|
|
return execFileSync('git', ['-C', cwd, '-c', 'protocol.file.allow=always', ...args], {
|
|
stdio: ['ignore', 'pipe', 'pipe'], encoding: 'utf-8', env: process.env,
|
|
}).trim();
|
|
}
|
|
function originHead(bare: string): string {
|
|
return git(bare, 'rev-parse', 'refs/heads/main');
|
|
}
|
|
|
|
const MANIFEST = {
|
|
format_version: 1 as const,
|
|
initialized: true as const,
|
|
agent_name: 'persist-test',
|
|
created_by: 'test',
|
|
created_at: '2026-01-01T00:00:00.000Z',
|
|
source_id: 'workspace',
|
|
};
|
|
|
|
let root: string;
|
|
let bare: string;
|
|
let work: string;
|
|
let saved: Record<string, string | undefined>;
|
|
|
|
/** Real push path, in-process: exactly what `gbrain sources push --path`
|
|
* invokes. Captured so the test can await the chain the hook kicks off. */
|
|
let pushes: Promise<WorkspacePushResult>[];
|
|
const realSpawnPush = (r: string) => {
|
|
pushes.push(workspacePush({ dir: r, branch: 'main', allowUnverifiedRemote: true }));
|
|
};
|
|
|
|
beforeEach(() => {
|
|
root = mkdtempSync(join(tmpdir(), 'gb-persist-'));
|
|
saved = {};
|
|
for (const k of ENV_KEYS) saved[k] = process.env[k];
|
|
for (const k of ['DATABASE_URL', 'GBRAIN_DATABASE_URL', 'GBRAIN_HOOKS'] as const) delete process.env[k];
|
|
process.env.HOME = mkdtempSync(join(root, 'home-'));
|
|
// CX2-8: GBRAIN_HOME is a PARENT dir → effective home is $HOME/.gbrain.
|
|
process.env.GBRAIN_HOME = process.env.HOME;
|
|
process.env.GBRAIN_GIT_ALLOW_FILE_TRANSPORT = '1';
|
|
|
|
bare = mkdtempSync(join(root, 'origin-')) + '.git';
|
|
execFileSync('git', ['init', '-q', '--bare', '-b', 'main', bare], { stdio: 'ignore', env: process.env });
|
|
work = mkdtempSync(join(root, 'work-'));
|
|
execFileSync('git', ['-c', 'protocol.file.allow=always', 'clone', '-q', bare, work], { stdio: 'ignore', env: process.env });
|
|
git(work, 'config', 'user.email', 't@t.t');
|
|
git(work, 'config', 'user.name', 'tester');
|
|
writeFileSync(join(work, 'README.md'), 'init\n');
|
|
git(work, 'add', 'README.md');
|
|
git(work, 'commit', '-qm', 'init');
|
|
git(work, 'push', '-q', 'origin', 'main');
|
|
try { git(work, 'remote', 'set-head', 'origin', 'main'); } catch { /* */ }
|
|
|
|
// The initialized manifest is the security boundary the hook gates on.
|
|
writeManifest(work, MANIFEST);
|
|
// Repo phase complete: the receipt binds this workspace to its verified
|
|
// origin (exact-URL binding for non-github transports) — without it the
|
|
// no-daemon pushes defer by design (create-repo-first race protection).
|
|
const toplevel = git(work, 'rev-parse', '--show-toplevel');
|
|
mkdirSync(join(process.env.HOME!, '.gbrain', 'bootstrap'), { recursive: true });
|
|
writeReceipt(join(process.env.HOME!, '.gbrain'), {
|
|
receipt_version: 1,
|
|
workspace_dir: toplevel,
|
|
source_id: 'workspace',
|
|
agent_name: 'persist-test',
|
|
created_at: '2026-01-01T00:00:00.000Z',
|
|
created_by: 'test',
|
|
brain_created_by_bootstrap: false,
|
|
created_paths: [],
|
|
registrations: [],
|
|
repo_url: bare,
|
|
} as Parameters<typeof writeReceipt>[1] & { repo_url: string });
|
|
pushes = [];
|
|
});
|
|
|
|
afterEach(() => {
|
|
for (const k of ENV_KEYS) {
|
|
if (saved[k] === undefined) delete process.env[k];
|
|
else process.env[k] = saved[k];
|
|
}
|
|
rmSync(root, { recursive: true, force: true });
|
|
});
|
|
|
|
describe('session-end → real workspace push', () => {
|
|
test('authored session content actually lands on the bare remote', async () => {
|
|
// The agent authored real work into the brain during the session.
|
|
mkdirSync(join(work, 'brain'), { recursive: true });
|
|
const authored = 'the acme-example roadmap decision: ship the memory layer in Q3';
|
|
writeFileSync(join(work, 'brain', 'decision.md'), `# decision\n\n${authored}\n`);
|
|
|
|
const before = originHead(bare);
|
|
const code = await runHook(['session-end'], {
|
|
write: () => {},
|
|
cwd: work,
|
|
spawnPush: realSpawnPush,
|
|
stdin: JSON.stringify({ session_id: 'persist-ok', cwd: work }),
|
|
});
|
|
expect(code).toBe(0);
|
|
|
|
// The hook kicked off exactly one real push; await its completion.
|
|
expect(pushes).toHaveLength(1);
|
|
const res = await pushes[0];
|
|
expect(res.status).toBe('pushed');
|
|
expect(res.ok).toBe(true);
|
|
expect(res.committed).toBe(true);
|
|
|
|
// Origin advanced, and the authored file+content are physically there.
|
|
expect(originHead(bare)).not.toBe(before);
|
|
const verify = mkdtempSync(join(root, 'verify-'));
|
|
execFileSync('git', ['-c', 'protocol.file.allow=always', 'clone', '-q', bare, verify], { stdio: 'ignore', env: process.env });
|
|
const landed = join(verify, 'brain', 'decision.md');
|
|
expect(existsSync(landed)).toBe(true);
|
|
expect(readFileSync(landed, 'utf-8')).toContain(authored);
|
|
}, 60_000);
|
|
|
|
test('a planted secret in authored content is BLOCKED at the push gate — nothing leaves', async () => {
|
|
mkdirSync(join(work, 'brain'), { recursive: true });
|
|
// A genuine note plus a file that leaks a synthetic OpenAI key.
|
|
writeFileSync(join(work, 'brain', 'safe.md'), '# safe\n\nno secrets here\n');
|
|
writeFileSync(join(work, 'brain', 'leak.md'), `# oops\n\napi key: ${OPENAI_KEY}\n`);
|
|
|
|
const before = originHead(bare);
|
|
const code = await runHook(['session-end'], {
|
|
write: () => {},
|
|
cwd: work,
|
|
spawnPush: realSpawnPush,
|
|
stdin: JSON.stringify({ session_id: 'persist-leak', cwd: work }),
|
|
});
|
|
expect(code).toBe(0);
|
|
|
|
expect(pushes).toHaveLength(1);
|
|
const res = await pushes[0];
|
|
// The scan gate fired: nothing committed, nothing pushed, secret named.
|
|
expect(res.status).toBe('blocked_secrets');
|
|
expect(res.ok).toBe(false);
|
|
expect(res.findings?.some((f) => f.file === 'brain/leak.md' && f.pattern === 'openai')).toBe(true);
|
|
// The secret VALUE never surfaces in the result payload.
|
|
expect(JSON.stringify(res).includes(OPENAI_KEY)).toBe(false);
|
|
|
|
// Origin is untouched — the leak never left the machine.
|
|
expect(originHead(bare)).toBe(before);
|
|
const shipped = git(bare, 'ls-tree', '-r', '--name-only', 'main');
|
|
expect(shipped).not.toContain('brain/leak.md');
|
|
expect(shipped).not.toContain('brain/safe.md'); // blocked atomically — nothing in the batch shipped
|
|
}, 60_000);
|
|
});
|
|
|
|
// ── per-turn Stop push [D3]: the /exit + VM-reclaim durability lane ─────────
|
|
//
|
|
// SessionEnd never fires on /exit; the Stop hook fires after EVERY assistant
|
|
// turn. This chain proves a turn's authored work physically lands on the real
|
|
// bare remote from a single `gbrain hook stop`, and that the per-root debounce
|
|
// holds across consecutive stops.
|
|
|
|
describe('stop → real workspace push (per-turn durability)', () => {
|
|
test('one stop banks the turn to origin; the next stop inside the window debounces', async () => {
|
|
process.env.GBRAIN_STOP_PUSH_DEBOUNCE_MIN = '60';
|
|
try {
|
|
mkdirSync(join(work, 'brain'), { recursive: true });
|
|
writeFileSync(join(work, 'brain', 'turn-note.md'), '# turn\n\nlearned during the turn\n');
|
|
|
|
const before = originHead(bare);
|
|
const code = await runHook(['stop'], {
|
|
write: () => {},
|
|
cwd: work,
|
|
spawnPush: realSpawnPush,
|
|
stdin: JSON.stringify({ session_id: 'persist-stop', cwd: work }),
|
|
});
|
|
expect(code).toBe(0);
|
|
expect(pushes).toHaveLength(1);
|
|
const res = await pushes[0];
|
|
expect(res.ok).toBe(true);
|
|
expect(res.status).toBe('pushed');
|
|
expect(originHead(bare)).not.toBe(before);
|
|
|
|
// Second stop, same window, clean tree — debounced, no second spawn.
|
|
const code2 = await runHook(['stop'], {
|
|
write: () => {},
|
|
cwd: work,
|
|
spawnPush: realSpawnPush,
|
|
stdin: JSON.stringify({ session_id: 'persist-stop-2', cwd: work }),
|
|
});
|
|
expect(code2).toBe(0);
|
|
expect(pushes).toHaveLength(1);
|
|
} finally {
|
|
delete process.env.GBRAIN_STOP_PUSH_DEBOUNCE_MIN;
|
|
}
|
|
}, 60_000);
|
|
|
|
test('debounce 0 (the cloud-sandbox default): consecutive dirty turns both land', async () => {
|
|
process.env.GBRAIN_STOP_PUSH_DEBOUNCE_MIN = '0';
|
|
try {
|
|
mkdirSync(join(work, 'brain'), { recursive: true });
|
|
writeFileSync(join(work, 'brain', 'a.md'), '# a\n');
|
|
await runHook(['stop'], {
|
|
write: () => {},
|
|
cwd: work,
|
|
spawnPush: realSpawnPush,
|
|
stdin: JSON.stringify({ session_id: 'ps-z1', cwd: work }),
|
|
});
|
|
expect(pushes).toHaveLength(1);
|
|
expect((await pushes[0]).ok).toBe(true);
|
|
writeFileSync(join(work, 'brain', 'b.md'), '# b\n');
|
|
await runHook(['stop'], {
|
|
write: () => {},
|
|
cwd: work,
|
|
spawnPush: realSpawnPush,
|
|
stdin: JSON.stringify({ session_id: 'ps-z2', cwd: work }),
|
|
});
|
|
expect(pushes).toHaveLength(2);
|
|
const res2 = await pushes[1];
|
|
expect(res2.ok).toBe(true);
|
|
const shipped = git(bare, 'ls-tree', '-r', '--name-only', 'main');
|
|
expect(shipped).toContain('brain/a.md');
|
|
expect(shipped).toContain('brain/b.md');
|
|
} finally {
|
|
delete process.env.GBRAIN_STOP_PUSH_DEBOUNCE_MIN;
|
|
}
|
|
}, 60_000);
|
|
});
|