Files
gbrain/test/e2e/bootstrap-persistence.serial.test.ts
Garry TanandClaude Fable 5 1ec6a6e842 v0.45.9.0 feat(bootstrap): first-class cloud-sandbox install + per-turn persistence + fail-closed privacy ladder (#4045)
* feat(core): execution-environment detection — local | cloud-sandbox | ephemeral-container

detectExecutionEnvironment() + isCredentialInjectingProxy() with injected
signals (CLAUDE_CODE_REMOTE, cse_ session-id prefix, proxy-injected token
placeholder, anthropic-egress proxy JWT, container markers). binaryOnPath
moves here as the canonical PATH probe. autopilot's detectInstallTarget
ephemeral branch now routes through the shared detector.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): repo-visibility ladder + per-turn Stop push + same-session failure banner

The durability lane. One repo-visibility verdict for every consumer (REST
first — never GraphQL, which sandbox proxies pin; authed ls-remote + an
attributed anonymous probe as the git-protocol fallback), replacing three
drifted probes. A 200 counts as public only with advertisement proof; a
401/404 counts as private-signal only with an auth challenge — fail-closed
in BOTH directions. Private verdicts cache 1h (private-only, per origin).

gbrain hook stop now spawns a debounced detached push per turn (per-root
state; cloud-sandbox defaults to every turn, elsewhere 5 min; a failing
status bypasses the debounce), closing the /exit and VM-reclaim gaps.
Push status is per workspace root, read through one shared reader by the
user-prompt banner (additionalContext + systemMessage — visible to the
human, not just the model), the SessionStart note, and doctor. Escape
hatches for self-hosted git: flag > env > file-plane config key.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): environment-aware install — honest cron skip, cloud repo-create guard, execution_env surfaces

installDurabilityCron probes for crontab before writing anything (containers
and cloud sandboxes ship without one — expected, reported as an honest skip
naming the event-driven pushes that still cover persistence). The repo phase
installs the container-friendly harden half (post-commit hook, no scheduler)
outside local machines. createPrivateRepo fails fast in cloud sandboxes with
the flow that works (create outside, open the session ON the repo, attach).
bootstrap verify gains a never-gating execution_env check; bootstrap status
--json carries execution_environment for installing agents to branch on.

Also fixes a live-PATH resolution class: binaryOnPath and the crontab execs
now pass the current env explicitly (Bun resolves against the startup
snapshot otherwise).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bootstrap): lifecycle hygiene — uninstall teardown, doctor job liveness, .mcp.json out of the repo, honest persistence copy

uninstall now tears down the durability wiring it installed (launchd/cron
job, untracked post-commit hook, credential wiring — the committed helper
and AGENTS rules stay). doctor gains bootstrap_durability_job: presence +
LIVENESS (launchctl load-state, crontab line, pull-log freshness) — a plist
on disk with a dead job no longer reads as healthy. Rendered .gitignore now
covers .mcp.json (absolute machine paths must not land in the private
repo); verify warns on pre-fix installs that committed it; the never-built
state/mcp.json promise is gone from GITHUB.md. Persistence copy tells the
truth everywhere: event-driven pushes do the durability work, the 30-min
job is a multi-machine pull freshener. Post-commit hook install/removal is
worktree-safe (git-path resolution; the git marker is a FILE there) and the
cron wrapper's self-disable tests the repo dir, not its git marker. Two
follow-up TODOs filed (plugin hook distribution; Channels push lane).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): committed hook carrier for cloud + cloud-setup-script emitter + cloud runbook

Cloud sessions clone fresh and snapshot hook config at session start — the
gitignored settings.local.json never exists there, so hooks never fired in
cloud at all. Cloud installs now write the repo-COMMITTED .claude/settings.json
with PATH-resolved, fail-open commands (no machine paths; a host without the
binary no-ops); local installs keep settings.local.json; the writers enforce
that one event never fires from both carriers, and removal cleans both.

New: gbrain bootstrap cloud-setup-script prints the paste-ready environment
setup script (npm transport — bun fetching is proxy-incompatible in cloud;
never the unrelated npm-registry package). Runbook gains a NEVER FABRICATE
TOOLING hard rule, a cloud-sandbox section (expected degradations as facts to
relay, the attach-first flow), and failure-table rows for the proxy-403 and
missing-crontab signatures. Codex lane stated honestly: AGENTS.md Gate 2 now
has the pull-side push-health check. Guide documents the new knobs and the
cloud contract; llms bundles regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(bootstrap): per-turn push e2e chain, cloud-sandbox sim, per-root status reader migration + registry regen

bootstrap-persistence e2e gains the Stop-hook chain (one stop banks the turn
to the real bare remote; the debounce holds across stops; debounce-0 lands
consecutive turns) and fixes a pre-existing post-#4024 break: repoPhaseComplete
required a github-parseable origin, so session-end pushes deferred FOREVER for
self-hosted/file-transport origins — non-github repo_urls now bind by exact
URL equality (redirect protection preserved). Degraded-modes e2e gains the
cloud-sandbox simulation (status reports the environment; repo creation
refuses with the attach flow). workspace-push tests read per-root status
through the shared reader. Flag registry regenerated (new git argv literals
from the git-path/ls-files calls — the accepted argv-bleed class).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(doctor): categorize bootstrap_durability_job (categories drift guard)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bootstrap): adversarial-review hardening — fail-closed privacy, no push-storm, no exfil paths

Both Claude and Codex adversarial passes ran; 11 findings fixed (union):
- Non-github 401+WWW-Authenticate no longer reads as private (RFC 7235 makes
  the header mandatory on every 401, so a middlebox 401s identically) — the
  top exfil path both models flagged; now unverifiable/fail-closed, operator
  confirms via the escape hatch. github.com still needs x-github-request-id.
- Escape hatches downgrade ONLY 'unverifiable' — a PROVEN-public origin still
  refuses (hatches never authorize a public push).
- treeNeedsPush measures against origin/<branch> (the push's own ref), not
  @{u}: a no-upstream branch no longer reports a committed-but-unpushed tree
  as push_clean and silently strands it.
- [D20] failing-retry uses a fixed 60s floor, not min(debounce,60s) — cloud
  debounce=0 no longer re-runs the network ladder every turn.
- Committed hook carrier: dedupe/suppress only on the EXACT portable-command
  shape, not a 'gbrain hook' substring (blocks the suppress-local-run-evil
  supply-chain vector); GBRAIN_HOME refused in the committed carrier.
- push-status reason sanitized (charset+length) at every surface (banner,
  doctor, status blob) so remote git stderr can't inject via the remediation.
- Per-root state: ghost-root records (deleted workspaces) filtered so a dead
  failing record can't re-fire the banner forever; uninstall removes them.
- statusReport support blob reads push status through the shared per-root
  reader; visibility cache strips URL userinfo (no PAT persisted).
- anonProbe strips userinfo + redirect:manual + SSRF flags + --end-of-options
  on ls-remote; cron self-disable uses git rev-parse (worktree-safe both
  ways); cloud-setup-script fails loud on a broken update; durability liveness
  won't certify a never-run crontab as live; config get/unset resolve the
  dotted file-plane keys; .bak/.broken gitignored; typed config fields.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* v0.45.8.0 feat(bootstrap): first-class cloud-sandbox install + per-turn persistence + fail-closed privacy ladder

VERSION + package.json + CHANGELOG + TODOS + llms bundles.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: sync KEY_FILES + README for the cloud-DX wave (v0.45.8.0)

New KEY_FILES entries for execution-env.ts and repo-visibility.ts; updated the
bootstrap repo/hooks, workspace-push, brain-repo-durability, and hook.ts
entries to current behavior (ladder verification, committed cloud hook carrier,
per-root push status, crontab probe + liveness, per-turn stop push + banner).
README's Claude Code line now states per-turn + cloud persistence honestly.
llms bundles regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bootstrap): doc-review pass — attach records repo_url (enables cloud persistence), verify reads per-root push status, honest docs

Cross-model doc review (Codex) caught that the headline cloud-persistence
claim was hollow and two docs over-claimed:
- attach now records repo_url from the adopted origin, so the no-daemon push
  gate (repoPhaseComplete) recognizes the repo phase as done — WITHOUT this,
  the per-turn/session-end pushes deferred forever after an attach, which is
  the ONLY install path in a cloud sandbox (repo is refused there). Privacy is
  still enforced at push time by the ladder.
- bootstrap verify's push_probe reads the shared per-root reader [D8], not the
  legacy single file — a fresh v0.45.8 install no longer reports 'no push
  recorded' when per-root status exists.
- Docs corrected to match code: runbook stamp → 0.45.8.0; README states the
  per-turn cadence honestly (debounced local, next-turn failure notice);
  GITHUB.md qualifies auto-push as Claude Code (Codex is pull); KEY_FILES says
  repo/status use REST (not the full ladder) and drops the stale --push;
  bootstrap.md hooks-location covers both carriers; README verbs surface says
  seven. llms bundles regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bootstrap): regenerate vendored template-repo tree + isolate env-mutating tests (verify gate)

The verify CI gate caught two things the piecemeal local runs missed:
- templates/bootstrap/template-repo/ is a GENERATED tree; the source-template
  edits (AGENTS.md Gate 2, CLAUDE.md cloud note) plus the version stamp had
  drifted. Regenerated from source (check:bootstrap-templates green).
- test/bootstrap-repo.test.ts + test/durability-cron.test.ts newly mutated
  process.env in non-serial files; converted to withEnv() (check:test-isolation
  green) — no .serial rename needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(ambient-recall): pin a keyless gateway so delta/context_pack writes never fire a real embed

Root cause of the shard-2 CI red after the 0.45.9.0 re-bump: adding this
wave's test files reshuffled the weight-packed shards, moving
ambient-recall.test.ts next to a neighbor that leaks CI's dummy
OPENAI_API_KEY (sk-test-*) into the gateway singleton (the bunfig preload
configures with env:{...process.env}, and a present-but-invalid key turns
remember's keyless-degrade embed into a hard 401). The delta/context_pack
tests exercise cursor + budget logic, not embedding quality, so this pins a
keyless gateway (env:{}) in beforeAll — isAvailable('embedding') is false,
writeSingleFact degrades (degraded_dedup) with no HTTP call, and the file is
deterministic regardless of shard bin-packing. Verified: passes with
OPENAI_API_KEY=sk-test-* set (the CI condition). Master's own comment in
legacy-embedding-preload.ts already warns 'adding any test file reshuffles
the mines'; this makes the file immune.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 16:12:59 -07:00

255 lines
11 KiB
TypeScript

/**
* GAP 7 — session-end persistence chain, UNMOCKED end-to-end.
*
* `test/hook-command.serial.test.ts` proves session-end up to the mocked
* `io.spawnPush` seam (it only records that a push WOULD fire). This file
* closes the remaining gap: it drives the session-end hook against a REAL
* local bare git remote with the REAL push implementation — the same
* `workspacePush` the detached `gbrain sources push` child runs — so the full
* hook → scan → commit → push chain is exercised and its outcome verified on
* the remote.
*
* The only concession to a hermetic test is `allowUnverifiedRemote: true`:
* `workspacePush`'s privacy gate can only confirm PRIVATE visibility for a
* real GitHub origin via `gh`, which a `file://` bare remote isn't — every
* sibling `workspace-push.serial.test.ts` case makes the same concession. The
* scan/commit/push machinery under test is otherwise identical to production.
*
* Serial: mutates HOME/GBRAIN_HOME + spawns git subprocesses. All secrets are
* synthetic fixtures; GBRAIN_GIT_ALLOW_FILE_TRANSPORT=1 permits file transport.
*/
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
import { mkdtempSync, mkdirSync, rmSync, writeFileSync, existsSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import { tmpdir } from 'node:os';
import { execFileSync } from 'node:child_process';
import { runHook } from '../../src/commands/hook.ts';
import { workspacePush, type WorkspacePushResult } from '../../src/core/workspace-push.ts';
import { writeManifest, writeReceipt } from '../../src/core/bootstrap/format.ts';
const OPENAI_KEY = 'sk-' + 'A1b2C3d4E5f6G7h8I9j0K1l2M3n4';
const ENV_KEYS = ['HOME', 'GBRAIN_HOME', 'DATABASE_URL', 'GBRAIN_DATABASE_URL', 'GBRAIN_HOOKS', 'GBRAIN_GIT_ALLOW_FILE_TRANSPORT'] as const;
// #2943: env: process.env is REQUIRED — Bun snapshots env at startup, so a
// spawned git would otherwise be blind to beforeEach's HOME/GBRAIN_HOME swap.
function git(cwd: string, ...args: string[]): string {
return execFileSync('git', ['-C', cwd, '-c', 'protocol.file.allow=always', ...args], {
stdio: ['ignore', 'pipe', 'pipe'], encoding: 'utf-8', env: process.env,
}).trim();
}
function originHead(bare: string): string {
return git(bare, 'rev-parse', 'refs/heads/main');
}
const MANIFEST = {
format_version: 1 as const,
initialized: true as const,
agent_name: 'persist-test',
created_by: 'test',
created_at: '2026-01-01T00:00:00.000Z',
source_id: 'workspace',
};
let root: string;
let bare: string;
let work: string;
let saved: Record<string, string | undefined>;
/** Real push path, in-process: exactly what `gbrain sources push --path`
* invokes. Captured so the test can await the chain the hook kicks off. */
let pushes: Promise<WorkspacePushResult>[];
const realSpawnPush = (r: string) => {
pushes.push(workspacePush({ dir: r, branch: 'main', allowUnverifiedRemote: true }));
};
beforeEach(() => {
root = mkdtempSync(join(tmpdir(), 'gb-persist-'));
saved = {};
for (const k of ENV_KEYS) saved[k] = process.env[k];
for (const k of ['DATABASE_URL', 'GBRAIN_DATABASE_URL', 'GBRAIN_HOOKS'] as const) delete process.env[k];
process.env.HOME = mkdtempSync(join(root, 'home-'));
// CX2-8: GBRAIN_HOME is a PARENT dir → effective home is $HOME/.gbrain.
process.env.GBRAIN_HOME = process.env.HOME;
process.env.GBRAIN_GIT_ALLOW_FILE_TRANSPORT = '1';
bare = mkdtempSync(join(root, 'origin-')) + '.git';
execFileSync('git', ['init', '-q', '--bare', '-b', 'main', bare], { stdio: 'ignore', env: process.env });
work = mkdtempSync(join(root, 'work-'));
execFileSync('git', ['-c', 'protocol.file.allow=always', 'clone', '-q', bare, work], { stdio: 'ignore', env: process.env });
git(work, 'config', 'user.email', 't@t.t');
git(work, 'config', 'user.name', 'tester');
writeFileSync(join(work, 'README.md'), 'init\n');
git(work, 'add', 'README.md');
git(work, 'commit', '-qm', 'init');
git(work, 'push', '-q', 'origin', 'main');
try { git(work, 'remote', 'set-head', 'origin', 'main'); } catch { /* */ }
// The initialized manifest is the security boundary the hook gates on.
writeManifest(work, MANIFEST);
// Repo phase complete: the receipt binds this workspace to its verified
// origin (exact-URL binding for non-github transports) — without it the
// no-daemon pushes defer by design (create-repo-first race protection).
const toplevel = git(work, 'rev-parse', '--show-toplevel');
mkdirSync(join(process.env.HOME!, '.gbrain', 'bootstrap'), { recursive: true });
writeReceipt(join(process.env.HOME!, '.gbrain'), {
receipt_version: 1,
workspace_dir: toplevel,
source_id: 'workspace',
agent_name: 'persist-test',
created_at: '2026-01-01T00:00:00.000Z',
created_by: 'test',
brain_created_by_bootstrap: false,
created_paths: [],
registrations: [],
repo_url: bare,
} as Parameters<typeof writeReceipt>[1] & { repo_url: string });
pushes = [];
});
afterEach(() => {
for (const k of ENV_KEYS) {
if (saved[k] === undefined) delete process.env[k];
else process.env[k] = saved[k];
}
rmSync(root, { recursive: true, force: true });
});
describe('session-end → real workspace push', () => {
test('authored session content actually lands on the bare remote', async () => {
// The agent authored real work into the brain during the session.
mkdirSync(join(work, 'brain'), { recursive: true });
const authored = 'the acme-example roadmap decision: ship the memory layer in Q3';
writeFileSync(join(work, 'brain', 'decision.md'), `# decision\n\n${authored}\n`);
const before = originHead(bare);
const code = await runHook(['session-end'], {
write: () => {},
cwd: work,
spawnPush: realSpawnPush,
stdin: JSON.stringify({ session_id: 'persist-ok', cwd: work }),
});
expect(code).toBe(0);
// The hook kicked off exactly one real push; await its completion.
expect(pushes).toHaveLength(1);
const res = await pushes[0];
expect(res.status).toBe('pushed');
expect(res.ok).toBe(true);
expect(res.committed).toBe(true);
// Origin advanced, and the authored file+content are physically there.
expect(originHead(bare)).not.toBe(before);
const verify = mkdtempSync(join(root, 'verify-'));
execFileSync('git', ['-c', 'protocol.file.allow=always', 'clone', '-q', bare, verify], { stdio: 'ignore', env: process.env });
const landed = join(verify, 'brain', 'decision.md');
expect(existsSync(landed)).toBe(true);
expect(readFileSync(landed, 'utf-8')).toContain(authored);
}, 60_000);
test('a planted secret in authored content is BLOCKED at the push gate — nothing leaves', async () => {
mkdirSync(join(work, 'brain'), { recursive: true });
// A genuine note plus a file that leaks a synthetic OpenAI key.
writeFileSync(join(work, 'brain', 'safe.md'), '# safe\n\nno secrets here\n');
writeFileSync(join(work, 'brain', 'leak.md'), `# oops\n\napi key: ${OPENAI_KEY}\n`);
const before = originHead(bare);
const code = await runHook(['session-end'], {
write: () => {},
cwd: work,
spawnPush: realSpawnPush,
stdin: JSON.stringify({ session_id: 'persist-leak', cwd: work }),
});
expect(code).toBe(0);
expect(pushes).toHaveLength(1);
const res = await pushes[0];
// The scan gate fired: nothing committed, nothing pushed, secret named.
expect(res.status).toBe('blocked_secrets');
expect(res.ok).toBe(false);
expect(res.findings?.some((f) => f.file === 'brain/leak.md' && f.pattern === 'openai')).toBe(true);
// The secret VALUE never surfaces in the result payload.
expect(JSON.stringify(res).includes(OPENAI_KEY)).toBe(false);
// Origin is untouched — the leak never left the machine.
expect(originHead(bare)).toBe(before);
const shipped = git(bare, 'ls-tree', '-r', '--name-only', 'main');
expect(shipped).not.toContain('brain/leak.md');
expect(shipped).not.toContain('brain/safe.md'); // blocked atomically — nothing in the batch shipped
}, 60_000);
});
// ── per-turn Stop push [D3]: the /exit + VM-reclaim durability lane ─────────
//
// SessionEnd never fires on /exit; the Stop hook fires after EVERY assistant
// turn. This chain proves a turn's authored work physically lands on the real
// bare remote from a single `gbrain hook stop`, and that the per-root debounce
// holds across consecutive stops.
describe('stop → real workspace push (per-turn durability)', () => {
test('one stop banks the turn to origin; the next stop inside the window debounces', async () => {
process.env.GBRAIN_STOP_PUSH_DEBOUNCE_MIN = '60';
try {
mkdirSync(join(work, 'brain'), { recursive: true });
writeFileSync(join(work, 'brain', 'turn-note.md'), '# turn\n\nlearned during the turn\n');
const before = originHead(bare);
const code = await runHook(['stop'], {
write: () => {},
cwd: work,
spawnPush: realSpawnPush,
stdin: JSON.stringify({ session_id: 'persist-stop', cwd: work }),
});
expect(code).toBe(0);
expect(pushes).toHaveLength(1);
const res = await pushes[0];
expect(res.ok).toBe(true);
expect(res.status).toBe('pushed');
expect(originHead(bare)).not.toBe(before);
// Second stop, same window, clean tree — debounced, no second spawn.
const code2 = await runHook(['stop'], {
write: () => {},
cwd: work,
spawnPush: realSpawnPush,
stdin: JSON.stringify({ session_id: 'persist-stop-2', cwd: work }),
});
expect(code2).toBe(0);
expect(pushes).toHaveLength(1);
} finally {
delete process.env.GBRAIN_STOP_PUSH_DEBOUNCE_MIN;
}
}, 60_000);
test('debounce 0 (the cloud-sandbox default): consecutive dirty turns both land', async () => {
process.env.GBRAIN_STOP_PUSH_DEBOUNCE_MIN = '0';
try {
mkdirSync(join(work, 'brain'), { recursive: true });
writeFileSync(join(work, 'brain', 'a.md'), '# a\n');
await runHook(['stop'], {
write: () => {},
cwd: work,
spawnPush: realSpawnPush,
stdin: JSON.stringify({ session_id: 'ps-z1', cwd: work }),
});
expect(pushes).toHaveLength(1);
expect((await pushes[0]).ok).toBe(true);
writeFileSync(join(work, 'brain', 'b.md'), '# b\n');
await runHook(['stop'], {
write: () => {},
cwd: work,
spawnPush: realSpawnPush,
stdin: JSON.stringify({ session_id: 'ps-z2', cwd: work }),
});
expect(pushes).toHaveLength(2);
const res2 = await pushes[1];
expect(res2.ok).toBe(true);
const shipped = git(bare, 'ls-tree', '-r', '--name-only', 'main');
expect(shipped).toContain('brain/a.md');
expect(shipped).toContain('brain/b.md');
} finally {
delete process.env.GBRAIN_STOP_PUSH_DEBOUNCE_MIN;
}
}, 60_000);
});