mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-14 08:53:22 +00:00
Compare commits
15
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5c648f419c | ||
|
|
a2fb5b2fec | ||
|
|
6a1fe58e41 | ||
|
|
938d005ad5 | ||
|
|
71f7e08c74 | ||
|
|
95c16c4718 | ||
|
|
399b59bd42 | ||
|
|
f6de957311 | ||
|
|
ad74ac5848 | ||
|
|
9775fbc689 | ||
|
|
bb3de0046e | ||
|
|
f08595453e | ||
|
|
f2cdeb39a9 | ||
|
|
29a2d7a102 | ||
|
|
6e740590b3 |
@@ -1,6 +1,94 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
## [0.22.7] - 2026-04-28
|
||||
|
||||
## **Built-in HTTP transport with bearer auth for remote MCP.**
|
||||
## **Postgres-backed tokens, default-deny CORS, two-bucket rate limit, body cap, per-request audit.**
|
||||
|
||||
v0.22.7 ships `gbrain serve --http`: a built-in HTTP transport for remote MCP, authenticating via the existing `access_tokens` table that `gbrain auth create/list/revoke` already manages. Bearer-only, no OAuth surface, no registration endpoint, no self-service tokens. SECURITY.md is the canonical reference for the hardening posture and recommended deployment.
|
||||
|
||||
The hardening lives inside the transport, not in the doc:
|
||||
|
||||
| Layer | Default | Configurable via |
|
||||
|---|---|---|
|
||||
| CORS | default-deny (no `Access-Control-Allow-Origin`) | `GBRAIN_HTTP_CORS_ORIGIN=a.com,b.com` |
|
||||
| Pre-auth IP rate limit | 30 req / 60s | `GBRAIN_HTTP_RATE_LIMIT_IP` |
|
||||
| Post-auth token rate limit | 60 req / 60s | `GBRAIN_HTTP_RATE_LIMIT_TOKEN` |
|
||||
| Body cap | 1 MiB, stream-counted | `GBRAIN_HTTP_MAX_BODY_BYTES` |
|
||||
| `last_used_at` debounce | once per token per 60s | (SQL-level WHERE clause, race-tolerant) |
|
||||
| Per-request audit | `mcp_request_log` row per `/mcp` | (existing schema, since v4) |
|
||||
| Reverse-proxy trust | off | `GBRAIN_HTTP_TRUST_PROXY=1` to honor X-Forwarded-For |
|
||||
|
||||
The IP rate-limit fires **before** the auth lookup so the limit caps load on the auth path itself, not just response codes. The token-id rate limit fires after auth so a runaway authenticated client gets throttled at the right principal. Both buckets live in a bounded LRU map (default 10K keys, TTL prune at 2× window) so unique-key growth can't drift into memory pressure.
|
||||
|
||||
### What changed for users
|
||||
|
||||
You can now expose GBrain remotely with the built-in transport:
|
||||
|
||||
```bash
|
||||
gbrain auth create my-laptop # tokens managed via the existing CLI
|
||||
gbrain serve --http --port 8787 # Postgres-only; PGLite users see a clear fail-fast
|
||||
ngrok http 8787 --url your-brain.ngrok.app # any tunnel works
|
||||
```
|
||||
|
||||
Then point Claude Desktop, claude.ai/code, or any MCP client at `http://your-tunnel/mcp` with `Authorization: Bearer <token>`. CORS, rate limits, and body caps are on by default. `gbrain auth` is now wired into the main CLI, so it works from the compiled binary the same as `gbrain doctor` or `gbrain serve`.
|
||||
|
||||
### For contributors
|
||||
|
||||
- `src/mcp/dispatch.ts` (new) — shared `dispatchToolCall(engine, name, params, opts)` consumed by both stdio (`server.ts`) and HTTP (`http-transport.ts`). One source of truth for `validateParams`, `OperationContext` construction, and handler invocation, so the two transports can't drift apart.
|
||||
- `src/mcp/rate-limit.ts` (new) — bounded-LRU token-bucket. Tracks `lastTouchedMs` separately from `lastRefillMs` so an exhausted key can't be reset by hammering past the TTL.
|
||||
- `src/mcp/http-transport.ts` — built on the new dispatch + rate-limit modules. `application/json` response shape (gbrain MCP tools are synchronous; the Streamable HTTP transport spec allows JSON for non-streaming responses).
|
||||
- `src/cli.ts` + `src/commands/auth.ts` — `auth` is now a wired CLI subcommand. Direct-script usage (`bun run src/commands/auth.ts ...`) still works for environments without a compiled binary.
|
||||
- 23 unit cases in `test/http-transport.test.ts`, 8 E2E cases in `test/e2e/http-transport.test.ts`. Unit covers the full dispatch round-trip with a real operation; E2E covers `last_used_at` debounce against real Postgres semantics.
|
||||
|
||||
### Known limits
|
||||
|
||||
- `gbrain serve --http` is **Postgres-only**. PGLite has no `access_tokens` or `mcp_request_log` table by design (`src/core/pglite-schema.ts:5-6`). Local agents continue to use stdio (`gbrain serve`).
|
||||
- Behind a tunnel (ngrok, Tailscale Funnel, Cloudflare Tunnel), all requests share one egress IP. The pre-auth IP bucket becomes effectively shared by all clients on that tunnel; the token-id bucket is the load-bearing limiter for tunnel deployments. Documented in SECURITY.md.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
- New: `gbrain serve --http [--port N]` ships the built-in HTTP transport
|
||||
- New: `gbrain auth create/list/revoke/test` wired into the main CLI (was a standalone script)
|
||||
- New: SECURITY.md documents the disclosure path, the recommended remote-MCP setup, and the full hardening reference
|
||||
- New: `src/mcp/dispatch.ts` — shared dispatch path for stdio + HTTP
|
||||
- New: `src/mcp/rate-limit.ts` — bounded-LRU token-bucket limiter
|
||||
- Hardening: CORS default-deny, two-bucket rate limit (per-IP pre-auth + per-token post-auth), 1 MiB body cap with stream-counted enforcement, `mcp_request_log` per-request audit, `last_used_at` SQL-level debounce
|
||||
- Tests: 23 unit + 8 E2E covering auth, dispatch, CORS, body cap, rate limit, and audit
|
||||
- Docs: SECURITY.md, DEPLOY.md, and per-client setup guides updated to recommend `--http` and document the env vars
|
||||
|
||||
## To take advantage of v0.22.7
|
||||
|
||||
`gbrain upgrade` should do this automatically. If it didn't, or if you want to expose your brain over HTTP:
|
||||
|
||||
1. **Confirm migrations are at v4 or higher** (the `access_tokens` + `mcp_request_log` tables were added in migration v4):
|
||||
```bash
|
||||
gbrain doctor # schema_version check should pass
|
||||
gbrain apply-migrations --yes # if not, run this
|
||||
```
|
||||
2. **Create a token for each remote client:**
|
||||
```bash
|
||||
gbrain auth create my-laptop # prints the token once — copy it
|
||||
```
|
||||
3. **Start the HTTP server:**
|
||||
```bash
|
||||
gbrain serve --http --port 8787
|
||||
```
|
||||
4. **(Optional) configure CORS allowlist if a browser client will hit it:**
|
||||
```bash
|
||||
GBRAIN_HTTP_CORS_ORIGIN=https://claude.ai gbrain serve --http --port 8787
|
||||
```
|
||||
5. **(Optional) audit who's hitting your brain:**
|
||||
```bash
|
||||
psql $DATABASE_URL -c "SELECT created_at, token_name, operation, status, latency_ms
|
||||
FROM mcp_request_log ORDER BY created_at DESC LIMIT 50"
|
||||
```
|
||||
6. **If `gbrain serve --http` exits with "Postgres engine required":** PGLite is local-only by design. Either keep using stdio (`gbrain serve`) for local agents, or migrate to Postgres (`gbrain migrate --to supabase`).
|
||||
|
||||
If anything breaks: `gbrain doctor`, `~/.gbrain/upgrade-errors.jsonl` (if present), and please file an issue at https://github.com/garrytan/gbrain/issues with both.
|
||||
|
||||
|
||||
|
||||
## [0.22.6.1] - 2026-04-26
|
||||
|
||||
|
||||
@@ -91,8 +91,11 @@ strict behavior when unset.
|
||||
- `src/commands/jobs.ts` — `gbrain jobs` CLI subcommands + `gbrain jobs work` daemon. v0.13.1 surfaces the full `MinionJobInput` retry/backoff/timeout/idempotency surface as first-class CLI flags on `jobs submit`: `--max-stalled`, `--backoff-type fixed|exponential`, `--backoff-delay`, `--backoff-jitter`, `--timeout-ms`, `--idempotency-key`. `jobs smoke --sigkill-rescue` is the opt-in regression guard for #219. v0.16 wires `registerBuiltinHandlers` to always register `subagent` + `subagent_aggregator` (no env flag — `ANTHROPIC_API_KEY` is the natural cost gate, trust is via `PROTECTED_JOB_NAMES`) and loads `GBRAIN_PLUGIN_PATH` plugins at worker startup with a loud startup-line per plugin. `shell` handler still gated by `GBRAIN_ALLOW_SHELL_JOBS=1` (RCE surface, separate concern).
|
||||
- `src/commands/features.ts` — `gbrain features --json --auto-fix`: usage scan + feature adoption salesman
|
||||
- `src/commands/autopilot.ts` — `gbrain autopilot --install`: self-maintaining brain daemon (sync+extract+embed)
|
||||
- `src/mcp/server.ts` — MCP stdio server (generated from operations)
|
||||
- `src/commands/auth.ts` — Standalone token management (create/list/revoke/test)
|
||||
- `src/mcp/server.ts` — MCP stdio server (generated from operations). v0.22.7: tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path.
|
||||
- `src/mcp/dispatch.ts` (v0.22.7) — Shared tool-call dispatch consumed by both stdio (`server.ts`) and HTTP (`http-transport.ts`). Exports `dispatchToolCall(engine, name, params, opts)`, `buildOperationContext(engine, params, opts)`, and `validateParams(op, params)`. Single source of truth for `(ctx, params)` handler arg order and the 5-field `OperationContext` shape (engine + config + logger + dryRun + remote). Defaults to `remote: true` (untrusted); local CLI callers pass `remote: false`. Closed F1 (reversed handler args) + F2 (incomplete OperationContext) + F3 (no param validation) drift bugs in the original v0.22.5 HTTP transport.
|
||||
- `src/mcp/rate-limit.ts` (v0.22.7) — Bounded-LRU token-bucket limiter for `gbrain serve --http`. `buildDefaultLimiters()` returns the two-bucket pipeline used by http-transport: pre-auth IP (default 30/60s, fires BEFORE the DB lookup so brute-force load against `access_tokens` is actually capped) + post-auth token-id (default 60/60s). Tracks `lastTouchedMs` separately from `lastRefillMs` so an exhausted key can't be reset by hammering past the TTL. LRU cap (default 10K keys) bounds memory under attacker-controlled key growth; TTL prune at 2× window evicts abandoned buckets.
|
||||
- `src/mcp/http-transport.ts` (v0.22.7, rewrite) — `gbrain serve --http` HTTP transport. Postgres-only — fails fast at startup on PGLite (the `access_tokens` table only exists on Postgres). Bearer auth against SHA-256 hashes in `access_tokens`. CORS default-deny via `GBRAIN_HTTP_CORS_ORIGIN` allowlist. Body cap stream-counted (1 MiB default via `GBRAIN_HTTP_MAX_BODY_BYTES`) so chunked transfers without Content-Length still hit the cap. `last_used_at` SQL-level debounce (one UPDATE per token per 60s). Per-request audit row in `mcp_request_log` with token_name + operation + status + latency. Optional `GBRAIN_HTTP_TRUST_PROXY=1` honors `X-Forwarded-For` — only safe when bound to a private interface AND the proxy strips client-supplied XFF (otherwise enables IP spoofing past the pre-auth rate limit). `/health` does `SELECT 1` against Postgres and returns 503 + `status:unhealthy` when the DB is unreachable so orchestration doesn't see green pods while clients get misleading 401s. Replaces the standalone OAuth wrapper that was vulnerable to unauthenticated client registration.
|
||||
- `src/commands/auth.ts` — Token management for the HTTP transport. `gbrain auth create/list/revoke/test`. As of v0.22.7 wired into the main CLI (`src/cli.ts`); also runs standalone via `bun run src/commands/auth.ts ...` for environments without a compiled binary. Tokens stored as SHA-256 hashes in `access_tokens` (Postgres-only).
|
||||
- `src/commands/upgrade.ts` — Self-update CLI. `runPostUpgrade()` enumerates migrations from the TS registry (src/commands/migrations/index.ts) and tail-calls `runApplyMigrations(['--yes', '--non-interactive'])` so the mechanical side of every outstanding migration runs unconditionally.
|
||||
- `src/commands/migrations/` — TS migration registry (compiled into the binary; no filesystem walk of `skills/migrations/*.md` needed at runtime). `index.ts` lists migrations in semver order. `v0_11_0.ts` = Minions adoption orchestrator (8 phases). `v0_12_0.ts` = Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify). `phaseASchema` has a 600s timeout (bumped from 60s in v0.12.1 for duplicate-heavy brains). `v0_12_2.ts` = JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record). `v0_14_0.ts` = shell-jobs + autopilot cooperative (2 phases: schema ALTER minion_jobs.max_stalled SET DEFAULT 3 — superseded by v0.14.3's schema-level DEFAULT 5 + UPDATE backfill; pending-host-work ping for skills/migrations/v0.14.0.md). All orchestrators are idempotent and resumable from `partial` status. As of v0.14.2 (Bug 3), the RUNNER owns all ledger writes — orchestrators return `OrchestratorResult` and `apply-migrations.ts` persists a canonical `{version, status, phases}` shape after return. Orchestrators no longer call `appendCompletedMigration` directly. `statusForVersion` prefers `complete` over `partial` (never regresses). 3 consecutive partials → wedged → `--force-retry <version>` writes a `'retry'` reset marker. v0.14.3 (fix wave) ships schema-only migrations v14 (`pages_updated_at_index`) + v15 (`minion_jobs_max_stalled_default_5` with UPDATE backfill) via the `MIGRATIONS` array in `src/core/migrate.ts` — no orchestrator phases needed.
|
||||
- `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent.
|
||||
@@ -277,7 +280,8 @@ parity), `test/cli.test.ts` (CLI structure), `test/config.test.ts` (config redac
|
||||
`test/skill-manifest.test.ts` (v0.19 skill manifest parser: drift detection, managed-block markers),
|
||||
`test/skillify-scaffold.test.ts` (v0.19 `gbrain skillify scaffold` stubs: SKILL.md, script, tests, routing-eval fixtures),
|
||||
`test/skillpack-install.test.ts` (v0.19 `gbrain skillpack install` managed-block install / update / no-clobber semantics),
|
||||
`test/skillpack-sync-guard.test.ts` (v0.19 sync-guard: bundled skills stay byte-identical to `skills/` source).
|
||||
`test/skillpack-sync-guard.test.ts` (v0.19 sync-guard: bundled skills stay byte-identical to `skills/` source),
|
||||
`test/http-transport.test.ts` (v0.22.7 HTTP transport: 23 unit cases covering bearer auth + missing/no-Bearer/unknown/revoked + `/health` bypass, F1+F2 round-trip via dispatch.ts, F3 invalid_params, application/json response shape (not SSE), CORS default-deny + allowlist, body cap on Content-Length AND chunked, two-bucket rate limit (refill, exhaust+Retry-After, LRU eviction, TTL prune, pre-auth IP fires before DB), and `mcp_request_log` audit on success + auth_failed).
|
||||
|
||||
E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_URL`.
|
||||
- `bun run test:e2e` runs Tier 1 (mechanical, all operations, no API keys). Includes 9 dedicated cases for the postgres-engine `addLinksBatch` / `addTimelineEntriesBatch` bind path — postgres-js's `unnest()` binding is structurally different from PGLite's and gets its own coverage.
|
||||
@@ -292,6 +296,7 @@ E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_U
|
||||
- `test/e2e/search-exclude.test.ts` (v0.22.0) — verifies `test/` + `archive/` pages are hidden by default, that `include_slug_prefixes` opts back in, and that caller-supplied `exclude_slug_prefixes` adds to defaults. Both keyword and vector search paths covered.
|
||||
- `test/e2e/engine-parity.test.ts` (v0.22.0) — Postgres ↔ PGLite top-result and result-set parity for `searchKeyword` + `searchVector`. Codex flagged that Postgres ranks pages then picks best chunk while PGLite returns chunks directly — without parity coverage the source-boost fix could pass on PGLite and fail on Postgres. Skips gracefully when `DATABASE_URL` is unset.
|
||||
- `test/e2e/postgres-bootstrap.test.ts` (v0.22.6.1) — exercises `PostgresEngine.initSchema()` directly against a fresh real Postgres database. Asserts the bootstrap path is no-op on fresh installs and that SCHEMA_SQL replays cleanly through the engine path (not via the standalone `db.initSchema` from `src/core/db.ts`, which would have produced false-positive coverage). Codex caught the E2E-shape gap during plan review.
|
||||
- `test/e2e/http-transport.test.ts` (v0.22.7) — 8 cases against real Postgres covering `gbrain serve --http` end-to-end: bearer auth round-trip, `last_used_at` SQL-level debounce semantics, `mcp_request_log` row insertion on success and auth_failed paths, `/health` DB-down → 503 (DB-probing health check), and the F1+F2+F3 dispatch round-trip with a real operation. Skips gracefully when `DATABASE_URL` is unset.
|
||||
- Tier 2 (`skills.test.ts`) requires OpenClaw + API keys, runs nightly in CI
|
||||
- If `.env.testing` doesn't exist in this directory, check sibling worktrees for one:
|
||||
`find ../ -maxdepth 2 -name .env.testing -print -quit` and copy it here if found.
|
||||
|
||||
@@ -80,12 +80,13 @@ Add to `~/.claude/server.json` (Claude Code), Settings > MCP Servers (Cursor), o
|
||||
### Remote MCP (Claude Desktop, Cowork, Perplexity)
|
||||
|
||||
```bash
|
||||
ngrok http 8787 --url your-brain.ngrok.app
|
||||
bun run src/commands/auth.ts create "claude-desktop"
|
||||
gbrain auth create "claude-desktop" # tokens via the existing CLI
|
||||
gbrain serve --http --port 8787 # built-in HTTP transport (Postgres-only)
|
||||
ngrok http 8787 --url your-brain.ngrok.app # any tunnel works
|
||||
claude mcp add gbrain -t http https://your-brain.ngrok.app/mcp -H "Authorization: Bearer TOKEN"
|
||||
```
|
||||
|
||||
Per-client guides: [`docs/mcp/`](docs/mcp/DEPLOY.md). ChatGPT requires OAuth 2.1 (not yet implemented).
|
||||
Per-client guides: [`docs/mcp/`](docs/mcp/DEPLOY.md). Hardening defaults, env vars, and threat model: [SECURITY.md](SECURITY.md). ChatGPT requires OAuth 2.1 (not yet implemented).
|
||||
|
||||
### Using gbrain with GStack
|
||||
|
||||
@@ -657,6 +658,8 @@ ADMIN
|
||||
gbrain doctor --locks List idle-in-tx backends (57014 diagnostic, Postgres only)
|
||||
gbrain stats Brain statistics
|
||||
gbrain serve MCP server (stdio)
|
||||
gbrain serve --http --port 8787 MCP server (HTTP, Postgres-only, bearer auth)
|
||||
gbrain auth create|list|revoke|test Token management for the HTTP transport
|
||||
gbrain integrations Integration recipe dashboard
|
||||
gbrain sources list|add|remove|... Multi-source brain management (v0.18)
|
||||
gbrain dream [--dry-run] [--phase N] One maintenance cycle then exit (cron-friendly)
|
||||
|
||||
+168
@@ -0,0 +1,168 @@
|
||||
# Security
|
||||
|
||||
## Reporting Vulnerabilities
|
||||
|
||||
If you discover a security issue in GBrain, please report it privately by opening
|
||||
a [private security advisory](https://github.com/garrytan/gbrain/security/advisories/new)
|
||||
on GitHub.
|
||||
|
||||
Do not open a public issue for security vulnerabilities.
|
||||
|
||||
## Remote MCP Security
|
||||
|
||||
### ⚠️ Do NOT use open OAuth client registration for remote MCP
|
||||
|
||||
If you deploy GBrain's MCP server behind an HTTP wrapper with OAuth 2.1
|
||||
support, **never allow unauthenticated client registration**. An attacker
|
||||
who discovers your server URL can:
|
||||
|
||||
1. Register a new OAuth client via `POST /register`
|
||||
2. Use `client_credentials` grant to obtain a bearer token
|
||||
3. Access all brain data via the MCP tools
|
||||
|
||||
### Recommended: `gbrain serve --http`
|
||||
|
||||
As of v0.22.7, GBrain ships a built-in HTTP transport that uses the
|
||||
existing `access_tokens` table for authentication:
|
||||
|
||||
```bash
|
||||
# Create a token
|
||||
gbrain auth create "my-client"
|
||||
|
||||
# Start the HTTP server
|
||||
gbrain serve --http --port 8787
|
||||
|
||||
# Connect via ngrok, Tailscale, or any tunnel
|
||||
ngrok http 8787 --url your-brain.ngrok.app
|
||||
```
|
||||
|
||||
This is the recommended way to expose GBrain remotely. No OAuth, no
|
||||
registration endpoint, no self-service tokens. Tokens are managed
|
||||
exclusively via `gbrain auth create/list/revoke`.
|
||||
|
||||
### If you must use a custom HTTP wrapper
|
||||
|
||||
1. **Require a secret for client registration** — check a header or body
|
||||
parameter before creating new OAuth clients
|
||||
2. **Disable `client_credentials` grant** — only allow `authorization_code`
|
||||
with browser-based approval
|
||||
3. **Restrict scopes** — never issue tokens with unlimited scope
|
||||
4. **Log all token issuance** — alert on unexpected registrations
|
||||
5. **Rate-limit registration and token endpoints**
|
||||
|
||||
### Token Management
|
||||
|
||||
```bash
|
||||
gbrain auth create "claude-desktop" # Create a new token
|
||||
gbrain auth list # List all tokens
|
||||
gbrain auth revoke "claude-desktop" # Revoke a token
|
||||
gbrain auth test <url> --token <tok> # Smoke-test a remote server
|
||||
```
|
||||
|
||||
Tokens are stored as SHA-256 hashes in the `access_tokens` table. The
|
||||
plaintext token is shown once at creation and never stored.
|
||||
|
||||
## `gbrain serve --http` hardening (v0.22.7+)
|
||||
|
||||
The built-in HTTP transport ships with several layers of hardening on by
|
||||
default. All env vars below are optional; the defaults are intentionally
|
||||
conservative.
|
||||
|
||||
### Postgres-only
|
||||
|
||||
`gbrain serve --http` requires a Postgres engine. PGLite is local-only by
|
||||
design and the `access_tokens` / `mcp_request_log` tables don't exist in
|
||||
the PGLite schema. Local agents continue to use stdio (`gbrain serve`).
|
||||
Running `--http` against a PGLite-backed install fails fast with a clear
|
||||
error message at startup.
|
||||
|
||||
### CORS
|
||||
|
||||
Default-deny: no `Access-Control-Allow-Origin` header is sent unless an
|
||||
allowlist is configured. To allow browser-based MCP clients:
|
||||
|
||||
```bash
|
||||
GBRAIN_HTTP_CORS_ORIGIN=https://claude.ai gbrain serve --http --port 8787
|
||||
# Multiple origins: comma-separated
|
||||
GBRAIN_HTTP_CORS_ORIGIN=https://claude.ai,https://your.app gbrain serve --http
|
||||
```
|
||||
|
||||
When the request `Origin` matches the allowlist, the server echoes it
|
||||
back in `Access-Control-Allow-Origin` (with `Vary: Origin`). Otherwise no
|
||||
CORS header is sent and the browser blocks the request.
|
||||
|
||||
### Rate limiting
|
||||
|
||||
Two buckets, both stored in a bounded LRU map (default 10K keys, evicts
|
||||
least-recently-used on overflow, prunes entries older than 2× the
|
||||
window):
|
||||
|
||||
| Bucket | When it fires | Default | Env var |
|
||||
|---|---|---|---|
|
||||
| Pre-auth IP | Before the DB lookup, on every `/mcp` request | 30 req / 60s | `GBRAIN_HTTP_RATE_LIMIT_IP` |
|
||||
| Post-auth token | After a valid token is resolved | 60 req / 60s | `GBRAIN_HTTP_RATE_LIMIT_TOKEN` |
|
||||
| LRU cap | Maximum distinct keys across both buckets | 10000 | `GBRAIN_HTTP_RATE_LIMIT_LRU` |
|
||||
|
||||
On exhaustion the server returns `429 Too Many Requests` with a
|
||||
`Retry-After` header.
|
||||
|
||||
**Caveat for tunneled deployments (ngrok, Tailscale Funnel, Cloudflare
|
||||
Tunnel):** all requests share one egress IP, so the pre-auth IP bucket
|
||||
becomes effectively shared by all clients on that tunnel. The
|
||||
post-auth token-id bucket is the load-bearing limiter for tunnel-fronted
|
||||
deployments.
|
||||
|
||||
### Reverse-proxy trust
|
||||
|
||||
Disabled by default. To honor `X-Forwarded-For` (or `X-Real-IP`) when
|
||||
gbrain runs behind a trusted reverse proxy:
|
||||
|
||||
```bash
|
||||
GBRAIN_HTTP_TRUST_PROXY=1 gbrain serve --http --port 8787
|
||||
```
|
||||
|
||||
**Critical safety contract:** only set `GBRAIN_HTTP_TRUST_PROXY=1` when
|
||||
**both** of these are true:
|
||||
|
||||
1. gbrain is reachable only via a trusted reverse proxy (not directly
|
||||
exposed to the internet on the configured port). The simplest
|
||||
guarantee is to bind gbrain to `127.0.0.1` or a private interface
|
||||
and have the proxy forward to it.
|
||||
2. The proxy strips any client-supplied `X-Forwarded-For` and `X-Real-IP`
|
||||
headers, then sets them itself. (nginx with `proxy_set_header
|
||||
X-Forwarded-For $remote_addr` does this; Cloudflare and most cloud
|
||||
load balancers handle it automatically.)
|
||||
|
||||
If gbrain is reachable directly AND `GBRAIN_HTTP_TRUST_PROXY=1` is set,
|
||||
clients can spoof their IP by sending arbitrary `X-Forwarded-For`
|
||||
headers, defeating the pre-auth IP rate limit. Without the flag, gbrain
|
||||
ignores all forwarded-for headers and uses the socket peer address,
|
||||
which is the safe default for direct-exposure deployments.
|
||||
|
||||
### Body size cap
|
||||
|
||||
Default 1 MiB, stream-counted (chunked transfers without
|
||||
`Content-Length` are still capped). Override:
|
||||
|
||||
```bash
|
||||
GBRAIN_HTTP_MAX_BODY_BYTES=2097152 gbrain serve --http # 2 MiB
|
||||
```
|
||||
|
||||
Over-cap requests get `413 Payload Too Large` immediately, before any
|
||||
body is materialized in memory.
|
||||
|
||||
### Audit log
|
||||
|
||||
Every `/mcp` request writes one row to `mcp_request_log`:
|
||||
|
||||
```bash
|
||||
psql "$DATABASE_URL" -c \
|
||||
"SELECT created_at, token_name, operation, status, latency_ms
|
||||
FROM mcp_request_log
|
||||
ORDER BY created_at DESC LIMIT 100"
|
||||
```
|
||||
|
||||
`status` is one of: `success`, `error`, `auth_failed`, `rate_limited`,
|
||||
`body_too_large`, `parse_error`, `unknown_method`. Failed-auth rows have
|
||||
`token_name = NULL`. Inserts are fire-and-forget so audit failures
|
||||
never block requests.
|
||||
@@ -585,3 +585,90 @@ iteration's residuals.
|
||||
**Effort estimate:** S (human: ~2 hr / CC: ~10 min).
|
||||
**Priority:** P3.
|
||||
**Depends on:** The above caller-opt-in retry (#1) is the natural co-lander since both touch the same error-classification surface.
|
||||
|
||||
## remote MCP / HTTP transport (v0.22.7 follow-ups)
|
||||
|
||||
### Audit-log write amplification on rejected `/mcp` traffic
|
||||
**What:** `src/mcp/http-transport.ts` writes a row to `mcp_request_log` for every
|
||||
incoming `/mcp` request, including rate-limited (429), oversized (413), and
|
||||
auth-failed (401) traffic. Under sustained attack the IP rate limit caps audit
|
||||
writes per IP at 30/min, but at scale (10K distinct IPs) that's still 300K
|
||||
inserts/min. Two follow-ups: (1) instrument the audit-write rate so we can see
|
||||
the actual production volume; (2) consider a separate "rejected" table or
|
||||
sampling for failed-auth rows so the success-path audit table doesn't get
|
||||
swamped.
|
||||
|
||||
**Why:** Codex flagged this during the v0.22.7 ship adversarial review. We kept
|
||||
the full audit on purpose — forensic data of an attack is valuable — but want
|
||||
to revisit once we have real volume numbers.
|
||||
|
||||
**Pros:** Bounds DB write volume under attack. Keeps the success-path audit
|
||||
table small enough for fast queries.
|
||||
|
||||
**Cons:** Adds a second table or a sampling rule. Not free complexity. Probably
|
||||
not worth it until production hits a real attack pattern.
|
||||
|
||||
**Context:** `src/mcp/http-transport.ts:222,235,245` (the three audit-on-reject
|
||||
call sites) + `src/schema.sql:342` (the unbounded table).
|
||||
|
||||
**Effort estimate:** M (human: ~half day / CC: ~30 min once we have volume data).
|
||||
**Priority:** P3 — wait for evidence.
|
||||
**Depends on:** Production telemetry on `mcp_request_log` insert rate.
|
||||
|
||||
### `validateParams` doesn't check enum values or array item types
|
||||
**What:** `src/mcp/dispatch.ts:27` (extracted from `src/mcp/server.ts` in
|
||||
v0.22.7) only checks top-level JS types. Operations declare `enum` constraints
|
||||
(e.g. `direction: 'in' | 'out' | 'both'`) and array `items: { type: ... }`
|
||||
schemas in `src/core/operations.ts`, but `validateParams` ignores both. Bad
|
||||
inputs still reach handlers — concretely, an invalid `direction` falls through
|
||||
the engine's else branch at `src/core/postgres-engine.ts:954`, widening
|
||||
traversal unexpectedly; malformed `pages_updated` arrays could be written as
|
||||
garbage JSONB.
|
||||
|
||||
**Why:** Codex flagged this during the v0.22.7 ship adversarial review. The
|
||||
validator was lifted verbatim from the pre-existing stdio path during the
|
||||
dispatch.ts extraction — same gap exists on the stdio MCP server today, so
|
||||
this isn't a v0.22.7 regression. Still worth tightening, since "shared
|
||||
validation" is now the architectural guarantee both transports rely on.
|
||||
|
||||
**Pros:** Better defense-in-depth at the MCP boundary. Catches malformed agent
|
||||
inputs before the engine layer has to.
|
||||
|
||||
**Cons:** Need to walk every operation's param schema and decide which enum
|
||||
violations are user-facing errors vs internal bugs. May need a typed Zod-style
|
||||
schema layer to do this cleanly.
|
||||
|
||||
**Context:** `src/mcp/dispatch.ts:27` + `src/core/operations.ts` (param defs).
|
||||
Same gap pre-existed on stdio MCP path.
|
||||
|
||||
**Effort estimate:** M (human: ~half day / CC: ~30 min if we use the existing
|
||||
ParamDef shape; XL if a Zod migration is the chosen direction).
|
||||
**Priority:** P2.
|
||||
**Depends on:** Whether we want to keep the lightweight ParamDef shape or
|
||||
migrate to typed schemas.
|
||||
|
||||
### Streaming MCP tool support (re-add SSE based on Accept header)
|
||||
**What:** v0.22.7 dropped SSE entirely from `gbrain serve --http` because no
|
||||
current MCP tool streams. When the first streaming tool ships (long-running
|
||||
agent delegation as an MCP tool, `resources/subscribe`, `sampling/createMessage`),
|
||||
re-add SSE in `/mcp` based on the `Accept` header per the Streamable HTTP
|
||||
transport spec. ~30 lines + spec compliance test.
|
||||
|
||||
**Why:** Removing SSE simplified the v0.22.7 transport (one response path,
|
||||
fewer test cases). Adding it back when actually needed is cheap and keeps the
|
||||
code lean in the meantime.
|
||||
|
||||
**Effort estimate:** S (human: ~2 hr / CC: ~15 min).
|
||||
**Priority:** P3 — wait for the first streaming tool.
|
||||
**Depends on:** A streaming MCP tool actually existing.
|
||||
|
||||
### `access_tokens.scopes` enforcement
|
||||
**What:** The `access_tokens` schema has had a `scopes TEXT[]` column since
|
||||
migration v4 (`src/core/migrate.ts:84`), but nothing enforces it. v0.22.7's
|
||||
`gbrain auth create` doesn't accept a `--scopes` flag, and `dispatchToolCall`
|
||||
doesn't gate on scopes. Adding per-tool scope enforcement would let
|
||||
"claude-desktop-readonly" and "ingest-only" tokens exist.
|
||||
|
||||
**Effort estimate:** M (human: ~1 day / CC: ~30 min for the schema-aware gate).
|
||||
**Priority:** P3.
|
||||
**Depends on:** Nothing.
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
# Remote MCP Deployment Options
|
||||
|
||||
GBrain's MCP server runs via `gbrain serve` (stdio transport). To make it
|
||||
accessible from other devices and AI clients, you need an HTTP wrapper and
|
||||
a public tunnel. Here are your options.
|
||||
accessible from other devices and AI clients, run `gbrain serve --http`
|
||||
(built-in HTTP transport with bearer auth, Postgres-only ... see
|
||||
[DEPLOY.md](DEPLOY.md)) behind a public tunnel. Here are your tunnel options.
|
||||
|
||||
## ngrok (recommended)
|
||||
|
||||
@@ -13,8 +14,9 @@ a public tunnel. Here are your options.
|
||||
# 1. Install ngrok
|
||||
brew install ngrok
|
||||
|
||||
# 2. Start your MCP server (behind an HTTP wrapper)
|
||||
# See docs/mcp/DEPLOY.md for the server setup
|
||||
# 2. Start the built-in HTTP transport
|
||||
gbrain serve --http --port 8787
|
||||
# See docs/mcp/DEPLOY.md for token setup
|
||||
|
||||
# 3. Expose via ngrok
|
||||
ngrok http 8787 --url your-brain.ngrok.app
|
||||
@@ -59,6 +61,7 @@ Both run Bun natively. No bundling, no Deno, no cold start, no timeout limits.
|
||||
| All 30 operations | Yes | Yes | Yes |
|
||||
| Setup time | 5 min | 10 min | 15 min |
|
||||
|
||||
**Note:** `gbrain serve --http` (built-in HTTP transport) is planned but not yet
|
||||
implemented. Currently, remote MCP requires a custom HTTP wrapper around `gbrain serve`.
|
||||
See [DEPLOY.md](DEPLOY.md) for details.
|
||||
**Note:** `gbrain serve --http` is the built-in HTTP transport (v0.22.7+). Bearer auth
|
||||
against the `access_tokens` table, default-deny CORS, two-bucket rate limit, body cap,
|
||||
per-request audit log. Postgres-only by design (PGLite is local-only). See
|
||||
[DEPLOY.md](DEPLOY.md) and [SECURITY.md](../../SECURITY.md) for env vars and tunables.
|
||||
|
||||
@@ -21,7 +21,7 @@ claude mcp add gbrain -t http \
|
||||
```
|
||||
|
||||
Replace `YOUR-DOMAIN` with your ngrok domain and `YOUR_TOKEN` with a token
|
||||
from `bun run src/commands/auth.ts create "claude-code"`.
|
||||
from `gbrain auth create "claude-code"`.
|
||||
|
||||
## Verify
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ For Team/Enterprise plans, an org Owner adds the connector:
|
||||
https://YOUR-DOMAIN.ngrok.app/mcp
|
||||
```
|
||||
3. Add Bearer token authentication in Advanced Settings
|
||||
(create one with `bun run src/commands/auth.ts create "cowork"`)
|
||||
(create one with `gbrain auth create "cowork"`)
|
||||
4. Save
|
||||
|
||||
Note: Cowork connects from Anthropic's cloud, not your device. Your server
|
||||
|
||||
@@ -16,7 +16,7 @@ Remote HTTP servers must be added through the GUI.
|
||||
Replace `YOUR-DOMAIN` with your ngrok domain (see
|
||||
[ngrok-tunnel recipe](../../recipes/ngrok-tunnel.md) for setup).
|
||||
5. Set authentication to **Bearer Token** and paste your token
|
||||
(create one with `bun run src/commands/auth.ts create "claude-desktop"`)
|
||||
(create one with `gbrain auth create "claude-desktop"`)
|
||||
6. Save
|
||||
|
||||
## Verify
|
||||
|
||||
+21
-14
@@ -1,8 +1,13 @@
|
||||
# Deploy GBrain Remote MCP Server
|
||||
|
||||
> **v0.22.7+:** Use `gbrain serve --http` for remote access. It includes built-in
|
||||
> bearer token auth, default-deny CORS, two-bucket rate limiting, body cap, and
|
||||
> per-request audit log. **Postgres-only** (PGLite is local-only by design).
|
||||
> See [SECURITY.md](../../SECURITY.md) for env vars and tunable defaults.
|
||||
|
||||
Access your brain from any device, any AI client. GBrain's MCP server runs locally
|
||||
via `gbrain serve` (stdio). For remote access, wrap it in an HTTP server behind a
|
||||
public tunnel.
|
||||
via `gbrain serve` (stdio). For remote access, expose it via the built-in HTTP
|
||||
transport behind a public tunnel.
|
||||
|
||||
## Two Paths
|
||||
|
||||
@@ -13,21 +18,23 @@ gbrain serve
|
||||
```
|
||||
|
||||
Works with Claude Code, Cursor, Windsurf, and any MCP client that supports stdio.
|
||||
No server, no tunnel, no token needed.
|
||||
No server, no tunnel, no token needed. Works on both PGLite and Postgres engines.
|
||||
|
||||
### Remote (any device, any AI client)
|
||||
### Remote (any device, any AI client) — Postgres only
|
||||
|
||||
```
|
||||
Your AI client (Claude Desktop, Perplexity, etc.)
|
||||
→ ngrok tunnel (https://YOUR-DOMAIN.ngrok.app)
|
||||
→ Your HTTP server (wraps gbrain serve)
|
||||
→ Supabase Postgres (via pooler connection string)
|
||||
→ gbrain serve --http (built-in transport with bearer auth)
|
||||
→ Postgres (pooler connection or self-hosted)
|
||||
```
|
||||
|
||||
This requires:
|
||||
1. A machine running `gbrain serve` behind an HTTP wrapper
|
||||
2. A public tunnel (ngrok, Tailscale, or cloud host)
|
||||
3. Bearer token auth for security
|
||||
1. A Postgres-backed brain (the `access_tokens` table only exists on Postgres;
|
||||
running `gbrain serve --http` against a PGLite install fails fast at startup)
|
||||
2. A machine running `gbrain serve --http`
|
||||
3. A public tunnel (ngrok, Tailscale, or cloud host)
|
||||
4. A bearer token created via `gbrain auth create <name>`
|
||||
|
||||
## Remote Setup
|
||||
|
||||
@@ -46,13 +53,13 @@ ngrok http 8787 --url your-brain.ngrok.app # Hobby tier for fixed domain
|
||||
|
||||
```bash
|
||||
# Create a token for each client
|
||||
bun run src/commands/auth.ts create "claude-desktop"
|
||||
gbrain auth create "claude-desktop"
|
||||
|
||||
# List all tokens
|
||||
bun run src/commands/auth.ts list
|
||||
gbrain auth list
|
||||
|
||||
# Revoke a token
|
||||
bun run src/commands/auth.ts revoke "claude-desktop"
|
||||
gbrain auth revoke "claude-desktop"
|
||||
```
|
||||
|
||||
Tokens are per-client. Create one for each device/app. Revoke individually
|
||||
@@ -68,7 +75,7 @@ if compromised. Tokens are stored SHA-256 hashed in your database.
|
||||
### 4. Verify
|
||||
|
||||
```bash
|
||||
bun run src/commands/auth.ts test \
|
||||
gbrain auth test \
|
||||
https://YOUR-DOMAIN.ngrok.app/mcp \
|
||||
--token YOUR_TOKEN
|
||||
```
|
||||
@@ -96,7 +103,7 @@ Funnel, and cloud hosts (Fly.io, Railway).
|
||||
Include the Authorization header: `Authorization: Bearer YOUR_TOKEN`
|
||||
|
||||
**"invalid_token" error**
|
||||
Run `bun run src/commands/auth.ts list` to see active tokens.
|
||||
Run `gbrain auth list` to see active tokens.
|
||||
|
||||
**"service_unavailable" error**
|
||||
Database connection failed. Check your Supabase dashboard for outages.
|
||||
|
||||
@@ -10,7 +10,7 @@ Perplexity Computer supports remote MCP servers with bearer token authentication
|
||||
- **URL:** `https://YOUR-DOMAIN.ngrok.app/mcp`
|
||||
- **Authentication:** API Key / Bearer Token
|
||||
- **Token:** your GBrain access token
|
||||
(create one with `bun run src/commands/auth.ts create "perplexity"`)
|
||||
(create one with `gbrain auth create "perplexity"`)
|
||||
4. Save
|
||||
|
||||
Replace `YOUR-DOMAIN` with your ngrok domain (see
|
||||
|
||||
+35
-20
@@ -170,8 +170,11 @@ strict behavior when unset.
|
||||
- `src/commands/jobs.ts` — `gbrain jobs` CLI subcommands + `gbrain jobs work` daemon. v0.13.1 surfaces the full `MinionJobInput` retry/backoff/timeout/idempotency surface as first-class CLI flags on `jobs submit`: `--max-stalled`, `--backoff-type fixed|exponential`, `--backoff-delay`, `--backoff-jitter`, `--timeout-ms`, `--idempotency-key`. `jobs smoke --sigkill-rescue` is the opt-in regression guard for #219. v0.16 wires `registerBuiltinHandlers` to always register `subagent` + `subagent_aggregator` (no env flag — `ANTHROPIC_API_KEY` is the natural cost gate, trust is via `PROTECTED_JOB_NAMES`) and loads `GBRAIN_PLUGIN_PATH` plugins at worker startup with a loud startup-line per plugin. `shell` handler still gated by `GBRAIN_ALLOW_SHELL_JOBS=1` (RCE surface, separate concern).
|
||||
- `src/commands/features.ts` — `gbrain features --json --auto-fix`: usage scan + feature adoption salesman
|
||||
- `src/commands/autopilot.ts` — `gbrain autopilot --install`: self-maintaining brain daemon (sync+extract+embed)
|
||||
- `src/mcp/server.ts` — MCP stdio server (generated from operations)
|
||||
- `src/commands/auth.ts` — Standalone token management (create/list/revoke/test)
|
||||
- `src/mcp/server.ts` — MCP stdio server (generated from operations). v0.22.7: tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path.
|
||||
- `src/mcp/dispatch.ts` (v0.22.7) — Shared tool-call dispatch consumed by both stdio (`server.ts`) and HTTP (`http-transport.ts`). Exports `dispatchToolCall(engine, name, params, opts)`, `buildOperationContext(engine, params, opts)`, and `validateParams(op, params)`. Single source of truth for `(ctx, params)` handler arg order and the 5-field `OperationContext` shape (engine + config + logger + dryRun + remote). Defaults to `remote: true` (untrusted); local CLI callers pass `remote: false`. Closed F1 (reversed handler args) + F2 (incomplete OperationContext) + F3 (no param validation) drift bugs in the original v0.22.5 HTTP transport.
|
||||
- `src/mcp/rate-limit.ts` (v0.22.7) — Bounded-LRU token-bucket limiter for `gbrain serve --http`. `buildDefaultLimiters()` returns the two-bucket pipeline used by http-transport: pre-auth IP (default 30/60s, fires BEFORE the DB lookup so brute-force load against `access_tokens` is actually capped) + post-auth token-id (default 60/60s). Tracks `lastTouchedMs` separately from `lastRefillMs` so an exhausted key can't be reset by hammering past the TTL. LRU cap (default 10K keys) bounds memory under attacker-controlled key growth; TTL prune at 2× window evicts abandoned buckets.
|
||||
- `src/mcp/http-transport.ts` (v0.22.7, rewrite) — `gbrain serve --http` HTTP transport. Postgres-only — fails fast at startup on PGLite (the `access_tokens` table only exists on Postgres). Bearer auth against SHA-256 hashes in `access_tokens`. CORS default-deny via `GBRAIN_HTTP_CORS_ORIGIN` allowlist. Body cap stream-counted (1 MiB default via `GBRAIN_HTTP_MAX_BODY_BYTES`) so chunked transfers without Content-Length still hit the cap. `last_used_at` SQL-level debounce (one UPDATE per token per 60s). Per-request audit row in `mcp_request_log` with token_name + operation + status + latency. Optional `GBRAIN_HTTP_TRUST_PROXY=1` honors `X-Forwarded-For` — only safe when bound to a private interface AND the proxy strips client-supplied XFF (otherwise enables IP spoofing past the pre-auth rate limit). `/health` does `SELECT 1` against Postgres and returns 503 + `status:unhealthy` when the DB is unreachable so orchestration doesn't see green pods while clients get misleading 401s. Replaces the standalone OAuth wrapper that was vulnerable to unauthenticated client registration.
|
||||
- `src/commands/auth.ts` — Token management for the HTTP transport. `gbrain auth create/list/revoke/test`. As of v0.22.7 wired into the main CLI (`src/cli.ts`); also runs standalone via `bun run src/commands/auth.ts ...` for environments without a compiled binary. Tokens stored as SHA-256 hashes in `access_tokens` (Postgres-only).
|
||||
- `src/commands/upgrade.ts` — Self-update CLI. `runPostUpgrade()` enumerates migrations from the TS registry (src/commands/migrations/index.ts) and tail-calls `runApplyMigrations(['--yes', '--non-interactive'])` so the mechanical side of every outstanding migration runs unconditionally.
|
||||
- `src/commands/migrations/` — TS migration registry (compiled into the binary; no filesystem walk of `skills/migrations/*.md` needed at runtime). `index.ts` lists migrations in semver order. `v0_11_0.ts` = Minions adoption orchestrator (8 phases). `v0_12_0.ts` = Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify). `phaseASchema` has a 600s timeout (bumped from 60s in v0.12.1 for duplicate-heavy brains). `v0_12_2.ts` = JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record). `v0_14_0.ts` = shell-jobs + autopilot cooperative (2 phases: schema ALTER minion_jobs.max_stalled SET DEFAULT 3 — superseded by v0.14.3's schema-level DEFAULT 5 + UPDATE backfill; pending-host-work ping for skills/migrations/v0.14.0.md). All orchestrators are idempotent and resumable from `partial` status. As of v0.14.2 (Bug 3), the RUNNER owns all ledger writes — orchestrators return `OrchestratorResult` and `apply-migrations.ts` persists a canonical `{version, status, phases}` shape after return. Orchestrators no longer call `appendCompletedMigration` directly. `statusForVersion` prefers `complete` over `partial` (never regresses). 3 consecutive partials → wedged → `--force-retry <version>` writes a `'retry'` reset marker. v0.14.3 (fix wave) ships schema-only migrations v14 (`pages_updated_at_index`) + v15 (`minion_jobs_max_stalled_default_5` with UPDATE backfill) via the `MIGRATIONS` array in `src/core/migrate.ts` — no orchestrator phases needed.
|
||||
- `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent.
|
||||
@@ -356,7 +359,8 @@ parity), `test/cli.test.ts` (CLI structure), `test/config.test.ts` (config redac
|
||||
`test/skill-manifest.test.ts` (v0.19 skill manifest parser: drift detection, managed-block markers),
|
||||
`test/skillify-scaffold.test.ts` (v0.19 `gbrain skillify scaffold` stubs: SKILL.md, script, tests, routing-eval fixtures),
|
||||
`test/skillpack-install.test.ts` (v0.19 `gbrain skillpack install` managed-block install / update / no-clobber semantics),
|
||||
`test/skillpack-sync-guard.test.ts` (v0.19 sync-guard: bundled skills stay byte-identical to `skills/` source).
|
||||
`test/skillpack-sync-guard.test.ts` (v0.19 sync-guard: bundled skills stay byte-identical to `skills/` source),
|
||||
`test/http-transport.test.ts` (v0.22.7 HTTP transport: 23 unit cases covering bearer auth + missing/no-Bearer/unknown/revoked + `/health` bypass, F1+F2 round-trip via dispatch.ts, F3 invalid_params, application/json response shape (not SSE), CORS default-deny + allowlist, body cap on Content-Length AND chunked, two-bucket rate limit (refill, exhaust+Retry-After, LRU eviction, TTL prune, pre-auth IP fires before DB), and `mcp_request_log` audit on success + auth_failed).
|
||||
|
||||
E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_URL`.
|
||||
- `bun run test:e2e` runs Tier 1 (mechanical, all operations, no API keys). Includes 9 dedicated cases for the postgres-engine `addLinksBatch` / `addTimelineEntriesBatch` bind path — postgres-js's `unnest()` binding is structurally different from PGLite's and gets its own coverage.
|
||||
@@ -371,6 +375,7 @@ E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_U
|
||||
- `test/e2e/search-exclude.test.ts` (v0.22.0) — verifies `test/` + `archive/` pages are hidden by default, that `include_slug_prefixes` opts back in, and that caller-supplied `exclude_slug_prefixes` adds to defaults. Both keyword and vector search paths covered.
|
||||
- `test/e2e/engine-parity.test.ts` (v0.22.0) — Postgres ↔ PGLite top-result and result-set parity for `searchKeyword` + `searchVector`. Codex flagged that Postgres ranks pages then picks best chunk while PGLite returns chunks directly — without parity coverage the source-boost fix could pass on PGLite and fail on Postgres. Skips gracefully when `DATABASE_URL` is unset.
|
||||
- `test/e2e/postgres-bootstrap.test.ts` (v0.22.6.1) — exercises `PostgresEngine.initSchema()` directly against a fresh real Postgres database. Asserts the bootstrap path is no-op on fresh installs and that SCHEMA_SQL replays cleanly through the engine path (not via the standalone `db.initSchema` from `src/core/db.ts`, which would have produced false-positive coverage). Codex caught the E2E-shape gap during plan review.
|
||||
- `test/e2e/http-transport.test.ts` (v0.22.7) — 8 cases against real Postgres covering `gbrain serve --http` end-to-end: bearer auth round-trip, `last_used_at` SQL-level debounce semantics, `mcp_request_log` row insertion on success and auth_failed paths, `/health` DB-down → 503 (DB-probing health check), and the F1+F2+F3 dispatch round-trip with a real operation. Skips gracefully when `DATABASE_URL` is unset.
|
||||
- Tier 2 (`skills.test.ts`) requires OpenClaw + API keys, runs nightly in CI
|
||||
- If `.env.testing` doesn't exist in this directory, check sibling worktrees for one:
|
||||
`find ../ -maxdepth 2 -name .env.testing -print -quit` and copy it here if found.
|
||||
@@ -1351,12 +1356,13 @@ Add to `~/.claude/server.json` (Claude Code), Settings > MCP Servers (Cursor), o
|
||||
### Remote MCP (Claude Desktop, Cowork, Perplexity)
|
||||
|
||||
```bash
|
||||
ngrok http 8787 --url your-brain.ngrok.app
|
||||
bun run src/commands/auth.ts create "claude-desktop"
|
||||
gbrain auth create "claude-desktop" # tokens via the existing CLI
|
||||
gbrain serve --http --port 8787 # built-in HTTP transport (Postgres-only)
|
||||
ngrok http 8787 --url your-brain.ngrok.app # any tunnel works
|
||||
claude mcp add gbrain -t http https://your-brain.ngrok.app/mcp -H "Authorization: Bearer TOKEN"
|
||||
```
|
||||
|
||||
Per-client guides: [`docs/mcp/`](docs/mcp/DEPLOY.md). ChatGPT requires OAuth 2.1 (not yet implemented).
|
||||
Per-client guides: [`docs/mcp/`](docs/mcp/DEPLOY.md). Hardening defaults, env vars, and threat model: [SECURITY.md](SECURITY.md). ChatGPT requires OAuth 2.1 (not yet implemented).
|
||||
|
||||
### Using gbrain with GStack
|
||||
|
||||
@@ -1928,6 +1934,8 @@ ADMIN
|
||||
gbrain doctor --locks List idle-in-tx backends (57014 diagnostic, Postgres only)
|
||||
gbrain stats Brain statistics
|
||||
gbrain serve MCP server (stdio)
|
||||
gbrain serve --http --port 8787 MCP server (HTTP, Postgres-only, bearer auth)
|
||||
gbrain auth create|list|revoke|test Token management for the HTTP transport
|
||||
gbrain integrations Integration recipe dashboard
|
||||
gbrain sources list|add|remove|... Multi-source brain management (v0.18)
|
||||
gbrain dream [--dry-run] [--phase N] One maintenance cycle then exit (cron-friendly)
|
||||
@@ -4108,9 +4116,14 @@ Source: https://raw.githubusercontent.com/garrytan/gbrain/master/docs/mcp/DEPLOY
|
||||
|
||||
# Deploy GBrain Remote MCP Server
|
||||
|
||||
> **v0.22.7+:** Use `gbrain serve --http` for remote access. It includes built-in
|
||||
> bearer token auth, default-deny CORS, two-bucket rate limiting, body cap, and
|
||||
> per-request audit log. **Postgres-only** (PGLite is local-only by design).
|
||||
> See [SECURITY.md](../../SECURITY.md) for env vars and tunable defaults.
|
||||
|
||||
Access your brain from any device, any AI client. GBrain's MCP server runs locally
|
||||
via `gbrain serve` (stdio). For remote access, wrap it in an HTTP server behind a
|
||||
public tunnel.
|
||||
via `gbrain serve` (stdio). For remote access, expose it via the built-in HTTP
|
||||
transport behind a public tunnel.
|
||||
|
||||
## Two Paths
|
||||
|
||||
@@ -4121,21 +4134,23 @@ gbrain serve
|
||||
```
|
||||
|
||||
Works with Claude Code, Cursor, Windsurf, and any MCP client that supports stdio.
|
||||
No server, no tunnel, no token needed.
|
||||
No server, no tunnel, no token needed. Works on both PGLite and Postgres engines.
|
||||
|
||||
### Remote (any device, any AI client)
|
||||
### Remote (any device, any AI client) — Postgres only
|
||||
|
||||
```
|
||||
Your AI client (Claude Desktop, Perplexity, etc.)
|
||||
→ ngrok tunnel (https://YOUR-DOMAIN.ngrok.app)
|
||||
→ Your HTTP server (wraps gbrain serve)
|
||||
→ Supabase Postgres (via pooler connection string)
|
||||
→ gbrain serve --http (built-in transport with bearer auth)
|
||||
→ Postgres (pooler connection or self-hosted)
|
||||
```
|
||||
|
||||
This requires:
|
||||
1. A machine running `gbrain serve` behind an HTTP wrapper
|
||||
2. A public tunnel (ngrok, Tailscale, or cloud host)
|
||||
3. Bearer token auth for security
|
||||
1. A Postgres-backed brain (the `access_tokens` table only exists on Postgres;
|
||||
running `gbrain serve --http` against a PGLite install fails fast at startup)
|
||||
2. A machine running `gbrain serve --http`
|
||||
3. A public tunnel (ngrok, Tailscale, or cloud host)
|
||||
4. A bearer token created via `gbrain auth create <name>`
|
||||
|
||||
## Remote Setup
|
||||
|
||||
@@ -4154,13 +4169,13 @@ ngrok http 8787 --url your-brain.ngrok.app # Hobby tier for fixed domain
|
||||
|
||||
```bash
|
||||
# Create a token for each client
|
||||
bun run src/commands/auth.ts create "claude-desktop"
|
||||
gbrain auth create "claude-desktop"
|
||||
|
||||
# List all tokens
|
||||
bun run src/commands/auth.ts list
|
||||
gbrain auth list
|
||||
|
||||
# Revoke a token
|
||||
bun run src/commands/auth.ts revoke "claude-desktop"
|
||||
gbrain auth revoke "claude-desktop"
|
||||
```
|
||||
|
||||
Tokens are per-client. Create one for each device/app. Revoke individually
|
||||
@@ -4176,7 +4191,7 @@ if compromised. Tokens are stored SHA-256 hashed in your database.
|
||||
### 4. Verify
|
||||
|
||||
```bash
|
||||
bun run src/commands/auth.ts test \
|
||||
gbrain auth test \
|
||||
https://YOUR-DOMAIN.ngrok.app/mcp \
|
||||
--token YOUR_TOKEN
|
||||
```
|
||||
@@ -4204,7 +4219,7 @@ Funnel, and cloud hosts (Fly.io, Railway).
|
||||
Include the Authorization header: `Authorization: Bearer YOUR_TOKEN`
|
||||
|
||||
**"invalid_token" error**
|
||||
Run `bun run src/commands/auth.ts list` to see active tokens.
|
||||
Run `gbrain auth list` to see active tokens.
|
||||
|
||||
**"service_unavailable" error**
|
||||
Database connection failed. Check your Supabase dashboard for outages.
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.22.6.1",
|
||||
"version": "0.22.7",
|
||||
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
|
||||
"type": "module",
|
||||
"main": "src/core/index.ts",
|
||||
|
||||
+7
-2
@@ -19,7 +19,7 @@ for (const op of operations) {
|
||||
}
|
||||
|
||||
// CLI-only commands that bypass the operation layer
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'repos', 'code-def', 'code-refs', 'reindex-code', 'code-callers', 'code-callees', 'frontmatter']);
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'repos', 'code-def', 'code-refs', 'reindex-code', 'code-callers', 'code-callees', 'frontmatter', 'auth']);
|
||||
|
||||
async function main() {
|
||||
// Parse global flags (--quiet / --progress-json / --progress-interval)
|
||||
@@ -285,6 +285,11 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
await runIntegrations(args);
|
||||
return;
|
||||
}
|
||||
if (command === 'auth') {
|
||||
const { runAuth } = await import('./commands/auth.ts');
|
||||
await runAuth(args);
|
||||
return;
|
||||
}
|
||||
if (command === 'resolvers') {
|
||||
const { runResolvers } = await import('./commands/resolvers.ts');
|
||||
await runResolvers(args);
|
||||
@@ -447,7 +452,7 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
}
|
||||
case 'serve': {
|
||||
const { runServe } = await import('./commands/serve.ts');
|
||||
await runServe(engine);
|
||||
await runServe(engine, args);
|
||||
return; // serve doesn't disconnect
|
||||
}
|
||||
case 'call': {
|
||||
|
||||
+52
-31
@@ -1,20 +1,29 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* GBrain token management — standalone script, no gbrain CLI dependency.
|
||||
* GBrain token management.
|
||||
*
|
||||
* Usage:
|
||||
* Wired into the CLI as of v0.22.5:
|
||||
* gbrain auth create "claude-desktop"
|
||||
* gbrain auth list
|
||||
* gbrain auth revoke "claude-desktop"
|
||||
* gbrain auth test <url> --token <token>
|
||||
*
|
||||
* Also runs standalone (no compiled binary required):
|
||||
* DATABASE_URL=... bun run src/commands/auth.ts create "claude-desktop"
|
||||
* DATABASE_URL=... bun run src/commands/auth.ts list
|
||||
* DATABASE_URL=... bun run src/commands/auth.ts revoke "claude-desktop"
|
||||
* DATABASE_URL=... bun run src/commands/auth.ts test <url> --token <token>
|
||||
*
|
||||
* Both paths require DATABASE_URL or GBRAIN_DATABASE_URL (except `test`,
|
||||
* which only hits the remote URL and doesn't need a local DB).
|
||||
*/
|
||||
import postgres from 'postgres';
|
||||
import { createHash, randomBytes } from 'crypto';
|
||||
|
||||
const DATABASE_URL = process.env.DATABASE_URL || process.env.GBRAIN_DATABASE_URL;
|
||||
if (!DATABASE_URL && process.argv[2] !== 'test') {
|
||||
console.error('Set DATABASE_URL or GBRAIN_DATABASE_URL environment variable.');
|
||||
process.exit(1);
|
||||
function getDatabaseUrl(requireDb: boolean): string | undefined {
|
||||
const url = process.env.DATABASE_URL || process.env.GBRAIN_DATABASE_URL;
|
||||
if (!url && requireDb) {
|
||||
console.error('Set DATABASE_URL or GBRAIN_DATABASE_URL environment variable.');
|
||||
process.exit(1);
|
||||
}
|
||||
return url;
|
||||
}
|
||||
|
||||
function hashToken(token: string): string {
|
||||
@@ -27,7 +36,7 @@ function generateToken(): string {
|
||||
|
||||
async function create(name: string) {
|
||||
if (!name) { console.error('Usage: auth create <name>'); process.exit(1); }
|
||||
const sql = postgres(DATABASE_URL!);
|
||||
const sql = postgres(getDatabaseUrl(true)!);
|
||||
const token = generateToken();
|
||||
const hash = hashToken(token);
|
||||
|
||||
@@ -53,7 +62,7 @@ async function create(name: string) {
|
||||
}
|
||||
|
||||
async function list() {
|
||||
const sql = postgres(DATABASE_URL!);
|
||||
const sql = postgres(getDatabaseUrl(true)!);
|
||||
try {
|
||||
const rows = await sql`
|
||||
SELECT name, created_at, last_used_at, revoked_at
|
||||
@@ -80,7 +89,7 @@ async function list() {
|
||||
|
||||
async function revoke(name: string) {
|
||||
if (!name) { console.error('Usage: auth revoke <name>'); process.exit(1); }
|
||||
const sql = postgres(DATABASE_URL!);
|
||||
const sql = postgres(getDatabaseUrl(true)!);
|
||||
try {
|
||||
const result = await sql`
|
||||
UPDATE access_tokens SET revoked_at = now()
|
||||
@@ -216,26 +225,38 @@ async function test(url: string, token: string) {
|
||||
console.log(`\n🧠 Your brain is live! (${elapsed}s)`);
|
||||
}
|
||||
|
||||
// CLI dispatch
|
||||
const [cmd, ...args] = process.argv.slice(2);
|
||||
switch (cmd) {
|
||||
case 'create': await create(args[0]); break;
|
||||
case 'list': await list(); break;
|
||||
case 'revoke': await revoke(args[0]); break;
|
||||
case 'test': {
|
||||
const tokenIdx = args.indexOf('--token');
|
||||
const url = args.find(a => !a.startsWith('--') && a !== args[tokenIdx + 1]);
|
||||
const token = tokenIdx >= 0 ? args[tokenIdx + 1] : '';
|
||||
await test(url || '', token || '');
|
||||
break;
|
||||
}
|
||||
default:
|
||||
console.log(`GBrain Token Management
|
||||
/**
|
||||
* Entry point for the `gbrain auth` CLI subcommand. Also reused by the
|
||||
* direct-script path (see bottom of file) so `bun run src/commands/auth.ts`
|
||||
* still works.
|
||||
*/
|
||||
export async function runAuth(args: string[]): Promise<void> {
|
||||
const [cmd, ...rest] = args;
|
||||
switch (cmd) {
|
||||
case 'create': await create(rest[0]); return;
|
||||
case 'list': await list(); return;
|
||||
case 'revoke': await revoke(rest[0]); return;
|
||||
case 'test': {
|
||||
const tokenIdx = rest.indexOf('--token');
|
||||
const url = rest.find(a => !a.startsWith('--') && a !== rest[tokenIdx + 1]);
|
||||
const token = tokenIdx >= 0 ? rest[tokenIdx + 1] : '';
|
||||
await test(url || '', token || '');
|
||||
return;
|
||||
}
|
||||
default:
|
||||
console.log(`GBrain Token Management
|
||||
|
||||
Usage:
|
||||
bun run src/commands/auth.ts create <name> Create a new access token
|
||||
bun run src/commands/auth.ts list List all tokens
|
||||
bun run src/commands/auth.ts revoke <name> Revoke a token
|
||||
bun run src/commands/auth.ts test <url> --token <token> Smoke test a remote MCP server
|
||||
gbrain auth create <name> Create a new access token
|
||||
gbrain auth list List all tokens
|
||||
gbrain auth revoke <name> Revoke a token
|
||||
gbrain auth test <url> --token <t> Smoke-test a remote MCP server
|
||||
`);
|
||||
}
|
||||
}
|
||||
|
||||
// Direct-script entry point — only runs when this file is invoked as the main module
|
||||
// (e.g. `bun run src/commands/auth.ts ...`). When imported by cli.ts, this block is skipped.
|
||||
if (import.meta.main) {
|
||||
await runAuth(process.argv.slice(2));
|
||||
}
|
||||
|
||||
+15
-3
@@ -1,7 +1,19 @@
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { startMcpServer } from '../mcp/server.ts';
|
||||
import { startHttpTransport } from '../mcp/http-transport.ts';
|
||||
|
||||
export async function runServe(engine: BrainEngine) {
|
||||
console.error('Starting GBrain MCP server (stdio)...');
|
||||
await startMcpServer(engine);
|
||||
export async function runServe(engine: BrainEngine, args: string[] = []) {
|
||||
const useHttp = args.includes('--http');
|
||||
const portIdx = args.indexOf('--port');
|
||||
const port = portIdx >= 0 ? parseInt(args[portIdx + 1]) || 8787 : 8787;
|
||||
|
||||
if (useHttp) {
|
||||
console.error(`Starting GBrain MCP server (HTTP on port ${port})...`);
|
||||
await startHttpTransport({ port, engine });
|
||||
// Keep alive
|
||||
await new Promise(() => {});
|
||||
} else {
|
||||
console.error('Starting GBrain MCP server (stdio)...');
|
||||
await startMcpServer(engine);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
/**
|
||||
* Shared MCP tool-call dispatch — single source of truth for stdio + HTTP transports.
|
||||
*
|
||||
* Both transports validate the same params, build the same OperationContext shape,
|
||||
* and serialize errors identically. Drift between transports caused PR #483's reversed-args
|
||||
* + missing-context bugs; this module exists to prevent that recurring.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { operations, OperationError } from '../core/operations.ts';
|
||||
import type { Operation, OperationContext } from '../core/operations.ts';
|
||||
import { loadConfig } from '../core/config.ts';
|
||||
|
||||
export interface ToolResult {
|
||||
content: { type: 'text'; text: string }[];
|
||||
isError?: boolean;
|
||||
}
|
||||
|
||||
export interface DispatchOpts {
|
||||
/** Defaults to true (remote/untrusted). Local CLI callers (`gbrain call`) pass false. */
|
||||
remote?: boolean;
|
||||
/** Override the default stderr logger (e.g. CLI uses console.* directly). */
|
||||
logger?: OperationContext['logger'];
|
||||
}
|
||||
|
||||
/** Validate required params exist and have the expected type. Returns null on success, error message on failure. */
|
||||
export function validateParams(op: Operation, params: Record<string, unknown>): string | null {
|
||||
for (const [key, def] of Object.entries(op.params)) {
|
||||
if (def.required && (params[key] === undefined || params[key] === null)) {
|
||||
return `Missing required parameter: ${key}`;
|
||||
}
|
||||
if (params[key] !== undefined && params[key] !== null) {
|
||||
const val = params[key];
|
||||
const expected = def.type;
|
||||
if (expected === 'string' && typeof val !== 'string') return `Parameter "${key}" must be a string`;
|
||||
if (expected === 'number' && typeof val !== 'number') return `Parameter "${key}" must be a number`;
|
||||
if (expected === 'boolean' && typeof val !== 'boolean') return `Parameter "${key}" must be a boolean`;
|
||||
if (expected === 'object' && (typeof val !== 'object' || Array.isArray(val))) return `Parameter "${key}" must be an object`;
|
||||
if (expected === 'array' && !Array.isArray(val)) return `Parameter "${key}" must be an array`;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
const stderrLogger: OperationContext['logger'] = {
|
||||
info: (msg: string) => process.stderr.write(`[info] ${msg}\n`),
|
||||
warn: (msg: string) => process.stderr.write(`[warn] ${msg}\n`),
|
||||
error: (msg: string) => process.stderr.write(`[error] ${msg}\n`),
|
||||
};
|
||||
|
||||
export function buildOperationContext(
|
||||
engine: BrainEngine,
|
||||
params: Record<string, unknown>,
|
||||
opts: DispatchOpts = {},
|
||||
): OperationContext {
|
||||
return {
|
||||
engine,
|
||||
config: loadConfig() || { engine: 'postgres' },
|
||||
logger: opts.logger || stderrLogger,
|
||||
dryRun: !!params.dry_run,
|
||||
remote: opts.remote ?? true,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve operation, validate params, build context, invoke handler, format result.
|
||||
*
|
||||
* Returns a `ToolResult` with the same shape both MCP transports need:
|
||||
* `{ content: [{ type: 'text', text }], isError?: boolean }`.
|
||||
*/
|
||||
export async function dispatchToolCall(
|
||||
engine: BrainEngine,
|
||||
name: string,
|
||||
params: Record<string, unknown> | undefined,
|
||||
opts: DispatchOpts = {},
|
||||
): Promise<ToolResult> {
|
||||
const op = operations.find(o => o.name === name);
|
||||
if (!op) {
|
||||
return { content: [{ type: 'text', text: `Error: Unknown tool: ${name}` }], isError: true };
|
||||
}
|
||||
|
||||
const safeParams = params || {};
|
||||
const validationError = validateParams(op, safeParams);
|
||||
if (validationError) {
|
||||
return {
|
||||
content: [{ type: 'text', text: JSON.stringify({ error: 'invalid_params', message: validationError }, null, 2) }],
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
const ctx = buildOperationContext(engine, safeParams, opts);
|
||||
|
||||
try {
|
||||
const result = await op.handler(ctx, safeParams);
|
||||
return { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] };
|
||||
} catch (e: unknown) {
|
||||
if (e instanceof OperationError) {
|
||||
return { content: [{ type: 'text', text: JSON.stringify(e.toJSON(), null, 2) }], isError: true };
|
||||
}
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
return { content: [{ type: 'text', text: `Error: ${msg}` }], isError: true };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,354 @@
|
||||
/**
|
||||
* HTTP transport for `gbrain serve --http`.
|
||||
*
|
||||
* Postgres-only. PGLite users get a clear fail-fast at startup (the access_tokens
|
||||
* table doesn't exist on PGLite per pglite-schema.ts).
|
||||
*
|
||||
* Security model:
|
||||
* - Every request must include `Authorization: Bearer <token>` (except /health)
|
||||
* - Tokens are validated against SHA-256 hashes in the access_tokens table
|
||||
* - Create/manage tokens with auth.ts (gbrain auth create/list/revoke)
|
||||
* - No open OAuth, no client_credentials, no self-service tokens
|
||||
*
|
||||
* Hardening:
|
||||
* - CORS default-deny: allowlist via GBRAIN_HTTP_CORS_ORIGIN (comma-separated)
|
||||
* - Rate limit: per-IP pre-auth (protects DB from brute-force load) + per-token-id post-auth
|
||||
* (limits runaway clients). Default 30 req/min per IP, 60 req/min per token. Bounded LRU
|
||||
* so attacker-controlled keys can't grow memory unbounded.
|
||||
* - Body cap: 1 MiB default (GBRAIN_HTTP_MAX_BODY_BYTES). Stream-counted, not buffered —
|
||||
* chunked transfers without Content-Length are still capped.
|
||||
* - last_used_at debounce: only one UPDATE per token per 60s (SQL-level WHERE clause).
|
||||
* - mcp_request_log: one row per request with token_name + operation + status + latency.
|
||||
*
|
||||
* Replaces the standalone HTTP+OAuth wrapper that was vulnerable to unauthenticated
|
||||
* client registration (see SECURITY.md).
|
||||
*/
|
||||
|
||||
import { createHash } from 'crypto';
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { buildToolDefs } from './tool-defs.ts';
|
||||
import { operations } from '../core/operations.ts';
|
||||
import { VERSION } from '../version.ts';
|
||||
import { dispatchToolCall } from './dispatch.ts';
|
||||
import { buildDefaultLimiters, type RateLimiter } from './rate-limit.ts';
|
||||
|
||||
const DEFAULT_BODY_CAP = 1024 * 1024; // 1 MiB
|
||||
|
||||
function hashToken(token: string): string {
|
||||
return createHash('sha256').update(token).digest('hex');
|
||||
}
|
||||
|
||||
function envInt(name: string, fallback: number): number {
|
||||
const v = process.env[name];
|
||||
if (!v) return fallback;
|
||||
const n = parseInt(v, 10);
|
||||
return Number.isFinite(n) && n > 0 ? n : fallback;
|
||||
}
|
||||
|
||||
function parseCorsAllowlist(): Set<string> | null {
|
||||
const v = process.env.GBRAIN_HTTP_CORS_ORIGIN;
|
||||
if (!v) return null;
|
||||
return new Set(v.split(',').map(s => s.trim()).filter(Boolean));
|
||||
}
|
||||
|
||||
interface HttpTransportOptions {
|
||||
port: number;
|
||||
engine: BrainEngine;
|
||||
/** Override limiters (for tests). Defaults to env-driven buildDefaultLimiters. */
|
||||
limiters?: { ip: RateLimiter; token: RateLimiter };
|
||||
}
|
||||
|
||||
interface AuthResult {
|
||||
ok: boolean;
|
||||
tokenId?: string;
|
||||
tokenName?: string;
|
||||
}
|
||||
|
||||
/** Read up to `cap` bytes off req.body. Returns null if cap exceeded. */
|
||||
async function readBodyWithCap(req: Request, cap: number): Promise<string | null> {
|
||||
const cl = req.headers.get('content-length');
|
||||
if (cl) {
|
||||
const n = parseInt(cl, 10);
|
||||
if (Number.isFinite(n) && n > cap) return null;
|
||||
}
|
||||
const reader = req.body?.getReader();
|
||||
if (!reader) return '';
|
||||
const chunks: Uint8Array[] = [];
|
||||
let total = 0;
|
||||
while (true) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done) break;
|
||||
if (!value) continue;
|
||||
total += value.byteLength;
|
||||
if (total > cap) {
|
||||
try { await reader.cancel(); } catch { /* noop */ }
|
||||
return null;
|
||||
}
|
||||
chunks.push(value);
|
||||
}
|
||||
// Concatenate without Buffer to keep this Node-vs-Bun-portable.
|
||||
const merged = new Uint8Array(total);
|
||||
let offset = 0;
|
||||
for (const c of chunks) {
|
||||
merged.set(c, offset);
|
||||
offset += c.byteLength;
|
||||
}
|
||||
return new TextDecoder().decode(merged);
|
||||
}
|
||||
|
||||
/** Resolve client IP. Honors X-Forwarded-For only when GBRAIN_HTTP_TRUST_PROXY=1. */
|
||||
function resolveClientIp(req: Request, server: { requestIP: (r: Request) => { address: string } | null }): string {
|
||||
if (process.env.GBRAIN_HTTP_TRUST_PROXY === '1') {
|
||||
const xff = req.headers.get('x-forwarded-for');
|
||||
if (xff) {
|
||||
const first = xff.split(',')[0]?.trim();
|
||||
if (first) return first;
|
||||
}
|
||||
const xRealIp = req.headers.get('x-real-ip');
|
||||
if (xRealIp) return xRealIp.trim();
|
||||
}
|
||||
const sock = server.requestIP(req);
|
||||
return sock?.address || 'unknown';
|
||||
}
|
||||
|
||||
export async function startHttpTransport(opts: HttpTransportOptions) {
|
||||
const { port, engine } = opts;
|
||||
|
||||
// Fail-fast: HTTP transport requires Postgres because access_tokens / mcp_request_log
|
||||
// only exist in the Postgres schema (see src/core/pglite-schema.ts:5-6).
|
||||
if ((engine as { kind?: string }).kind !== 'postgres') {
|
||||
console.error('Error: gbrain serve --http requires a Postgres engine for remote auth tokens.');
|
||||
console.error('PGLite is local-only by design (access_tokens table is Postgres-only).');
|
||||
console.error('Either:');
|
||||
console.error(' - Use stdio: gbrain serve');
|
||||
console.error(' - Migrate to Postgres: gbrain migrate --to supabase');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const sql = (engine as unknown as { sql: any }).sql;
|
||||
if (!sql) {
|
||||
console.error('Error: Postgres engine has no .sql client. Engine may not be connected.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const limiters = opts.limiters || buildDefaultLimiters();
|
||||
const bodyCap = envInt('GBRAIN_HTTP_MAX_BODY_BYTES', DEFAULT_BODY_CAP);
|
||||
const corsAllowlist = parseCorsAllowlist();
|
||||
const tools = buildToolDefs(operations);
|
||||
|
||||
function corsHeaders(origin: string | null, extra: Record<string, string> = {}): Record<string, string> {
|
||||
const headers: Record<string, string> = { ...extra };
|
||||
if (corsAllowlist && origin && corsAllowlist.has(origin)) {
|
||||
headers['Access-Control-Allow-Origin'] = origin;
|
||||
headers['Vary'] = 'Origin';
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
function corsPreflightHeaders(origin: string | null): Record<string, string> {
|
||||
const headers: Record<string, string> = {
|
||||
'Access-Control-Allow-Methods': 'POST, GET, OPTIONS',
|
||||
'Access-Control-Allow-Headers': 'Content-Type, Authorization, Accept',
|
||||
};
|
||||
if (corsAllowlist && origin && corsAllowlist.has(origin)) {
|
||||
headers['Access-Control-Allow-Origin'] = origin;
|
||||
headers['Vary'] = 'Origin';
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
async function validateToken(authHeader: string | null): Promise<AuthResult> {
|
||||
if (!authHeader?.startsWith('Bearer ')) return { ok: false };
|
||||
const token = authHeader.slice(7);
|
||||
const hash = hashToken(token);
|
||||
try {
|
||||
const [row] = await sql`
|
||||
SELECT id, name FROM access_tokens
|
||||
WHERE token_hash = ${hash} AND revoked_at IS NULL
|
||||
`;
|
||||
if (!row) return { ok: false };
|
||||
// Debounced last_used_at update — only writes once per token per 60s.
|
||||
// SQL-level WHERE clause keeps this race-tolerant even under concurrent requests.
|
||||
sql`UPDATE access_tokens
|
||||
SET last_used_at = now()
|
||||
WHERE id = ${row.id}
|
||||
AND (last_used_at IS NULL OR last_used_at < now() - interval '60 seconds')`
|
||||
.catch(() => { /* fire-and-forget */ });
|
||||
return { ok: true, tokenId: row.id, tokenName: row.name };
|
||||
} catch {
|
||||
return { ok: false };
|
||||
}
|
||||
}
|
||||
|
||||
function logRequest(tokenName: string | null, operation: string, status: string, latencyMs: number) {
|
||||
sql`INSERT INTO mcp_request_log (token_name, operation, latency_ms, status)
|
||||
VALUES (${tokenName}, ${operation}, ${latencyMs}, ${status})`
|
||||
.catch(() => { /* best-effort */ });
|
||||
}
|
||||
|
||||
const server = Bun.serve({
|
||||
port,
|
||||
async fetch(req, server) {
|
||||
const startedMs = Date.now();
|
||||
const url = new URL(req.url);
|
||||
const path = url.pathname;
|
||||
const origin = req.headers.get('origin');
|
||||
|
||||
// CORS preflight
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { headers: corsPreflightHeaders(origin) });
|
||||
}
|
||||
|
||||
// Health check — no auth, no rate limit. Probes the DB so orchestration
|
||||
// doesn't see "ok" while clients are getting misleading 401s during a DB outage.
|
||||
if (path === '/health') {
|
||||
try {
|
||||
await sql`SELECT 1`;
|
||||
return Response.json(
|
||||
{ status: 'ok', version: VERSION, transport: 'http', db: 'ok' },
|
||||
{ headers: corsHeaders(origin) },
|
||||
);
|
||||
} catch (e: any) {
|
||||
return Response.json(
|
||||
{ status: 'unhealthy', version: VERSION, transport: 'http', db: 'unreachable', error: e?.message ?? 'unknown' },
|
||||
{ status: 503, headers: corsHeaders(origin) },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (path !== '/mcp') {
|
||||
return Response.json({ error: 'not_found' }, { status: 404, headers: corsHeaders(origin) });
|
||||
}
|
||||
if (req.method !== 'POST') {
|
||||
return Response.json({ error: 'method_not_allowed' }, { status: 405, headers: corsHeaders(origin) });
|
||||
}
|
||||
|
||||
const ip = resolveClientIp(req, server);
|
||||
|
||||
// Pre-auth IP rate limit. Fires BEFORE the DB lookup so we actually limit brute-force load.
|
||||
const ipCheck = limiters.ip.check(ip);
|
||||
if (!ipCheck.allowed) {
|
||||
logRequest(null, 'unknown', 'rate_limited', Date.now() - startedMs);
|
||||
return Response.json(
|
||||
{ error: 'rate_limited', message: 'Too many requests' },
|
||||
{
|
||||
status: 429,
|
||||
headers: corsHeaders(origin, { 'Retry-After': String(ipCheck.retryAfter ?? 60) }),
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// Body cap (stream-counted; chunked transfers caught here, not at req.json).
|
||||
const bodyText = await readBodyWithCap(req, bodyCap);
|
||||
if (bodyText === null) {
|
||||
logRequest(null, 'unknown', 'body_too_large', Date.now() - startedMs);
|
||||
return Response.json(
|
||||
{ error: 'payload_too_large', message: `Request body exceeds ${bodyCap} bytes` },
|
||||
{ status: 413, headers: corsHeaders(origin) },
|
||||
);
|
||||
}
|
||||
|
||||
// Auth.
|
||||
const auth = await validateToken(req.headers.get('Authorization'));
|
||||
if (!auth.ok) {
|
||||
logRequest(null, 'unknown', 'auth_failed', Date.now() - startedMs);
|
||||
return Response.json(
|
||||
{ error: 'invalid_token', message: 'Bearer token required. Create one: gbrain auth create <name>' },
|
||||
{ status: 401, headers: corsHeaders(origin) },
|
||||
);
|
||||
}
|
||||
|
||||
// Post-auth token-id rate limit. Limits runaway authed clients.
|
||||
const tokCheck = limiters.token.check(auth.tokenId!);
|
||||
if (!tokCheck.allowed) {
|
||||
logRequest(auth.tokenName!, 'unknown', 'rate_limited', Date.now() - startedMs);
|
||||
return Response.json(
|
||||
{ error: 'rate_limited', message: 'Too many requests for this token' },
|
||||
{
|
||||
status: 429,
|
||||
headers: corsHeaders(origin, { 'Retry-After': String(tokCheck.retryAfter ?? 60) }),
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// Parse JSON-RPC body.
|
||||
let body: { method?: string; params?: any; id?: any };
|
||||
try {
|
||||
body = JSON.parse(bodyText);
|
||||
} catch (e: any) {
|
||||
logRequest(auth.tokenName!, 'unknown', 'parse_error', Date.now() - startedMs);
|
||||
return Response.json(
|
||||
{ error: 'parse_error', message: e?.message ?? 'invalid JSON' },
|
||||
{ status: 400, headers: corsHeaders(origin) },
|
||||
);
|
||||
}
|
||||
|
||||
const { method, params, id } = body;
|
||||
|
||||
// initialize
|
||||
if (method === 'initialize') {
|
||||
logRequest(auth.tokenName!, 'initialize', 'success', Date.now() - startedMs);
|
||||
return Response.json(
|
||||
{
|
||||
result: {
|
||||
protocolVersion: '2025-03-26',
|
||||
serverInfo: { name: 'gbrain', version: VERSION },
|
||||
capabilities: { tools: {} },
|
||||
},
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
},
|
||||
{ headers: corsHeaders(origin) },
|
||||
);
|
||||
}
|
||||
|
||||
// notifications/initialized — acknowledge with 204
|
||||
if (method === 'notifications/initialized') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders(origin) });
|
||||
}
|
||||
|
||||
// tools/list
|
||||
if (method === 'tools/list') {
|
||||
logRequest(auth.tokenName!, 'tools/list', 'success', Date.now() - startedMs);
|
||||
return Response.json(
|
||||
{ result: { tools }, jsonrpc: '2.0', id },
|
||||
{ headers: corsHeaders(origin) },
|
||||
);
|
||||
}
|
||||
|
||||
// tools/call — dispatch through shared dispatch.ts (parity with stdio)
|
||||
if (method === 'tools/call') {
|
||||
const toolName: string = params?.name ?? 'unknown';
|
||||
const args: Record<string, unknown> = params?.arguments ?? {};
|
||||
const result = await dispatchToolCall(engine, toolName, args, { remote: true });
|
||||
const status = result.isError ? 'error' : 'success';
|
||||
logRequest(auth.tokenName!, `tools/call:${toolName}`, status, Date.now() - startedMs);
|
||||
return Response.json(
|
||||
{ result, jsonrpc: '2.0', id },
|
||||
{ headers: corsHeaders(origin) },
|
||||
);
|
||||
}
|
||||
|
||||
logRequest(auth.tokenName!, method ?? 'unknown', 'unknown_method', Date.now() - startedMs);
|
||||
return Response.json(
|
||||
{ error: 'unknown_method', message: `Unknown method: ${method}` },
|
||||
{ status: 400, headers: corsHeaders(origin) },
|
||||
);
|
||||
},
|
||||
});
|
||||
|
||||
console.error(`GBrain HTTP MCP server running on port ${port}`);
|
||||
console.error(` Health: http://localhost:${port}/health`);
|
||||
console.error(` MCP: http://localhost:${port}/mcp`);
|
||||
console.error(` Auth: Bearer token required (create with: gbrain auth create <name>)`);
|
||||
if (!corsAllowlist) {
|
||||
console.error(' CORS: default-deny. Set GBRAIN_HTTP_CORS_ORIGIN=https://your.app to allow browser clients.');
|
||||
} else {
|
||||
console.error(` CORS: allowlist = ${[...corsAllowlist].join(', ')}`);
|
||||
}
|
||||
console.error('');
|
||||
console.error('⚠️ Do NOT use open OAuth registration for remote MCP access.');
|
||||
console.error(' Tokens are managed via: gbrain auth create/list/revoke');
|
||||
|
||||
return server;
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
/**
|
||||
* Rate limiter for `gbrain serve --http`.
|
||||
*
|
||||
* Token-bucket per key, stored in a bounded LRU map so attacker-controlled keys
|
||||
* can't grow memory unbounded. TTL prune on every access (entries older than
|
||||
* 2× window are evicted) so abandoned keys don't sit around forever.
|
||||
*
|
||||
* Two buckets in the request pipeline (see http-transport.ts):
|
||||
* 1. Pre-auth IP bucket — fires BEFORE the DB lookup so we actually limit
|
||||
* brute-force load against access_tokens, not just response codes.
|
||||
* 2. Post-auth token-id bucket — fires after auth so legitimate-but-runaway
|
||||
* clients get throttled at the right principal.
|
||||
*
|
||||
* Both buckets behave identically; only the key differs.
|
||||
*/
|
||||
|
||||
export interface RateLimitOpts {
|
||||
/** Maximum requests in the window. */
|
||||
limit: number;
|
||||
/** Window length in milliseconds. */
|
||||
windowMs: number;
|
||||
/** LRU cap on distinct keys. Evicts least-recently-used on overflow. */
|
||||
lruCap: number;
|
||||
}
|
||||
|
||||
export interface RateLimitResult {
|
||||
allowed: boolean;
|
||||
/** Seconds until next request would be allowed (only set when !allowed). */
|
||||
retryAfter?: number;
|
||||
/** Tokens remaining in the bucket after this check. */
|
||||
remaining: number;
|
||||
}
|
||||
|
||||
interface Bucket {
|
||||
tokens: number;
|
||||
/** Used for refill math: tokens accrue based on elapsed time since this. */
|
||||
lastRefillMs: number;
|
||||
/** Used for TTL eviction: time of last check, regardless of refill. Prevents bucket-reset attack
|
||||
* where an exhausted key would otherwise get TTL-evicted and recreated fresh. */
|
||||
lastTouchedMs: number;
|
||||
}
|
||||
|
||||
/** Clock function — defaults to Date.now, overridable for tests. */
|
||||
type Clock = () => number;
|
||||
|
||||
export class RateLimiter {
|
||||
readonly opts: RateLimitOpts;
|
||||
private readonly buckets: Map<string, Bucket> = new Map();
|
||||
private readonly clock: Clock;
|
||||
|
||||
constructor(opts: RateLimitOpts, clock: Clock = Date.now) {
|
||||
if (opts.limit <= 0) throw new Error('RateLimiter: limit must be > 0');
|
||||
if (opts.windowMs <= 0) throw new Error('RateLimiter: windowMs must be > 0');
|
||||
if (opts.lruCap <= 0) throw new Error('RateLimiter: lruCap must be > 0');
|
||||
this.opts = opts;
|
||||
this.clock = clock;
|
||||
}
|
||||
|
||||
check(key: string): RateLimitResult {
|
||||
const now = this.clock();
|
||||
this.prune(now);
|
||||
|
||||
let bucket = this.buckets.get(key);
|
||||
if (!bucket) {
|
||||
bucket = { tokens: this.opts.limit, lastRefillMs: now, lastTouchedMs: now };
|
||||
} else {
|
||||
// Refill: tokens accrue continuously over the window. limit/windowMs tokens per ms.
|
||||
const elapsed = now - bucket.lastRefillMs;
|
||||
const refilled = Math.floor((elapsed * this.opts.limit) / this.opts.windowMs);
|
||||
if (refilled > 0) {
|
||||
bucket.tokens = Math.min(this.opts.limit, bucket.tokens + refilled);
|
||||
bucket.lastRefillMs = now;
|
||||
}
|
||||
bucket.lastTouchedMs = now;
|
||||
// LRU bookkeeping: re-insert to move to end (Map iteration order = insertion).
|
||||
this.buckets.delete(key);
|
||||
}
|
||||
|
||||
if (bucket.tokens > 0) {
|
||||
bucket.tokens -= 1;
|
||||
this.buckets.set(key, bucket);
|
||||
this.evictIfOver();
|
||||
return { allowed: true, remaining: bucket.tokens };
|
||||
}
|
||||
|
||||
// No tokens. Compute Retry-After from when the next token will accrue.
|
||||
const msPerToken = this.opts.windowMs / this.opts.limit;
|
||||
const msUntilNext = msPerToken - (now - bucket.lastRefillMs);
|
||||
const retryAfter = Math.max(1, Math.ceil(msUntilNext / 1000));
|
||||
this.buckets.set(key, bucket);
|
||||
this.evictIfOver();
|
||||
return { allowed: false, retryAfter, remaining: 0 };
|
||||
}
|
||||
|
||||
/** Evict TTL-expired entries (older than 2× window since last touch). Cheap: O(n) but n is bounded by lruCap.
|
||||
* Uses lastTouchedMs (not lastRefillMs) so an attacker can't reset their bucket by hammering an exhausted key
|
||||
* past the TTL — every check updates lastTouchedMs even when refill produces 0 tokens. */
|
||||
private prune(now: number): void {
|
||||
const ttl = this.opts.windowMs * 2;
|
||||
for (const [key, bucket] of this.buckets) {
|
||||
if (now - bucket.lastTouchedMs > ttl) {
|
||||
this.buckets.delete(key);
|
||||
} else {
|
||||
// Map iteration is in insertion order; once we hit a fresh entry, the rest are also fresh
|
||||
// ONLY if we maintain insertion-order = recency. That holds because check() does delete+set on every call.
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private evictIfOver(): void {
|
||||
while (this.buckets.size > this.opts.lruCap) {
|
||||
// Map iteration starts at oldest (first-inserted). Delete it.
|
||||
const oldestKey = this.buckets.keys().next().value;
|
||||
if (oldestKey === undefined) break;
|
||||
this.buckets.delete(oldestKey);
|
||||
}
|
||||
}
|
||||
|
||||
/** Test helper: current key count. */
|
||||
get size(): number {
|
||||
return this.buckets.size;
|
||||
}
|
||||
}
|
||||
|
||||
/** Parse a positive integer env var, falling back to default. */
|
||||
function envInt(name: string, fallback: number): number {
|
||||
const v = process.env[name];
|
||||
if (!v) return fallback;
|
||||
const n = parseInt(v, 10);
|
||||
return Number.isFinite(n) && n > 0 ? n : fallback;
|
||||
}
|
||||
|
||||
/** Build limiters from env. Keep this lazy — tests can construct RateLimiter directly. */
|
||||
export function buildDefaultLimiters(clock: Clock = Date.now): { ip: RateLimiter; token: RateLimiter } {
|
||||
const lruCap = envInt('GBRAIN_HTTP_RATE_LIMIT_LRU', 10000);
|
||||
const windowMs = 60_000;
|
||||
return {
|
||||
ip: new RateLimiter({ limit: envInt('GBRAIN_HTTP_RATE_LIMIT_IP', 30), windowMs, lruCap }, clock),
|
||||
token: new RateLimiter({ limit: envInt('GBRAIN_HTTP_RATE_LIMIT_TOKEN', 60), windowMs, lruCap }, clock),
|
||||
};
|
||||
}
|
||||
+13
-66
@@ -2,30 +2,10 @@ import { Server } from '@modelcontextprotocol/sdk/server/index.js';
|
||||
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
|
||||
import { ListToolsRequestSchema, CallToolRequestSchema } from '@modelcontextprotocol/sdk/types.js';
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { operations, OperationError } from '../core/operations.ts';
|
||||
import type { Operation, OperationContext } from '../core/operations.ts';
|
||||
import { loadConfig } from '../core/config.ts';
|
||||
import { operations } from '../core/operations.ts';
|
||||
import { VERSION } from '../version.ts';
|
||||
import { buildToolDefs } from './tool-defs.ts';
|
||||
|
||||
/** Validate required params exist and have the expected type */
|
||||
function validateParams(op: Operation, params: Record<string, unknown>): string | null {
|
||||
for (const [key, def] of Object.entries(op.params)) {
|
||||
if (def.required && (params[key] === undefined || params[key] === null)) {
|
||||
return `Missing required parameter: ${key}`;
|
||||
}
|
||||
if (params[key] !== undefined && params[key] !== null) {
|
||||
const val = params[key];
|
||||
const expected = def.type;
|
||||
if (expected === 'string' && typeof val !== 'string') return `Parameter "${key}" must be a string`;
|
||||
if (expected === 'number' && typeof val !== 'number') return `Parameter "${key}" must be a number`;
|
||||
if (expected === 'boolean' && typeof val !== 'boolean') return `Parameter "${key}" must be a boolean`;
|
||||
if (expected === 'object' && (typeof val !== 'object' || Array.isArray(val))) return `Parameter "${key}" must be an object`;
|
||||
if (expected === 'array' && !Array.isArray(val)) return `Parameter "${key}" must be an array`;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
import { dispatchToolCall, validateParams, buildOperationContext } from './dispatch.ts';
|
||||
|
||||
export async function startMcpServer(engine: BrainEngine) {
|
||||
const server = new Server(
|
||||
@@ -40,50 +20,21 @@ export async function startMcpServer(engine: BrainEngine) {
|
||||
tools: buildToolDefs(operations),
|
||||
}));
|
||||
|
||||
// Dispatch tool calls to operation handlers
|
||||
server.setRequestHandler(CallToolRequestSchema, async (request: any) => {
|
||||
// Dispatch tool calls via shared dispatch.ts (parity with HTTP transport).
|
||||
// MCP stdio callers are remote/untrusted; dispatch defaults remote=true.
|
||||
// The MCP SDK's response type widened in 1.29 to allow a managed-task wrapper;
|
||||
// gbrain ops are synchronous, so we return the legacy `{ content, isError? }`
|
||||
// shape and cast through `any` (the SDK accepts it via the ServerResult union).
|
||||
server.setRequestHandler(CallToolRequestSchema, async (request: any): Promise<any> => {
|
||||
const { name, arguments: params } = request.params;
|
||||
const op = operations.find(o => o.name === name);
|
||||
if (!op) {
|
||||
return { content: [{ type: 'text', text: `Error: Unknown tool: ${name}` }], isError: true };
|
||||
}
|
||||
|
||||
const ctx: OperationContext = {
|
||||
engine,
|
||||
config: loadConfig() || { engine: 'postgres' },
|
||||
logger: {
|
||||
info: (msg: string) => process.stderr.write(`[info] ${msg}\n`),
|
||||
warn: (msg: string) => process.stderr.write(`[warn] ${msg}\n`),
|
||||
error: (msg: string) => process.stderr.write(`[error] ${msg}\n`),
|
||||
},
|
||||
dryRun: !!(params?.dry_run),
|
||||
// MCP stdio callers are remote/untrusted; enforce strict file confinement.
|
||||
remote: true,
|
||||
};
|
||||
|
||||
const safeParams = params || {};
|
||||
const validationError = validateParams(op, safeParams);
|
||||
if (validationError) {
|
||||
return { content: [{ type: 'text', text: JSON.stringify({ error: 'invalid_params', message: validationError }, null, 2) }], isError: true };
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await op.handler(ctx, safeParams);
|
||||
return { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] };
|
||||
} catch (e: unknown) {
|
||||
if (e instanceof OperationError) {
|
||||
return { content: [{ type: 'text', text: JSON.stringify(e.toJSON(), null, 2) }], isError: true };
|
||||
}
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
return { content: [{ type: 'text', text: `Error: ${msg}` }], isError: true };
|
||||
}
|
||||
return dispatchToolCall(engine, name, params, { remote: true });
|
||||
});
|
||||
|
||||
const transport = new StdioServerTransport();
|
||||
await server.connect(transport);
|
||||
}
|
||||
|
||||
// Backward compat: used by `gbrain call` command
|
||||
// Backward compat: used by `gbrain call` command (trusted local path).
|
||||
export async function handleToolCall(
|
||||
engine: BrainEngine,
|
||||
tool: string,
|
||||
@@ -95,14 +46,10 @@ export async function handleToolCall(
|
||||
const validationError = validateParams(op, params);
|
||||
if (validationError) throw new Error(validationError);
|
||||
|
||||
const ctx: OperationContext = {
|
||||
engine,
|
||||
config: loadConfig() || { engine: 'postgres' },
|
||||
logger: { info: console.log, warn: console.warn, error: console.error },
|
||||
dryRun: !!(params?.dry_run),
|
||||
// Backing path for `gbrain call` CLI command — trusted local invocation.
|
||||
const ctx = buildOperationContext(engine, params, {
|
||||
remote: false,
|
||||
};
|
||||
logger: { info: console.log, warn: console.warn, error: console.error },
|
||||
});
|
||||
|
||||
return op.handler(ctx, params);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,233 @@
|
||||
/**
|
||||
* E2E tests for src/mcp/http-transport.ts against real Postgres.
|
||||
*
|
||||
* Catches schema drift (column-name typos that would slip past the unit suite's
|
||||
* stubbed engine.sql) and proves the F1+F2+F3 dispatch pipeline works against a
|
||||
* real handler doing real DB work. Also exercises the SQL-level last_used_at
|
||||
* debounce against real Postgres semantics.
|
||||
*
|
||||
* Run: DATABASE_URL=... bun test test/e2e/http-transport.test.ts
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { createHash, randomBytes } from 'crypto';
|
||||
import { startHttpTransport } from '../../src/mcp/http-transport.ts';
|
||||
import { hasDatabase, setupDB, teardownDB, getEngine, getConn } from './helpers.ts';
|
||||
|
||||
const skip = !hasDatabase();
|
||||
const describeE2E = skip ? describe.skip : describe;
|
||||
|
||||
if (skip) {
|
||||
console.log('Skipping E2E http-transport tests (DATABASE_URL not set)');
|
||||
}
|
||||
|
||||
interface ServerHandle {
|
||||
port: number;
|
||||
stop: () => Promise<void>;
|
||||
}
|
||||
|
||||
function generateToken(): string {
|
||||
return 'gbrain_test_' + randomBytes(16).toString('hex');
|
||||
}
|
||||
|
||||
function hashToken(token: string): string {
|
||||
return createHash('sha256').update(token).digest('hex');
|
||||
}
|
||||
|
||||
async function startServer(): Promise<ServerHandle> {
|
||||
const engine = getEngine();
|
||||
const server = await startHttpTransport({ port: 0, engine: engine as any });
|
||||
return {
|
||||
port: (server as any).port,
|
||||
stop: async () => { (server as any).stop(true); },
|
||||
};
|
||||
}
|
||||
|
||||
function rpc(method: string, params?: unknown, id: number = 1) {
|
||||
return JSON.stringify({ jsonrpc: '2.0', id, method, ...(params !== undefined ? { params } : {}) });
|
||||
}
|
||||
|
||||
describeE2E('http-transport E2E (real Postgres)', () => {
|
||||
let srv: ServerHandle;
|
||||
let validToken: string;
|
||||
let revokedToken: string;
|
||||
let validTokenName: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
await setupDB();
|
||||
const conn = getConn();
|
||||
|
||||
// Seed a valid + revoked token directly via SQL (mirrors auth.ts's create path).
|
||||
validToken = generateToken();
|
||||
validTokenName = 'e2e-valid-' + randomBytes(4).toString('hex');
|
||||
await conn.unsafe(
|
||||
'INSERT INTO access_tokens (name, token_hash) VALUES ($1, $2)',
|
||||
[validTokenName, hashToken(validToken)],
|
||||
);
|
||||
revokedToken = generateToken();
|
||||
await conn.unsafe(
|
||||
'INSERT INTO access_tokens (name, token_hash, revoked_at) VALUES ($1, $2, now())',
|
||||
['e2e-revoked-' + randomBytes(4).toString('hex'), hashToken(revokedToken)],
|
||||
);
|
||||
|
||||
srv = await startServer();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
if (srv) await srv.stop();
|
||||
await teardownDB();
|
||||
});
|
||||
|
||||
test('1. /health → 200 with expected JSON shape', async () => {
|
||||
const r = await fetch(`http://localhost:${srv.port}/health`);
|
||||
expect(r.status).toBe(200);
|
||||
const body = await r.json();
|
||||
expect(body.status).toBe('ok');
|
||||
expect(body.transport).toBe('http');
|
||||
expect(body.version).toBeString();
|
||||
});
|
||||
|
||||
test('2. /mcp tools/list with valid Bearer → 200 + ops list', async () => {
|
||||
const r = await fetch(`http://localhost:${srv.port}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
expect(r.status).toBe(200);
|
||||
const body = await r.json();
|
||||
expect(body.result.tools).toBeArray();
|
||||
expect(body.result.tools.length).toBeGreaterThan(5);
|
||||
expect(r.headers.get('content-type')).toContain('application/json');
|
||||
});
|
||||
|
||||
test('3. /mcp tools/call (real op: list_pages) round-trips successfully — F1+F2+F3 guard', async () => {
|
||||
const r = await fetch(`http://localhost:${srv.port}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/call', { name: 'list_pages', arguments: { limit: 5 } }),
|
||||
});
|
||||
expect(r.status).toBe(200);
|
||||
const body = await r.json();
|
||||
expect(body.jsonrpc).toBe('2.0');
|
||||
expect(body.result.content).toBeArray();
|
||||
// Should NOT be an error — handler ran successfully against the real engine.
|
||||
expect(body.result.isError).toBeUndefined();
|
||||
// Result text should parse as JSON (list_pages returns an object/array)
|
||||
const resultText = body.result.content[0].text;
|
||||
const parsed = JSON.parse(resultText);
|
||||
expect(parsed).toBeDefined();
|
||||
});
|
||||
|
||||
test('4. revoked token → 401', async () => {
|
||||
const r = await fetch(`http://localhost:${srv.port}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${revokedToken}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
expect(r.status).toBe(401);
|
||||
});
|
||||
|
||||
test('5. last_used_at debounce: two consecutive valid calls → only one UPDATE within 60s', async () => {
|
||||
const conn = getConn();
|
||||
|
||||
// Reset last_used_at to NULL so the first call definitely updates
|
||||
await conn.unsafe('UPDATE access_tokens SET last_used_at = NULL WHERE name = $1', [validTokenName]);
|
||||
|
||||
// First request — should update last_used_at
|
||||
await fetch(`http://localhost:${srv.port}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
// Give the fire-and-forget UPDATE a moment to land
|
||||
await new Promise(r => setTimeout(r, 50));
|
||||
|
||||
const [row1] = await conn.unsafe(
|
||||
'SELECT last_used_at FROM access_tokens WHERE name = $1',
|
||||
[validTokenName],
|
||||
) as { last_used_at: Date | null }[];
|
||||
expect(row1.last_used_at).not.toBeNull();
|
||||
const firstUpdate = row1.last_used_at;
|
||||
|
||||
// Second request immediately — should NOT trigger another UPDATE (debounced by SQL WHERE)
|
||||
await fetch(`http://localhost:${srv.port}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
await new Promise(r => setTimeout(r, 50));
|
||||
|
||||
const [row2] = await conn.unsafe(
|
||||
'SELECT last_used_at FROM access_tokens WHERE name = $1',
|
||||
[validTokenName],
|
||||
) as { last_used_at: Date | null }[];
|
||||
// Same timestamp = same UPDATE = debounce held
|
||||
expect(row2.last_used_at?.getTime()).toBe(firstUpdate?.getTime());
|
||||
});
|
||||
|
||||
test('6. last_used_at debounce: simulating 65s gap → second request DOES update', async () => {
|
||||
const conn = getConn();
|
||||
|
||||
// Set last_used_at to 65 seconds ago — simulates the time gap without waiting in real time
|
||||
await conn.unsafe(
|
||||
`UPDATE access_tokens SET last_used_at = now() - interval '65 seconds' WHERE name = $1`,
|
||||
[validTokenName],
|
||||
);
|
||||
const [before] = await conn.unsafe(
|
||||
'SELECT last_used_at FROM access_tokens WHERE name = $1',
|
||||
[validTokenName],
|
||||
) as { last_used_at: Date | null }[];
|
||||
|
||||
await fetch(`http://localhost:${srv.port}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
await new Promise(r => setTimeout(r, 50));
|
||||
|
||||
const [after] = await conn.unsafe(
|
||||
'SELECT last_used_at FROM access_tokens WHERE name = $1',
|
||||
[validTokenName],
|
||||
) as { last_used_at: Date | null }[];
|
||||
expect(after.last_used_at?.getTime()).toBeGreaterThan(before.last_used_at!.getTime());
|
||||
});
|
||||
|
||||
test('7. mcp_request_log gets a row per request', async () => {
|
||||
const conn = getConn();
|
||||
const beforeRows = await conn.unsafe('SELECT count(*)::int AS n FROM mcp_request_log') as { n: number }[];
|
||||
const beforeN = beforeRows[0].n;
|
||||
|
||||
await fetch(`http://localhost:${srv.port}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
// Fire-and-forget audit insert — give it a tick
|
||||
await new Promise(r => setTimeout(r, 100));
|
||||
|
||||
const afterRows = await conn.unsafe('SELECT count(*)::int AS n FROM mcp_request_log') as { n: number }[];
|
||||
expect(afterRows[0].n).toBeGreaterThan(beforeN);
|
||||
|
||||
const [row] = await conn.unsafe(
|
||||
`SELECT token_name, operation, status, latency_ms FROM mcp_request_log
|
||||
WHERE token_name = $1 ORDER BY created_at DESC LIMIT 1`,
|
||||
[validTokenName],
|
||||
) as { token_name: string; operation: string; status: string; latency_ms: number }[];
|
||||
expect(row.token_name).toBe(validTokenName);
|
||||
expect(row.operation).toBe('tools/list');
|
||||
expect(row.status).toBe('success');
|
||||
expect(row.latency_ms).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
test('8. tools/call with malformed params → isError result with invalid_params', async () => {
|
||||
const r = await fetch(`http://localhost:${srv.port}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/call', { name: 'get_page', arguments: { slug: 42 } }),
|
||||
});
|
||||
expect(r.status).toBe(200);
|
||||
const body = await r.json();
|
||||
expect(body.result.isError).toBe(true);
|
||||
expect(body.result.content[0].text).toContain('invalid_params');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,519 @@
|
||||
/**
|
||||
* Unit tests for src/mcp/http-transport.ts.
|
||||
*
|
||||
* Covers:
|
||||
* - Auth path (valid, missing header, no Bearer prefix, unknown, revoked, /health bypass)
|
||||
* - F1+F2+F3 round-trip guards (handler arg order, full OperationContext, param validation)
|
||||
* - JSON-only response shape (no SSE)
|
||||
* - CORS default-deny + allowlist
|
||||
* - Body cap (Content-Length + chunked)
|
||||
* - Rate limit (token + IP buckets, LRU eviction, TTL prune, /health bypass)
|
||||
*
|
||||
* No DATABASE_URL needed — engine.sql is mocked. E2E coverage of the real Postgres
|
||||
* round-trip lives in test/e2e/http-transport.test.ts.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { createHash } from 'crypto';
|
||||
import { startHttpTransport } from '../src/mcp/http-transport.ts';
|
||||
import { RateLimiter } from '../src/mcp/rate-limit.ts';
|
||||
|
||||
type SqlResult = unknown[] | unknown;
|
||||
type SqlHandler = (query: string, values: unknown[]) => SqlResult | Promise<SqlResult>;
|
||||
|
||||
interface FakeEngine {
|
||||
kind: 'postgres';
|
||||
sql: ReturnType<typeof makeSqlTag>;
|
||||
audit: { token_name: string | null; operation: string; status: string; latency_ms: number }[];
|
||||
}
|
||||
|
||||
function makeSqlTag(handler: SqlHandler) {
|
||||
return (strings: TemplateStringsArray, ...values: unknown[]) => {
|
||||
let query = '';
|
||||
for (let i = 0; i < strings.length; i++) {
|
||||
query += strings[i];
|
||||
if (i < values.length) query += '?';
|
||||
}
|
||||
const result = handler(query.trim(), values);
|
||||
return Promise.resolve(result);
|
||||
};
|
||||
}
|
||||
|
||||
function hash(token: string): string {
|
||||
return createHash('sha256').update(token).digest('hex');
|
||||
}
|
||||
|
||||
interface FakeEngineConfig {
|
||||
validTokens?: Map<string, { id: string; name: string }>;
|
||||
/** Tokens that are present but revoked (revoked_at IS NOT NULL — query returns empty). */
|
||||
revokedTokens?: Set<string>;
|
||||
/** If true, every SELECT throws (simulating DB outage). */
|
||||
dbDown?: boolean;
|
||||
}
|
||||
|
||||
function makeFakeEngine(cfg: FakeEngineConfig = {}): FakeEngine {
|
||||
const validTokens = cfg.validTokens ?? new Map();
|
||||
const revokedTokens = cfg.revokedTokens ?? new Set();
|
||||
const audit: FakeEngine['audit'] = [];
|
||||
|
||||
const sql = makeSqlTag((query, values) => {
|
||||
if (cfg.dbDown && query.startsWith('SELECT')) throw new Error('db down');
|
||||
|
||||
if (query === 'SELECT 1') {
|
||||
// /health DB probe
|
||||
return [{ '?column?': 1 }];
|
||||
}
|
||||
|
||||
if (query.startsWith('SELECT id, name FROM access_tokens')) {
|
||||
const tokenHash = values[0] as string;
|
||||
if (revokedTokens.has(tokenHash)) return [];
|
||||
const row = validTokens.get(tokenHash);
|
||||
return row ? [row] : [];
|
||||
}
|
||||
|
||||
if (query.startsWith('UPDATE access_tokens')) {
|
||||
// last_used_at debounce — succeed silently
|
||||
return [];
|
||||
}
|
||||
|
||||
if (query.startsWith('INSERT INTO mcp_request_log')) {
|
||||
audit.push({
|
||||
token_name: values[0] as string | null,
|
||||
operation: values[1] as string,
|
||||
latency_ms: values[2] as number,
|
||||
status: values[3] as string,
|
||||
});
|
||||
return [];
|
||||
}
|
||||
|
||||
return [];
|
||||
});
|
||||
|
||||
return { kind: 'postgres', sql, audit };
|
||||
}
|
||||
|
||||
interface TestServer {
|
||||
url: string;
|
||||
stop: () => void;
|
||||
engine: FakeEngine;
|
||||
ipLimiter: RateLimiter;
|
||||
tokenLimiter: RateLimiter;
|
||||
}
|
||||
|
||||
let mockNow = 0;
|
||||
function freezeClock(at: number) { mockNow = at; }
|
||||
function advanceClock(deltaMs: number) { mockNow += deltaMs; }
|
||||
|
||||
async function startTest(cfg: FakeEngineConfig & { lruCap?: number; ipLimit?: number; tokenLimit?: number; corsOrigin?: string; bodyCap?: number; trustProxy?: boolean } = {}): Promise<TestServer> {
|
||||
if (cfg.corsOrigin) process.env.GBRAIN_HTTP_CORS_ORIGIN = cfg.corsOrigin;
|
||||
else delete process.env.GBRAIN_HTTP_CORS_ORIGIN;
|
||||
if (cfg.bodyCap) process.env.GBRAIN_HTTP_MAX_BODY_BYTES = String(cfg.bodyCap);
|
||||
else delete process.env.GBRAIN_HTTP_MAX_BODY_BYTES;
|
||||
if (cfg.trustProxy) process.env.GBRAIN_HTTP_TRUST_PROXY = '1';
|
||||
else delete process.env.GBRAIN_HTTP_TRUST_PROXY;
|
||||
|
||||
const engine = makeFakeEngine(cfg);
|
||||
const clock = () => mockNow || Date.now();
|
||||
const ipLimiter = new RateLimiter(
|
||||
{ limit: cfg.ipLimit ?? 1000, windowMs: 60_000, lruCap: cfg.lruCap ?? 10000 },
|
||||
clock,
|
||||
);
|
||||
const tokenLimiter = new RateLimiter(
|
||||
{ limit: cfg.tokenLimit ?? 1000, windowMs: 60_000, lruCap: cfg.lruCap ?? 10000 },
|
||||
clock,
|
||||
);
|
||||
const server = await startHttpTransport({
|
||||
port: 0,
|
||||
engine: engine as any,
|
||||
limiters: { ip: ipLimiter, token: tokenLimiter },
|
||||
});
|
||||
return {
|
||||
url: `http://localhost:${(server as any).port}`,
|
||||
stop: () => (server as any).stop(true),
|
||||
engine,
|
||||
ipLimiter,
|
||||
tokenLimiter,
|
||||
};
|
||||
}
|
||||
|
||||
function rpc(method: string, params?: unknown, id: number = 1) {
|
||||
return JSON.stringify({ jsonrpc: '2.0', id, method, ...(params !== undefined ? { params } : {}) });
|
||||
}
|
||||
|
||||
// --------------------------------------------------------------------------
|
||||
// Auth path
|
||||
// --------------------------------------------------------------------------
|
||||
|
||||
describe('http-transport: auth', () => {
|
||||
let srv: TestServer;
|
||||
const VALID_TOKEN = 'valid-token-abc';
|
||||
const REVOKED_TOKEN = 'revoked-token-xyz';
|
||||
|
||||
beforeAll(async () => {
|
||||
srv = await startTest({
|
||||
validTokens: new Map([[hash(VALID_TOKEN), { id: 'tok-1', name: 'test' }]]),
|
||||
revokedTokens: new Set([hash(REVOKED_TOKEN)]),
|
||||
});
|
||||
});
|
||||
afterAll(() => srv.stop());
|
||||
|
||||
test('1. valid token → 200 + tools/list returns ops', async () => {
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${VALID_TOKEN}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
expect(r.status).toBe(200);
|
||||
const body = await r.json();
|
||||
expect(body.result.tools).toBeArray();
|
||||
expect(body.result.tools.length).toBeGreaterThan(0);
|
||||
expect(body.jsonrpc).toBe('2.0');
|
||||
});
|
||||
|
||||
test('2. missing Authorization header → 401', async () => {
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
expect(r.status).toBe(401);
|
||||
});
|
||||
|
||||
test('3. header missing Bearer prefix → 401', async () => {
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': VALID_TOKEN, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
expect(r.status).toBe(401);
|
||||
});
|
||||
|
||||
test('4. unknown token → 401', async () => {
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': 'Bearer not-a-real-token', 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
expect(r.status).toBe(401);
|
||||
});
|
||||
|
||||
test('5. revoked token → 401', async () => {
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${REVOKED_TOKEN}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
expect(r.status).toBe(401);
|
||||
});
|
||||
|
||||
test('6. /health → 200 without auth, body has expected fields, probes DB', async () => {
|
||||
const r = await fetch(`${srv.url}/health`);
|
||||
expect(r.status).toBe(200);
|
||||
const body = await r.json();
|
||||
expect(body.status).toBe('ok');
|
||||
expect(body.transport).toBe('http');
|
||||
expect(body.version).toBeString();
|
||||
expect(body.db).toBe('ok');
|
||||
});
|
||||
|
||||
test('6b. /health → 503 when DB is unreachable', async () => {
|
||||
const dbDownSrv = await startTest({ dbDown: true });
|
||||
try {
|
||||
const r = await fetch(`${dbDownSrv.url}/health`);
|
||||
expect(r.status).toBe(503);
|
||||
const body = await r.json();
|
||||
expect(body.status).toBe('unhealthy');
|
||||
expect(body.db).toBe('unreachable');
|
||||
} finally { dbDownSrv.stop(); }
|
||||
});
|
||||
});
|
||||
|
||||
// --------------------------------------------------------------------------
|
||||
// F1+F2+F3 regression guards (the actual existing-PR bugs)
|
||||
// --------------------------------------------------------------------------
|
||||
|
||||
describe('http-transport: tools/call dispatch', () => {
|
||||
let srv: TestServer;
|
||||
const TOK = 'tok-fix';
|
||||
|
||||
beforeAll(async () => {
|
||||
srv = await startTest({ validTokens: new Map([[hash(TOK), { id: 'tok-fix-id', name: 'fix' }]]) });
|
||||
});
|
||||
afterAll(() => srv.stop());
|
||||
|
||||
test('7. tools/call with a real op (list_pages) round-trips successfully (F1+F2 guard)', async () => {
|
||||
// list_pages doesn't need real DB rows in this stub — it'll call engine methods we don't mock,
|
||||
// so we expect EITHER a successful tool-result OR an isError result with a meaningful message.
|
||||
// The point is that the handler IS invoked with (ctx, params) order — not (params, ctx).
|
||||
// If F1 regressed, the handler would receive {limit: 1} as ctx and crash trying to read ctx.engine.
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/call', { name: 'list_pages', arguments: { limit: 1 } }),
|
||||
});
|
||||
expect(r.status).toBe(200);
|
||||
const body = await r.json();
|
||||
expect(body.jsonrpc).toBe('2.0');
|
||||
expect(body.result).toBeDefined();
|
||||
expect(body.result.content).toBeArray();
|
||||
// Either success (handler ran) or a structured error (handler ran and returned an error)
|
||||
// — both prove dispatch reached the handler with the correct shape.
|
||||
});
|
||||
|
||||
test('8. tools/call with malformed params → 200 wrapping an isError result (F3 guard via dispatch.ts)', async () => {
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' },
|
||||
// get_page expects `slug` as required string; passing a number triggers validateParams
|
||||
body: rpc('tools/call', { name: 'get_page', arguments: { slug: 42 } }),
|
||||
});
|
||||
expect(r.status).toBe(200);
|
||||
const body = await r.json();
|
||||
expect(body.result.isError).toBe(true);
|
||||
const text = body.result.content[0].text;
|
||||
expect(text).toContain('invalid_params');
|
||||
});
|
||||
|
||||
test('9. /mcp response has Content-Type: application/json (not SSE)', async () => {
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/list'),
|
||||
});
|
||||
expect(r.headers.get('content-type')).toContain('application/json');
|
||||
expect(r.headers.get('content-type')).not.toContain('event-stream');
|
||||
});
|
||||
|
||||
test('9b. unknown tool name → 200 wrapping an isError result', async () => {
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' },
|
||||
body: rpc('tools/call', { name: 'definitely_not_a_real_tool', arguments: {} }),
|
||||
});
|
||||
expect(r.status).toBe(200);
|
||||
const body = await r.json();
|
||||
expect(body.result.isError).toBe(true);
|
||||
expect(body.result.content[0].text).toContain('Unknown tool');
|
||||
});
|
||||
});
|
||||
|
||||
// --------------------------------------------------------------------------
|
||||
// CORS
|
||||
// --------------------------------------------------------------------------
|
||||
|
||||
describe('http-transport: CORS', () => {
|
||||
test('10. no GBRAIN_HTTP_CORS_ORIGIN + browser request → no ACAO header', async () => {
|
||||
const srv = await startTest({});
|
||||
try {
|
||||
const r = await fetch(`${srv.url}/health`, { headers: { 'Origin': 'https://evil.example' } });
|
||||
expect(r.headers.get('access-control-allow-origin')).toBeNull();
|
||||
} finally { srv.stop(); }
|
||||
});
|
||||
|
||||
test('11. env set + matching Origin → ACAO echoes', async () => {
|
||||
const srv = await startTest({ corsOrigin: 'https://claude.ai' });
|
||||
try {
|
||||
const r = await fetch(`${srv.url}/health`, { headers: { 'Origin': 'https://claude.ai' } });
|
||||
expect(r.headers.get('access-control-allow-origin')).toBe('https://claude.ai');
|
||||
expect(r.headers.get('vary')).toBe('Origin');
|
||||
} finally { srv.stop(); }
|
||||
});
|
||||
|
||||
test('12. env set + non-matching Origin → no ACAO header', async () => {
|
||||
const srv = await startTest({ corsOrigin: 'https://claude.ai' });
|
||||
try {
|
||||
const r = await fetch(`${srv.url}/health`, { headers: { 'Origin': 'https://evil.example' } });
|
||||
expect(r.headers.get('access-control-allow-origin')).toBeNull();
|
||||
} finally { srv.stop(); }
|
||||
});
|
||||
});
|
||||
|
||||
// --------------------------------------------------------------------------
|
||||
// Body cap
|
||||
// --------------------------------------------------------------------------
|
||||
|
||||
describe('http-transport: body cap', () => {
|
||||
const TOK = 'body-cap-tok';
|
||||
|
||||
test('13. Content-Length over cap → 413', async () => {
|
||||
const srv = await startTest({
|
||||
validTokens: new Map([[hash(TOK), { id: 'b-1', name: 'b' }]]),
|
||||
bodyCap: 100,
|
||||
});
|
||||
try {
|
||||
const big = 'x'.repeat(200);
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' },
|
||||
body: big,
|
||||
});
|
||||
expect(r.status).toBe(413);
|
||||
} finally { srv.stop(); }
|
||||
});
|
||||
|
||||
test('14. chunked transfer (no Content-Length) over cap → 413', async () => {
|
||||
const srv = await startTest({
|
||||
validTokens: new Map([[hash(TOK), { id: 'b-2', name: 'b' }]]),
|
||||
bodyCap: 100,
|
||||
});
|
||||
try {
|
||||
// Build a chunked body via a ReadableStream — Bun fetch sends without Content-Length.
|
||||
const stream = new ReadableStream({
|
||||
start(controller) {
|
||||
for (let i = 0; i < 10; i++) controller.enqueue(new TextEncoder().encode('y'.repeat(50)));
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
const r = await fetch(`${srv.url}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' },
|
||||
body: stream as any,
|
||||
// @ts-expect-error Bun fetch supports duplex for streaming bodies
|
||||
duplex: 'half',
|
||||
});
|
||||
expect(r.status).toBe(413);
|
||||
} finally { srv.stop(); }
|
||||
});
|
||||
});
|
||||
|
||||
// --------------------------------------------------------------------------
|
||||
// Rate limit
|
||||
// --------------------------------------------------------------------------
|
||||
|
||||
describe('http-transport: rate limit', () => {
|
||||
const TOK = 'rl-tok';
|
||||
|
||||
test('15. token bucket: refill mechanic over time', async () => {
|
||||
freezeClock(1000);
|
||||
const srv = await startTest({
|
||||
validTokens: new Map([[hash(TOK), { id: 'rl-id', name: 'rl' }]]),
|
||||
tokenLimit: 2,
|
||||
ipLimit: 100,
|
||||
});
|
||||
try {
|
||||
// Use up 2 tokens
|
||||
const ok1 = await fetch(`${srv.url}/mcp`, { method: 'POST', headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' }, body: rpc('tools/list') });
|
||||
expect(ok1.status).toBe(200);
|
||||
const ok2 = await fetch(`${srv.url}/mcp`, { method: 'POST', headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' }, body: rpc('tools/list') });
|
||||
expect(ok2.status).toBe(200);
|
||||
|
||||
// Third should 429
|
||||
const blocked = await fetch(`${srv.url}/mcp`, { method: 'POST', headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' }, body: rpc('tools/list') });
|
||||
expect(blocked.status).toBe(429);
|
||||
|
||||
// Advance past the refill window (60s for 2 limit = 30s/token; advance 35s)
|
||||
advanceClock(35_000);
|
||||
const refilled = await fetch(`${srv.url}/mcp`, { method: 'POST', headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' }, body: rpc('tools/list') });
|
||||
expect(refilled.status).toBe(200);
|
||||
} finally { srv.stop(); freezeClock(0); }
|
||||
});
|
||||
|
||||
test('16. token bucket exhausted → 429 + Retry-After header', async () => {
|
||||
freezeClock(1000);
|
||||
const srv = await startTest({
|
||||
validTokens: new Map([[hash(TOK), { id: 'rl16', name: 'rl' }]]),
|
||||
tokenLimit: 1,
|
||||
ipLimit: 100,
|
||||
});
|
||||
try {
|
||||
await fetch(`${srv.url}/mcp`, { method: 'POST', headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' }, body: rpc('tools/list') });
|
||||
const r = await fetch(`${srv.url}/mcp`, { method: 'POST', headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' }, body: rpc('tools/list') });
|
||||
expect(r.status).toBe(429);
|
||||
expect(r.headers.get('retry-after')).not.toBeNull();
|
||||
expect(parseInt(r.headers.get('retry-after')!, 10)).toBeGreaterThan(0);
|
||||
} finally { srv.stop(); freezeClock(0); }
|
||||
});
|
||||
|
||||
test('17. LRU eviction at cap (insert > cap evicts LRU)', () => {
|
||||
let now = 0;
|
||||
const lim = new RateLimiter({ limit: 10, windowMs: 60_000, lruCap: 3 }, () => now);
|
||||
lim.check('a'); now += 1;
|
||||
lim.check('b'); now += 1;
|
||||
lim.check('c'); now += 1;
|
||||
expect(lim.size).toBe(3);
|
||||
lim.check('d'); now += 1;
|
||||
expect(lim.size).toBe(3);
|
||||
// 'a' should have been evicted (oldest by insertion). After re-checking 'a' it's a fresh bucket again.
|
||||
// Easiest verification: hammer 'a' should NOT be already exhausted — fresh bucket starts at limit.
|
||||
for (let i = 0; i < 10; i++) {
|
||||
const r = lim.check('a');
|
||||
expect(r.allowed).toBe(true);
|
||||
}
|
||||
// 11th should fail (no refill since clock barely moved)
|
||||
expect(lim.check('a').allowed).toBe(false);
|
||||
});
|
||||
|
||||
test('18. TTL prune (entries older than 2× window evicted)', () => {
|
||||
let now = 1000;
|
||||
const lim = new RateLimiter({ limit: 10, windowMs: 1000, lruCap: 100 }, () => now);
|
||||
lim.check('stale'); // touched at t=1000
|
||||
expect(lim.size).toBe(1);
|
||||
now = 1000 + 2001; // advance past 2× window
|
||||
lim.check('fresh'); // triggers prune
|
||||
expect(lim.size).toBe(1); // 'stale' evicted, only 'fresh' remains
|
||||
});
|
||||
|
||||
test('19. pre-auth IP bucket fires BEFORE auth (DB not called when IP exhausted)', async () => {
|
||||
freezeClock(1000);
|
||||
const srv = await startTest({
|
||||
ipLimit: 1,
|
||||
tokenLimit: 100,
|
||||
validTokens: new Map([[hash(TOK), { id: 'rl19', name: 'rl' }]]),
|
||||
});
|
||||
try {
|
||||
// First request consumes IP token (will hit auth and succeed)
|
||||
const r1 = await fetch(`${srv.url}/mcp`, { method: 'POST', headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' }, body: rpc('tools/list') });
|
||||
expect(r1.status).toBe(200);
|
||||
// Second: IP bucket exhausted. We send WITHOUT auth header. Should be 429 (IP-limited),
|
||||
// not 401 (auth-failed) — proving IP check happened first.
|
||||
const r2 = await fetch(`${srv.url}/mcp`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: rpc('tools/list') });
|
||||
expect(r2.status).toBe(429);
|
||||
} finally { srv.stop(); freezeClock(0); }
|
||||
});
|
||||
|
||||
test('20. /health bypasses rate limit', async () => {
|
||||
freezeClock(1000);
|
||||
const srv = await startTest({ ipLimit: 1, tokenLimit: 1 });
|
||||
try {
|
||||
// Hammer health 5 times — none should 429
|
||||
for (let i = 0; i < 5; i++) {
|
||||
const r = await fetch(`${srv.url}/health`);
|
||||
expect(r.status).toBe(200);
|
||||
}
|
||||
} finally { srv.stop(); freezeClock(0); }
|
||||
});
|
||||
});
|
||||
|
||||
// --------------------------------------------------------------------------
|
||||
// mcp_request_log audit
|
||||
// --------------------------------------------------------------------------
|
||||
|
||||
describe('http-transport: mcp_request_log audit', () => {
|
||||
test('21. successful request → audit row with token_name + operation + status', async () => {
|
||||
const TOK = 'audit-tok';
|
||||
const srv = await startTest({ validTokens: new Map([[hash(TOK), { id: 'a-1', name: 'audit-test' }]]) });
|
||||
try {
|
||||
await fetch(`${srv.url}/mcp`, { method: 'POST', headers: { 'Authorization': `Bearer ${TOK}`, 'Content-Type': 'application/json' }, body: rpc('tools/list') });
|
||||
// Audit insert is fire-and-forget; give it a tick to land in the fake handler
|
||||
await new Promise(r => setTimeout(r, 10));
|
||||
expect(srv.engine.audit.length).toBeGreaterThanOrEqual(1);
|
||||
const row = srv.engine.audit[srv.engine.audit.length - 1];
|
||||
expect(row.token_name).toBe('audit-test');
|
||||
expect(row.operation).toBe('tools/list');
|
||||
expect(row.status).toBe('success');
|
||||
expect(row.latency_ms).toBeGreaterThanOrEqual(0);
|
||||
} finally { srv.stop(); }
|
||||
});
|
||||
|
||||
test('22. failed auth → audit row with null token_name + auth_failed status', async () => {
|
||||
const srv = await startTest({});
|
||||
try {
|
||||
await fetch(`${srv.url}/mcp`, { method: 'POST', headers: { 'Authorization': 'Bearer wrong', 'Content-Type': 'application/json' }, body: rpc('tools/list') });
|
||||
await new Promise(r => setTimeout(r, 10));
|
||||
expect(srv.engine.audit.length).toBeGreaterThanOrEqual(1);
|
||||
const row = srv.engine.audit[srv.engine.audit.length - 1];
|
||||
expect(row.token_name).toBeNull();
|
||||
expect(row.status).toBe('auth_failed');
|
||||
} finally { srv.stop(); }
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user