Files
Garry TanandClaude Opus 5 4e8ab6db36 v0.42.69.0 feat(ai): default reranker to cohere:rerank-v3.5 before the ZeroEntropy sunset
ZeroEntropy's hosted API (including /v1/models/rerank) stops serving
2026-09-04. gbrain's default reranker pointed at it.

The load-bearing fix is the pre-gate, not the provider swap: there was no
isAvailable('reranker') check before applyReranker, while the embedding arm
has had one for releases. `balanced` is DEFAULT_SEARCH_MODE with
reranker_enabled: true, so post-sunset (and today, for anyone with no
reranker key at all) EVERY query on the default mode issued a doomed HTTP
request and burned the full 5s reranker_timeout_ms before failing open,
silently. hybrid.ts now probes the resolved per-call reranker model the same
way it probes the resolved embedding column's provider (#3421).

Provider choice: Cohere created the commercial rerank-API category and has
shipped v2 -> v3 -> v3.5 -> v4 while KEEPING v3.5 served after v4 launched,
which is the non-abandonment record a default-path provider needs. It is also
the most widely integrated reranker across RAG frameworks and is the target
ZeroEntropy's own migration guide names. Deliberately NOT routed via
OpenRouter (which would have needed zero code): putting the DEFAULT retrieval
path behind a startup proxy adds a single point of failure. OpenRouter
reranking stays the opt-in it already is.

No adapter work was needed. gbrain's so-called "ZeroEntropy wire shape" IS
the industry-standard Cohere dialect (`top_n` request,
`results[{index, relevance_score}]` response) — verified against
docs.cohere.com/reference/rerank and already asserted in the dashscope-rerank
recipe's own comments. Voyage (`top_k` / `data[]`) is the sole outlier, so
issue #3439 does not block this.

Changes:
- New rerank-only `cohere` recipe (api.cohere.com/v2/rerank; rerank-v3.5 +
  rerank-v4.0-fast + rerank-v4.0-pro), registered in recipes/index.ts.
- Availability pre-gate before applyReranker; `rerankerFn` test seam exempt.
- DEFAULT_RERANKER_MODEL + all three mode bundles + the retrieval-upgrade
  planner (which was persisting the soon-dead model string into user config
  at two sites) now point at cohere:rerank-v3.5.
- cohere_api_key threaded file plane -> loadConfig env merge ->
  buildGatewayConfig -> recipe, closing the seam that silently broke ZE
  (v0.37) and Voyage (#2662) in daemon/launchd/MCP contexts.
- Cohere rerank models priced in embedding-pricing.ts (the table
  budget-tracker's rerank lookup falls back to) so `--max-cost` callers
  don't TX2 hard-fail on the default path. Conservative pseudo-rate; Cohere
  bills per search and does not publish a machine-readable per-search rate.
- OpenRouter reranker pseudo-rate corrected (0.001 -> 0.10 per 1M pseudo-
  tokens); its own comment's arithmetic was off by ~250x.
- doctor/models auth + fix hints no longer hardcode one provider's env var.
- Reranker provider doc now leads with self-hosting: vLLM recommended (its
  /rerank is documented as the Cohere Rerank API, so it works through this
  recipe by repointing base_url), plus a correctness sanity-check for
  llama.cpp rerank builds. ZeroEntropy provider doc gets a deprecation
  banner with migration steps.
- test/e2e/zeroentropy-live.test.ts skips on/after 2026-09-04 so a stale key
  can't turn someone's test run red for unrelated reasons.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 01:23:12 -07:00
..

Getting Data Into Your Brain

GBrain is the retrieval layer. But retrieval is only as good as what you put in. This directory covers how to get data flowing into your brain automatically.

How Data Flows In

Signal arrives (phone call, email, tweet, calendar event)
  ↓
Collector captures it (deterministic code, reliable)
  ↓
Agent analyzes it (LLM, judgment, entity detection)
  ↓
Brain pages created/updated (compiled truth + timeline)
  ↓
GBrain indexes it (chunking, embedding, search-ready)
  ↓
Next query is smarter (the compounding effect)

Available Integrations

Self-Installing Recipes

These are integration recipes your agent can set up for you. Run gbrain integrations to see what's available and their status.

Recipe Category Requires What It Does Setup Time
ngrok-tunnel Infra Fixed public URL for MCP + voice ($8/mo) 10 min
credential-gateway Infra Gmail + Calendar access (ClawVisor or Google OAuth) 15 min
voice-to-brain Sense ngrok-tunnel Phone calls create brain pages via Twilio + OpenAI Realtime 30 min
email-to-brain Sense credential-gateway Gmail messages flow into entity pages via deterministic collector 20 min
x-to-brain Sense Twitter timeline, mentions, keyword monitoring with deletion detection 15 min
calendar-to-brain Sense credential-gateway Google Calendar events become searchable daily brain pages 20 min
meeting-sync Sense Circleback meeting transcripts auto-import with attendee propagation 15 min

Manual Integration Guides

These require manual setup (no self-installing recipe yet):

Guide What It Does
Credential Gateway Set up ClawVisor or Hermes for Gmail, Calendar, Contacts access
Meeting & Call Webhooks Circleback meeting transcripts + Quo/OpenPhone SMS/calls

How to Read a Recipe

Integration recipes are markdown files with YAML frontmatter. Your agent reads the recipe and walks you through setup.

---
id: voice-to-brain              # unique identifier
name: Voice-to-Brain            # human-readable name
version: 0.7.0                  # recipe version
description: Phone calls...     # what it does
category: sense                 # sense (data input) or reflex (automated response)
requires: []                    # other recipes that must be set up first
secrets:                        # API keys and credentials needed
  - name: TWILIO_ACCOUNT_SID
    description: Twilio account SID
    where: https://console.twilio.com    # exact URL to get this key
health_checks:                  # typed DSL to verify the integration is working
  - type: http
    url: "https://api.twilio.com/2010-04-01/Accounts/$TWILIO_ACCOUNT_SID.json"
    auth: basic
    auth_user: "$TWILIO_ACCOUNT_SID"
    auth_token: "$TWILIO_AUTH_TOKEN"
    label: "Twilio account"
setup_time: 30 min              # estimated time to complete setup
---

[Setup instructions the agent follows step by step...]

The recipe IS the installer. Your agent (OpenClaw, Hermes, Claude Code) reads the markdown body and executes the setup steps. It asks you for API keys, validates each one, configures the integration, and runs a smoke test.

Recipe trust boundary

Only recipes shipped inside the gbrain package itself (the recipes/ directory in a source install, or the global install copy) are trusted. Recipes discovered at runtime from $GBRAIN_RECIPES_DIR or a cwd-local ./recipes/ are marked untrusted: they cannot run command health checks, cannot run http health checks (SSRF defense), and cannot use the deprecated string health_check form. Untrusted recipes can still use env_exists and any_of compositions. To ship a recipe that runs live checks, contribute it upstream so it becomes package-bundled.

The Deterministic Collector Pattern

When an LLM keeps failing at a mechanical task despite repeated prompt fixes, stop fighting the LLM. Move the mechanical work to code.

Code for data. LLMs for judgment.

  • Email collection: code pulls emails with baked-in links (100% reliable). LLM reads the digest, classifies, enriches brain entries (judgment).
  • Tweet collection: code pulls timeline, detects deletions, tracks engagement (deterministic). LLM extracts entities, writes brain updates (judgment).
  • Calendar sync: code pulls events and attendees (deterministic). LLM enriches attendee brain pages (judgment).

This pattern prevents the "LLM forgot the links" failure mode. Mechanical work must be 100% reliable. Judgment work is where LLMs shine.

See Deterministic Collectors for the full pattern.

Architecture

For details on the shared infrastructure that all integrations build on (import pipeline, chunking, embedding, search), see the Infrastructure Layer.

For the philosophy behind thin harness + fat skills, see Thin Harness, Fat Skills.