mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-15 01:12:20 +00:00
Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f83449be28 | ||
|
|
4e8ab6db36 |
@@ -2,6 +2,72 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.42.69.0] - 2026-07-28
|
||||
|
||||
**The default reranker moves to Cohere `rerank-v3.5` before ZeroEntropy's hosted API shuts down, and search stops calling a reranker it has no key for.**
|
||||
|
||||
ZeroEntropy's hosted endpoints stop serving on 2026-09-04. gbrain's default reranker pointed at one of them, so this release moves the default to Cohere `rerank-v3.5` and fixes the thing that made the sunset worse than it needed to be.
|
||||
|
||||
The important fix is the one that helps people who never configure a reranker at all: `balanced` is the default search mode and it turns the reranker on, so every query was issuing a request to the reranker provider whether or not a key existed — waiting out the full timeout each time and then quietly falling back to unranked order. Search now checks provider availability first, the same way it already did for embeddings. No key means no request and no wasted wait.
|
||||
|
||||
Cohere was picked for the default path because it has the longest proven record of keeping models served: it created the commercial rerank API, has shipped v2 → v3 → v3.5 → v4, and kept v3.5 serving after v4 launched. It is also the most widely integrated reranker across RAG frameworks, and ZeroEntropy's own migration guide names it as the target. Routing the default retrieval path through a proxy was deliberately rejected — OpenRouter reranking stays the opt-in it already was.
|
||||
|
||||
No code change was needed to talk to Cohere: the wire shape gbrain already spoke is the Cohere dialect.
|
||||
|
||||
Self-hosting is a first-class fallback. vLLM's rerank endpoint is Cohere-compatible, so pointing `provider_base_urls.cohere` at your own vLLM server is the entire setup. The reranker provider doc now leads with that, and carries a correctness check worth running against any llama.cpp build.
|
||||
|
||||
### Changed
|
||||
|
||||
- Default reranker is now `cohere:rerank-v3.5` across all three search-mode bundles, the gateway default, and the retrieval-upgrade planner (which previously wrote a soon-to-be-dead model string into user config).
|
||||
- New `cohere` reranker recipe (`rerank-v3.5`, `rerank-v4.0-fast`, `rerank-v4.0-pro`). Set `COHERE_API_KEY`, or `cohere_api_key` in `~/.gbrain/config.json` — the config-file route is threaded all the way into the gateway, so daemon, launchd, and MCP contexts work without a shell export.
|
||||
- `zeroentropyai:zerank-2` remains a supported override until its hosted API sunsets. The provider doc carries a deprecation banner with the migration steps; `gbrain upgrade` already warns affected brains.
|
||||
- Reranker provider doc now leads with the self-hosted options (vLLM recommended) and documents the correctness sanity-check for llama.cpp rerank builds.
|
||||
- OpenRouter reranker cost estimate corrected — the previous pseudo-rate under-estimated spend by orders of magnitude for budget-capped callers.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Search no longer issues a reranker request when no reranker provider is reachable. Previously every query on the default mode paid the full reranker timeout before failing over to unranked order, with nothing surfaced to the user. (#3421)
|
||||
- `gbrain doctor`'s reranker auth hint no longer names a single provider's environment variable regardless of which reranker is configured.
|
||||
- The ZeroEntropy live end-to-end test now skips on and after the sunset date, so a stale key in someone's shell can't turn their test run red for unrelated reasons.
|
||||
|
||||
## To take advantage of v0.42.69.0
|
||||
|
||||
`gbrain upgrade` handles this. No schema migration ships in this release.
|
||||
|
||||
1. **Upgrade:**
|
||||
```bash
|
||||
gbrain upgrade
|
||||
```
|
||||
|
||||
2. **If you want reranking (on by default in `balanced` and `tokenmax`), set a key:**
|
||||
```bash
|
||||
export COHERE_API_KEY=<your-key>
|
||||
gbrain models doctor
|
||||
```
|
||||
For daemon, launchd, and MCP contexts that don't inherit your shell, add
|
||||
`"cohere_api_key": "<your-key>"` to `~/.gbrain/config.json` instead — that
|
||||
route is threaded into the gateway. (`gbrain config set` writes the DB
|
||||
plane, which is not merged for API-key fields; that predates this release.)
|
||||
Skipping this is fine — search runs without the rerank arm rather than retrying a provider it cannot reach.
|
||||
|
||||
3. **If you previously overrode `search.reranker.model` to a ZeroEntropy model, move it:**
|
||||
```bash
|
||||
gbrain config get search.reranker.model
|
||||
gbrain config set search.reranker.model cohere:rerank-v3.5
|
||||
```
|
||||
|
||||
4. **If you embed with a ZeroEntropy model, that is the urgent one and it is not automatic.** Queries embed through the same endpoint that produced your stored vectors, so semantic retrieval stops on the sunset date. Preview a migration or self-host the weights:
|
||||
```bash
|
||||
gbrain migrate embeddings --to <provider:model> --dry-run
|
||||
```
|
||||
See `docs/ai-providers/zeroentropy.md`.
|
||||
|
||||
5. **Prefer no API spend?** vLLM serves the same dialect:
|
||||
```bash
|
||||
gbrain config set provider_base_urls.cohere http://your-host:8000/v1
|
||||
```
|
||||
Walkthrough in `docs/ai-providers/llama-server-reranker.md`.
|
||||
|
||||
## [0.42.66.1] - 2026-07-27
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -291,7 +291,7 @@ Data flowing into the brain. Each integration is a recipe — markdown + setup h
|
||||
- **Voice**: Phone calls create brain pages via Twilio + OpenAI Realtime (or DIY STT+LLM+TTS). Setup recipe: [`recipes/twilio-voice-brain.md`](recipes/twilio-voice-brain.md).
|
||||
- **Email + calendar**: webhook handlers that route to brain signals. [`docs/integrations/meeting-webhooks.md`](docs/integrations/meeting-webhooks.md).
|
||||
- **Embedding providers**: 16 recipes covering OpenAI (default fallback), OpenRouter, Voyage, ZeroEntropy (default), Google Gemini, Azure OpenAI, MiniMax, Alibaba DashScope, Zhipu, Ollama (local), llama.cpp llama-server (local), LiteLLM proxy. Pricing matrix + decision tree in [`docs/integrations/embedding-providers.md`](docs/integrations/embedding-providers.md).
|
||||
- **Rerankers**: ZeroEntropy `zerank-2` hosted (default in `tokenmax` mode) plus the v0.40.6.1 `llama-server-reranker` recipe for fully-local cross-encoder rerank via llama.cpp — runs Qwen3-Reranker or self-hosted ZeroEntropy weights against the same `gateway.rerank()` seam. Setup walkthrough in [`docs/ai-providers/llama-server-reranker.md`](docs/ai-providers/llama-server-reranker.md).
|
||||
- **Rerankers**: Cohere `rerank-v3.5` hosted (the default; on in `balanced` and `tokenmax` modes — set `COHERE_API_KEY`, and if you don't, search just skips the rerank arm) plus self-hosted options for zero API spend: vLLM's Cohere-compatible `/rerank` works through the same recipe by repointing `base_url`, and the `llama-server-reranker` recipe runs Qwen3-Reranker or self-hosted ZeroEntropy weights via llama.cpp against the same `gateway.rerank()` seam. Setup walkthrough in [`docs/ai-providers/llama-server-reranker.md`](docs/ai-providers/llama-server-reranker.md). ZeroEntropy `zerank-2` still works as an override until its hosted API sunsets 2026-09-04.
|
||||
- **Credential gateway**: vault-aware secret distribution. [`docs/integrations/credential-gateway.md`](docs/integrations/credential-gateway.md).
|
||||
- **MCP clients**: every major MCP client is supported. [`docs/mcp/`](docs/mcp/) per-client setup.
|
||||
|
||||
|
||||
@@ -1,10 +1,95 @@
|
||||
# llama-server reranker (local) — Qwen3-Reranker, self-hosted ZE, any ZE-wire-shape provider
|
||||
# Self-hosted rerankers — vLLM (recommended), llama-server, any Cohere-dialect endpoint
|
||||
|
||||
## Start here: which reranker should you run?
|
||||
|
||||
gbrain's default reranker is **`cohere:rerank-v3.5`** (hosted, v0.42.69.0+).
|
||||
Set `COHERE_API_KEY` and you are done — no config needed:
|
||||
|
||||
```bash
|
||||
export COHERE_API_KEY=<your-key>
|
||||
gbrain models doctor # expect: ✔ reranker_config cohere:rerank-v3.5 ok
|
||||
```
|
||||
|
||||
For daemon, launchd, and MCP contexts that don't inherit your shell, put the
|
||||
key in `~/.gbrain/config.json` instead — it is threaded all the way into the
|
||||
gateway:
|
||||
|
||||
```json
|
||||
{ "cohere_api_key": "<your-key>" }
|
||||
```
|
||||
|
||||
`gbrain config set cohere_api_key …` writes the DB plane, which `loadConfig()`
|
||||
deliberately does **not** merge for any `*_api_key` field (same long-standing
|
||||
gap as `zeroentropy_api_key` / `voyage_api_key`). Use the env var or the
|
||||
config file.
|
||||
|
||||
If you have no key, nothing breaks: search skips the reranker arm entirely and
|
||||
returns RRF order (#3421). It does **not** issue a doomed request per query.
|
||||
|
||||
If you want rerank with **no API spend or no egress**, self-host. The rest of
|
||||
this page is that path.
|
||||
|
||||
| Option | Endpoint | Verdict |
|
||||
|---|---|---|
|
||||
| **vLLM** `--task score` | `/rerank`, `/v1/rerank`, `/v2/rerank` | **Recommended.** vLLM's docs label it the "Cohere Rerank API", which is the dialect `gateway.rerank()` already speaks — so it works through the existing `cohere` recipe by repointing `base_url`. No new recipe, no adapter. |
|
||||
| **llama.cpp** `llama-server --reranking` | `/v1/rerank` | Works, but read the scoring warning below before you trust it. Uses the `llama-server-reranker` recipe. |
|
||||
|
||||
### vLLM (recommended) — reuse the cohere recipe
|
||||
|
||||
Because vLLM serves the Cohere dialect, you do not need a gbrain code change:
|
||||
|
||||
```bash
|
||||
vllm serve BAAI/bge-reranker-v2-m3 --task score --port 8000
|
||||
|
||||
# Point the cohere recipe at your own server. /v1 and /v2 both work on vLLM.
|
||||
gbrain config set provider_base_urls.cohere http://your-host:8000/v1
|
||||
gbrain config set search.reranker.model cohere:rerank-v3.5 # already the default
|
||||
export COHERE_API_KEY=not-used-but-the-recipe-requires-one
|
||||
```
|
||||
|
||||
The model string after the colon must be one of the recipe's allowlisted ids;
|
||||
vLLM ignores the `model` field when it serves a single model, so leaving the
|
||||
default is fine. If you need a different id in the request body, use the
|
||||
`llama-server-reranker` recipe instead — its allowlist is open.
|
||||
|
||||
### ⚠️ llama.cpp scoring correctness — verify your build
|
||||
|
||||
llama.cpp [issue #16407](https://github.com/ggml-org/llama.cpp/issues/16407)
|
||||
(opened 2025-10-03, **closed as completed 2025-10-09**) reported
|
||||
`llama-server --reranking` returning near-zero, uncorrelated scores — on the
|
||||
order of `1e-28` — for Qwen3-Reranker (0.6B/4B/8B), and wrong or
|
||||
non-matching scores for BGE, mxbai, and Jina rerankers. It is *fixed
|
||||
upstream*, but the failure mode is silent: gbrain's fail-open contract cannot
|
||||
detect "the server answered 200 with garbage numbers", and neither can
|
||||
`gbrain models doctor`. So:
|
||||
|
||||
- **Build from a commit after 2025-10-09.** Anything older can silently
|
||||
destroy your ranking quality while looking perfectly healthy.
|
||||
- **Sanity-check the scores by hand** after any llama.cpp upgrade:
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:8081/v1/rerank -H 'Content-Type: application/json' \
|
||||
-d '{"model":"m","query":"how do I reset my password?",
|
||||
"documents":["Password reset instructions","Cheese is a dairy product"]}' \
|
||||
| jq '.results'
|
||||
```
|
||||
|
||||
The password document must score clearly higher, and scores must not be
|
||||
~`1e-28`. If they are, your build predates the fix (or the GGUF conversion
|
||||
lost the pooling/rank metadata).
|
||||
- Prefer vLLM if you cannot pin and verify a llama.cpp build.
|
||||
|
||||
---
|
||||
|
||||
## llama-server (local) — Qwen3-Reranker, self-hosted ZE, any Cohere-dialect provider
|
||||
|
||||
[`llama-server`](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md)
|
||||
is the HTTP wrapper that ships with llama.cpp. With `--reranking`, it
|
||||
exposes an OpenAI-style `POST /v1/rerank` endpoint that returns
|
||||
`{results: [{index, relevance_score}]}` — exactly the wire shape gbrain
|
||||
already drives for ZeroEntropy's hosted reranker. The
|
||||
exposes a `POST /v1/rerank` endpoint that returns
|
||||
`{results: [{index, relevance_score}]}` — the Cohere rerank dialect, which is
|
||||
the single wire shape `gateway.rerank()` drives for every provider it supports
|
||||
(Cohere, ZeroEntropy, DashScope, OpenRouter). Voyage (`top_k` / `data[]`) is
|
||||
the sole outlier. The
|
||||
`llama-server-reranker` recipe (added in v0.40.6.1) routes
|
||||
`gateway.rerank()` at your local llama.cpp instance instead of ZE.
|
||||
|
||||
@@ -13,18 +98,21 @@ Two flavors of "local" this recipe covers:
|
||||
- **Qwen3-Reranker** (0.6B / 4B / 8B) — open-weight cross-encoder; pull
|
||||
the GGUF from HuggingFace and serve.
|
||||
- **Self-hosted ZeroEntropy** (`zerank-2`, `zerank-1-small`) — the
|
||||
weights are on HuggingFace too. GGUF-convert them and serve them the
|
||||
same way. **Quality is not guaranteed to match ZE-hosted:** GGUF
|
||||
weights are on HuggingFace too and are **Apache-2.0** upstream (verified
|
||||
2026-07-28; they were previously CC-BY-NC-4.0 and third-party model pages
|
||||
still say non-commercial — check the upstream repo, not the mirrors). Self-hosting is
|
||||
the way to keep using them after ZE's hosted API sunsets 2026-09-04 —
|
||||
GGUF-convert them and serve them the same way. **Quality is not guaranteed to match ZE-hosted:** GGUF
|
||||
conversion + quantization + pooling/rank metadata + tokenizer special
|
||||
tokens all affect scores. If you self-host ZE for production
|
||||
retrieval, pin your own brain-relevant eval (
|
||||
[docs/eval-bench.md](../eval-bench.md)) as a regression guard.
|
||||
|
||||
This recipe is the path override + recipe shape. Any provider whose
|
||||
request/response wire matches ZE/llama.cpp can use it by just pointing
|
||||
at a different base URL. Providers whose wire shape differs (Voyage uses
|
||||
`top_k` not `top_n`, returns `data[]` not `results[]`) need a separate
|
||||
recipe with adapter hooks — that lands in a follow-up plan.
|
||||
This recipe is the path override + recipe shape. Any provider serving the
|
||||
Cohere dialect can use it (or the `cohere` recipe) by just pointing at a
|
||||
different base URL. Providers whose wire shape differs (Voyage uses `top_k`
|
||||
not `top_n`, returns `data[]` not `results[]`) need a separate recipe with
|
||||
adapter hooks — that lands in a follow-up plan.
|
||||
|
||||
## Setup
|
||||
|
||||
@@ -159,3 +247,24 @@ returns the original RRF order unchanged. Search reliability beats
|
||||
reranker quality. If your llama.cpp host goes down, your searches keep
|
||||
working — they just stop ranking against the cross-encoder until you
|
||||
restart the server.
|
||||
|
||||
## Cohere trial-tier caveat (live-verified 2026-07-28)
|
||||
|
||||
`rerank-v4.0-pro` is listed in the Cohere recipe but was **unreachable on a
|
||||
fresh trial key**. Back-to-back calls with the same key, same second:
|
||||
|
||||
| model | result |
|
||||
|---|---|
|
||||
| `rerank-v3.5` | HTTP 200 in 0.21s |
|
||||
| `rerank-v4.0-fast` | HTTP 200 in 0.20s |
|
||||
| `rerank-v4.0-pro` | no response at all, 20s+ |
|
||||
|
||||
Its siblings answered instantly, so this is model-specific, not account-wide
|
||||
throttling. On a trial key, selecting `-pro` means every query waits out the
|
||||
recipe's 5s timeout and then fail-opens silently, returning un-reranked order
|
||||
with no error surfaced. Use `rerank-v3.5` (the default) or `rerank-v4.0-fast`
|
||||
on trial keys; `-pro` is for paid keys.
|
||||
|
||||
Also worth knowing: the trial tier throttles bursts. Un-throttled latency is
|
||||
~0.2s, but a throttled call can exceed the 5s timeout and hit the same silent
|
||||
fail-open path.
|
||||
|
||||
@@ -1,5 +1,39 @@
|
||||
# ZeroEntropy — zembed-1 + zerank-2
|
||||
|
||||
> ## ⚠️ DEPRECATED — hosted API shuts down 2026-09-04
|
||||
>
|
||||
> ZeroEntropy announced (2026-07-24) that its hosted endpoints, including
|
||||
> `/v1/models/embed` and `/v1/models/rerank`, stop serving on **2026-09-04**
|
||||
> (#3390). This page is kept for installs that still point at ZE and for
|
||||
> anyone self-hosting the weights.
|
||||
>
|
||||
> **What changed in gbrain v0.42.69.0:** the default reranker is now
|
||||
> `cohere:rerank-v3.5` (see
|
||||
> [`llama-server-reranker.md`](llama-server-reranker.md) for the self-host
|
||||
> fallback). `zerank-2` is still a valid `search.reranker.model` value — it
|
||||
> just stops working on the sunset date.
|
||||
>
|
||||
> **If you are on ZE today:**
|
||||
> - **Reranker** — do nothing. New installs and anyone who never overrode
|
||||
> `search.reranker.model` are already on Cohere. If you *did* override it,
|
||||
> run `gbrain config set search.reranker.model cohere:rerank-v3.5` and set
|
||||
> `COHERE_API_KEY`. Missing key is a non-event: search skips the reranker
|
||||
> arm entirely and returns RRF order (#3421).
|
||||
> - **Embedding** — this one is urgent and NOT automatic. Queries embed
|
||||
> through the same endpoint that produced your stored vectors, so on the
|
||||
> sunset date semantic retrieval stops working entirely. Either migrate
|
||||
> (`gbrain migrate embeddings --to <provider:model> --dry-run` first) or
|
||||
> self-host `zembed-1`, whose weights are Apache-2.0 — self-hosting
|
||||
> preserves your existing vectors, migrating re-embeds them.
|
||||
> - `gbrain upgrade` prints this as a one-shot banner for affected brains.
|
||||
>
|
||||
> **Licensing note:** `zerank-1` / `zerank-2` weights are **Apache-2.0**
|
||||
> upstream (`huggingface.co/zeroentropy/zerank-2`, verified 2026-07-28). They
|
||||
> were previously CC-BY-NC-4.0 and the relicense is recent, so third-party
|
||||
> model pages and aggregator listings still say "non-commercial" — check the
|
||||
> upstream HuggingFace repo, not the mirrors, before ruling self-hosting out.
|
||||
|
||||
|
||||
[ZeroEntropy](https://zeroentropy.dev) ships two specialized small models
|
||||
for retrieval pipelines:
|
||||
|
||||
@@ -82,6 +116,10 @@ enforcement in hybrid search.
|
||||
|
||||
### Default-on with `tokenmax` mode
|
||||
|
||||
> Since v0.42.69.0 the mode bundles default to `cohere:rerank-v3.5`, not
|
||||
> `zeroentropyai:zerank-2`. The prose below describes the enable/disable
|
||||
> mechanics, which are unchanged; substitute your chosen model string.
|
||||
|
||||
`tokenmax` mode now defaults `search.reranker.enabled = true` with
|
||||
`zerank-2`. If you already use `tokenmax` AND have `ZEROENTROPY_API_KEY`
|
||||
set, reranker fires automatically. Without the key, every rerank call
|
||||
|
||||
@@ -63,8 +63,8 @@ The doctor distinguishes two repair paths:
|
||||
- **Cost-sensitive, English-only**: Ollama (free, local) or Voyage (paid, best quality per dollar).
|
||||
- **Quality-first**: Voyage `voyage-4-large` (1024-2048 dims, ~3-4× more dense tokens than OpenAI tiktoken).
|
||||
- **Code-heavy brain (gstack per-worktree, source repos)**: Voyage `voyage-code-3` (1024 default; supports 256/512/1024/2048). Tuned on programming languages. Voyage publishes head-to-head numbers showing it outperforms their general flagships on code retrieval ([voyageai.com/blog](https://voyageai.com/blog)). For gstack's per-worktree pglite-backed code brain, this is the right default — see Topology 3 in `docs/architecture/topologies.md`.
|
||||
- **Reranking pair**: ZeroEntropy `zerank-2` is the hosted default in `tokenmax` mode (see [`docs/ai-providers/zeroentropy.md`](../ai-providers/zeroentropy.md)). Voyage `rerank-2.5` pairs cleanly with Voyage embeddings.
|
||||
- **Local reranking (no API spend)**: `llama-server-reranker` recipe (v0.40.6.1) — point gbrain at your own `llama-server --reranking` instance running Qwen3-Reranker or self-hosted ZeroEntropy weights. Same `gateway.rerank()` seam, $0 per call. Walkthrough in [`docs/ai-providers/llama-server-reranker.md`](../ai-providers/llama-server-reranker.md).
|
||||
- **Reranking pair**: Cohere `rerank-v3.5` is the hosted default (on in `balanced` + `tokenmax`; needs `COHERE_API_KEY`, and search silently skips the rerank arm without one). ZeroEntropy `zerank-2` is override-only and its hosted API sunsets 2026-09-04 (see [`docs/ai-providers/zeroentropy.md`](../ai-providers/zeroentropy.md)). Voyage `rerank-2.5` pairs cleanly with Voyage embeddings.
|
||||
- **Local reranking (no API spend)**: vLLM (recommended — its `/rerank` is Cohere-compatible, so repointing `provider_base_urls.cohere` is the whole setup) or the `llama-server-reranker` recipe (v0.40.6.1) against your own `llama-server --reranking` instance. Same `gateway.rerank()` seam, $0 per call. Walkthrough — including a correctness check you should run on any llama.cpp build — in [`docs/ai-providers/llama-server-reranker.md`](../ai-providers/llama-server-reranker.md).
|
||||
- **One key for many hosted models**: OpenRouter. Set `OPENROUTER_API_KEY` and use `openrouter:<provider>/<model>` for chat against GPT-5.2, Claude 4.x, Gemini 3, DeepSeek, and dozens more without juggling per-provider keys. Embedding catalog includes OpenAI, Google, Qwen, BGE-M3.
|
||||
- **Enterprise compliance**: Azure OpenAI (data residency + private endpoints) or self-hosted via llama-server / Ollama.
|
||||
- **China region**: DashScope (Alibaba) or Zhipu (BigModel). DashScope's international endpoint at `dashscope-intl.aliyuncs.com`; override `provider_base_urls.dashscope` for the China endpoint.
|
||||
|
||||
+1
-1
@@ -1785,7 +1785,7 @@ Data flowing into the brain. Each integration is a recipe — markdown + setup h
|
||||
- **Voice**: Phone calls create brain pages via Twilio + OpenAI Realtime (or DIY STT+LLM+TTS). Setup recipe: [`recipes/twilio-voice-brain.md`](recipes/twilio-voice-brain.md).
|
||||
- **Email + calendar**: webhook handlers that route to brain signals. [`docs/integrations/meeting-webhooks.md`](docs/integrations/meeting-webhooks.md).
|
||||
- **Embedding providers**: 16 recipes covering OpenAI (default fallback), OpenRouter, Voyage, ZeroEntropy (default), Google Gemini, Azure OpenAI, MiniMax, Alibaba DashScope, Zhipu, Ollama (local), llama.cpp llama-server (local), LiteLLM proxy. Pricing matrix + decision tree in [`docs/integrations/embedding-providers.md`](docs/integrations/embedding-providers.md).
|
||||
- **Rerankers**: ZeroEntropy `zerank-2` hosted (default in `tokenmax` mode) plus the v0.40.6.1 `llama-server-reranker` recipe for fully-local cross-encoder rerank via llama.cpp — runs Qwen3-Reranker or self-hosted ZeroEntropy weights against the same `gateway.rerank()` seam. Setup walkthrough in [`docs/ai-providers/llama-server-reranker.md`](docs/ai-providers/llama-server-reranker.md).
|
||||
- **Rerankers**: Cohere `rerank-v3.5` hosted (the default; on in `balanced` and `tokenmax` modes — set `COHERE_API_KEY`, and if you don't, search just skips the rerank arm) plus self-hosted options for zero API spend: vLLM's Cohere-compatible `/rerank` works through the same recipe by repointing `base_url`, and the `llama-server-reranker` recipe runs Qwen3-Reranker or self-hosted ZeroEntropy weights via llama.cpp against the same `gateway.rerank()` seam. Setup walkthrough in [`docs/ai-providers/llama-server-reranker.md`](docs/ai-providers/llama-server-reranker.md). ZeroEntropy `zerank-2` still works as an override until its hosted API sunsets 2026-09-04.
|
||||
- **Credential gateway**: vault-aware secret distribution. [`docs/integrations/credential-gateway.md`](docs/integrations/credential-gateway.md).
|
||||
- **MCP clients**: every major MCP client is supported. [`docs/mcp/`](docs/mcp/) per-client setup.
|
||||
|
||||
|
||||
+1
-1
@@ -146,7 +146,7 @@
|
||||
"bun": ">=1.3.10"
|
||||
},
|
||||
"license": "MIT",
|
||||
"version": "0.42.66.1",
|
||||
"version": "0.42.69.0",
|
||||
"overrides": {
|
||||
"@hono/node-server": "^2.0.5",
|
||||
"fast-uri": "^3.1.4",
|
||||
|
||||
@@ -1607,7 +1607,7 @@ export async function checkRerankerHealth(engine: BrainEngine): Promise<Check> {
|
||||
return {
|
||||
name: 'reranker_health',
|
||||
status: 'warn',
|
||||
message: `${authFails.length} reranker auth failure(s) in last 7 days. Fix: verify ZEROENTROPY_API_KEY and run \`gbrain models doctor\`.`,
|
||||
message: `${authFails.length} reranker auth failure(s) in last 7 days. Fix: verify the API key for your configured \`search.reranker.model\` provider (default Cohere: COHERE_API_KEY) and run \`gbrain models doctor\`.`,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1627,7 +1627,7 @@ export async function checkRerankerHealth(engine: BrainEngine): Promise<Check> {
|
||||
return {
|
||||
name: 'reranker_health',
|
||||
status: 'warn',
|
||||
message: `${transientFails.length} transient reranker failure(s) in last 7 days. Search fails open to RRF order; check ZE status if persistent.`,
|
||||
message: `${transientFails.length} transient reranker failure(s) in last 7 days. Search fails open to RRF order; check your reranker provider's status if persistent.`,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1638,9 +1638,9 @@ export async function checkRerankerHealth(engine: BrainEngine): Promise<Check> {
|
||||
if (unknownFails.length >= 3) {
|
||||
const setupHint = unknownFails.some((f) => {
|
||||
const summary = String(f.error_summary ?? '');
|
||||
return summary.includes('ZEROENTROPY_API_KEY') || summary.toLowerCase().includes('api key');
|
||||
return summary.includes('API_KEY') || summary.toLowerCase().includes('api key');
|
||||
})
|
||||
? ' Fix: verify ZEROENTROPY_API_KEY and run `gbrain models doctor`.'
|
||||
? ' Fix: verify the API key for your configured `search.reranker.model` provider (default Cohere: COHERE_API_KEY) and run `gbrain models doctor`.'
|
||||
: '';
|
||||
return {
|
||||
name: 'reranker_health',
|
||||
|
||||
@@ -325,7 +325,7 @@ export async function resolveLiveRerankerTimeoutMs(engine: BrainEngine): Promise
|
||||
*
|
||||
* CDX2-F11: `assertTouchpoint()` does NOT enforce allowlists for
|
||||
* openai-compatible recipes — the probe does it directly here. Without
|
||||
* this, `search.reranker.model=zeroentropyai:made-up-name` would silently
|
||||
* this, `search.reranker.model=cohere:made-up-name` would silently
|
||||
* pass config probes and fail at first rerank call.
|
||||
*
|
||||
* v0.40.6.1: resolves via `resolveLiveRerankerModel(engine)` so probe and
|
||||
@@ -363,7 +363,7 @@ async function probeRerankerConfig(engine: BrainEngine): Promise<ProbeResult> {
|
||||
touchpoint: 'reranker_config',
|
||||
status: 'config',
|
||||
message: `Provider "${recipe.id}" does not declare a reranker touchpoint.`,
|
||||
fix: 'Switch to a provider that does (e.g. zeroentropyai:zerank-2).',
|
||||
fix: 'Switch to a provider that does (e.g. cohere:rerank-v3.5, the default).',
|
||||
elapsed_ms: Date.now() - start,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -48,7 +48,7 @@ const KNOB_DESCRIPTIONS: Record<keyof ModeBundle, string> = {
|
||||
tokenBudget: 'Per-call token-budget cap (undefined = no cap)',
|
||||
expansion: 'LLM multi-query expansion (Haiku call per search)',
|
||||
searchLimit: 'Default `limit` for the operation layer',
|
||||
reranker_enabled: 'Cross-encoder reranker (ZE zerank-2) on/off',
|
||||
reranker_enabled: 'Cross-encoder reranker (Cohere rerank-v3.5) on/off',
|
||||
reranker_model: 'Provider:model for the reranker',
|
||||
reranker_top_n_in: 'Candidates sent to reranker per call',
|
||||
reranker_top_n_out: 'Cap on reranked output (null = no truncate)',
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
* import it from `../../src/cli.ts`.
|
||||
*
|
||||
* The single ownership site for: (a) folding file-plane API keys
|
||||
* (openai/anthropic/zeroentropy/openrouter/voyage) into the gateway env, and (b) threading
|
||||
* (openai/anthropic/zeroentropy/openrouter/voyage/cohere) into the gateway env, and (b) threading
|
||||
* local-server `*_BASE_URL` env vars into base_urls. Both matter for the
|
||||
* init-time embedding-key probe — without (a) it would false-warn on
|
||||
* config.json-keyed users, and without (b) a live probe could hit the wrong
|
||||
@@ -44,6 +44,12 @@ export function buildGatewayConfig(c: GBrainConfig): AIGatewayConfig {
|
||||
// multimodal/image embeds despite config.json looking complete. process.env
|
||||
// still wins via the later spread.
|
||||
if (c.voyage_api_key) envFromConfig.VOYAGE_API_KEY = c.voyage_api_key;
|
||||
// v0.42.69.0: same seam for Cohere, which is now the DEFAULT reranker. The
|
||||
// two comments above document this identical bug shipping twice (ZE, Voyage)
|
||||
// — a key accepted at the file plane that never reaches the gateway env
|
||||
// fails silently in daemon/launchd/MCP contexts. process.env still wins via
|
||||
// the later spread.
|
||||
if (c.cohere_api_key) envFromConfig.COHERE_API_KEY = c.cohere_api_key;
|
||||
// Azure OpenAI (keyless/Entra): fold the non-secret endpoint/deployment + the
|
||||
// Entra opt-in into the gateway env so the azure-openai recipe works in any
|
||||
// shell (incl. non-interactive agent shells). The bearer token is minted at
|
||||
|
||||
@@ -119,7 +119,7 @@ const DEFAULT_CHAT_MODEL = 'anthropic:claude-sonnet-4-6';
|
||||
// v0.35.0.0+: reranker default. Used only when search.reranker.enabled is set
|
||||
// AND no explicit reranker_model is configured. Mode bundles' per-mode
|
||||
// `reranker_model` default to this same value but can be overridden.
|
||||
const DEFAULT_RERANKER_MODEL = 'zeroentropyai:zerank-2';
|
||||
const DEFAULT_RERANKER_MODEL = 'cohere:rerank-v3.5';
|
||||
|
||||
let _config: AIGatewayConfig | null = null;
|
||||
const _modelCache = new Map<string, any>();
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
import type { Recipe } from '../types.ts';
|
||||
|
||||
/**
|
||||
* Cohere — reranker only (same rerank-only recipe topology as
|
||||
* `dashscope-rerank` and `llama-server-reranker`; Cohere's embed/chat
|
||||
* surfaces are deliberately out of scope).
|
||||
*
|
||||
* This is gbrain's DEFAULT reranker as of v0.42.69.0, replacing
|
||||
* `zeroentropyai:zerank-2` ahead of ZeroEntropy's 2026-09-04 hosted-API
|
||||
* shutdown (#3390).
|
||||
*
|
||||
* Why Cohere for the default retrieval path:
|
||||
* - It created the commercial rerank-API category and has shipped
|
||||
* v2 → v3 → v3.5 → v4 while KEEPING v3.5 served after v4 launched —
|
||||
* a demonstrated non-abandonment record, which is what the
|
||||
* default-provider policy actually asks for.
|
||||
* - It is the most widely integrated reranker across RAG frameworks, so
|
||||
* its request/response dialect is the de-facto standard (see below).
|
||||
* - ZeroEntropy's own migration guide names it as the target.
|
||||
*
|
||||
* Wire shape (verified against docs.cohere.com/reference/rerank, 2026-07-28):
|
||||
* POST https://api.cohere.com/v2/rerank
|
||||
* → { model, query, documents[], top_n? }
|
||||
* ← { results: [{ index, relevance_score }] }
|
||||
* That is byte-identical to what `gateway.rerank()` already sends and parses
|
||||
* for ZeroEntropy and DashScope — gbrain's "ZeroEntropy wire shape" IS the
|
||||
* Cohere dialect. So this recipe needs no adapter hooks, only the
|
||||
* recipe-pluggable `path` override (v0.40.6.1). Voyage (`top_k` / `data[]`)
|
||||
* remains the sole outlier, which is why issue #3439 does not block this.
|
||||
*
|
||||
* Models (docs.cohere.com/docs/rerank, verified 2026-07-28): rerank-v3.5 is
|
||||
* the multilingual 4096-token workhorse and stays the default — v4.0
|
||||
* pro/fast are listed so users can opt up without a recipe edit. The v3.0
|
||||
* english/multilingual generation is intentionally NOT listed (superseded;
|
||||
* new installs should not start there).
|
||||
*
|
||||
* PRICING — read before trusting the number below. Cohere bills rerank PER
|
||||
* SEARCH (one query + up to 100 documents), not per token, and the per-search
|
||||
* rate is rendered client-side on cohere.com/pricing so it is not
|
||||
* machine-readable; third-party trackers disagree ($1 vs $2 per 1K searches).
|
||||
* `cost_per_1m_tokens_usd` is therefore a deliberately CONSERVATIVE
|
||||
* pseudo-rate for the budget tracker's `chars/4` estimator, not a real
|
||||
* published rate: at gbrain's `balanced` shape (top_n_in=25 chunks ≈ 10K
|
||||
* estimated tokens) 0.20/1M lands at ~$0.002/search, i.e. the higher of the
|
||||
* two reported rates. It over-estimates on `tokenmax` (50 docs), which is the
|
||||
* safe direction for `--max-cost` callers. Kept as a pseudo-rate rather than
|
||||
* adding a `cost_per_search_usd` field because the budget tracker has one
|
||||
* $/1M-token unit end to end; a per-search unit is a tracker change, not a
|
||||
* recipe change. See src/core/embedding-pricing.ts for the matching key.
|
||||
*/
|
||||
export const cohere: Recipe = {
|
||||
id: 'cohere',
|
||||
name: 'Cohere',
|
||||
tier: 'openai-compat',
|
||||
implementation: 'openai-compatible',
|
||||
base_url_default: 'https://api.cohere.com/v2',
|
||||
auth_env: {
|
||||
required: ['COHERE_API_KEY'],
|
||||
setup_url: 'https://dashboard.cohere.com/api-keys',
|
||||
},
|
||||
touchpoints: {
|
||||
reranker: {
|
||||
models: ['rerank-v3.5', 'rerank-v4.0-fast', 'rerank-v4.0-pro'],
|
||||
default_model: 'rerank-v3.5',
|
||||
// Conservative pseudo-rate — see the PRICING note above.
|
||||
cost_per_1m_tokens_usd: 0.20,
|
||||
price_last_verified: '2026-07-28',
|
||||
// Cohere doesn't publish an explicit body cap; 5MB matches the
|
||||
// gateway's pre-flight ceiling used for every other rerank provider.
|
||||
max_payload_bytes: 5_000_000,
|
||||
// base_url_default already ends in /v2 → …/v2/rerank.
|
||||
path: '/rerank',
|
||||
// Hosted API, p50 well under 1s. Same 5s default the gateway uses.
|
||||
default_timeout_ms: 5_000,
|
||||
},
|
||||
},
|
||||
setup_hint:
|
||||
'Get an API key at https://dashboard.cohere.com/api-keys, then ' +
|
||||
'`export COHERE_API_KEY=...` — or add `"cohere_api_key": "..."` to ' +
|
||||
'~/.gbrain/config.json so daemon/launchd/MCP contexts see it without a ' +
|
||||
'shell export. (NOTE: `gbrain config set cohere_api_key` writes the DB ' +
|
||||
'plane, which loadConfig does NOT merge for *_api_key fields — same ' +
|
||||
'pre-existing gap as zeroentropy_api_key/voyage_api_key.) This is the ' +
|
||||
'default reranker; no `search.reranker.model` change needed.',
|
||||
};
|
||||
@@ -21,6 +21,7 @@ import { llamaServer } from './llama-server.ts';
|
||||
import { minimax } from './minimax.ts';
|
||||
import { dashscope } from './dashscope.ts';
|
||||
import { dashscopeRerank } from './dashscope-rerank.ts';
|
||||
import { cohere } from './cohere.ts';
|
||||
import { zhipu } from './zhipu.ts';
|
||||
import { azureOpenAI } from './azure-openai.ts';
|
||||
import { zeroentropyai } from './zeroentropyai.ts';
|
||||
@@ -47,6 +48,7 @@ const ALL: Recipe[] = [
|
||||
minimax,
|
||||
dashscope,
|
||||
dashscopeRerank,
|
||||
cohere,
|
||||
zhipu,
|
||||
azureOpenAI,
|
||||
zeroentropyai,
|
||||
|
||||
@@ -215,11 +215,18 @@ export const openrouter: Recipe = {
|
||||
default_model: 'cohere/rerank-v3.5',
|
||||
// Cohere bills per-search, not per-token. This is a pseudo-per-1M rate
|
||||
// for the budget tracker's heuristic (estimates tokens as chars/4).
|
||||
// At ~4K chars/search the tracker estimates ~$0.00025 — in the right
|
||||
// ballpark for the per-search bill. Patch budget-tracker.ts to honour a
|
||||
// `cost_per_search_usd` field for exact accounting.
|
||||
cost_per_1m_tokens_usd: 0.001,
|
||||
price_last_verified: '2026-06-13',
|
||||
// v0.42.69.0 correction: the old 0.001 value was ~250x LOW. OpenRouter
|
||||
// publishes $0.001/search for cohere/rerank-v3.5 (verified
|
||||
// openrouter.ai/cohere/rerank-v3.5, 2026-07-28), and gbrain's `balanced`
|
||||
// shape sends top_n_in=25 chunks ≈ 10K estimated tokens — at 0.001/1M
|
||||
// the tracker estimated $0.00001/search. The old comment's "~$0.00025 at
|
||||
// ~4K chars" was itself arithmetically wrong (4K chars → 1K tokens →
|
||||
// $0.000001). 0.10/1M puts a 10K-token search at exactly $0.001.
|
||||
// Deliberately still a pseudo-rate: the budget tracker has one
|
||||
// $/1M-token unit end to end, so a real `cost_per_search_usd` field is a
|
||||
// tracker change, not a recipe change.
|
||||
cost_per_1m_tokens_usd: 0.10,
|
||||
price_last_verified: '2026-07-28',
|
||||
// OpenRouter doesn't publish an explicit payload cap; 5MB matches
|
||||
// ZeroEntropy's upstream limit and the gateway's pre-flight ceiling.
|
||||
max_payload_bytes: 5_000_000,
|
||||
|
||||
@@ -63,6 +63,17 @@ export interface GBrainConfig {
|
||||
* config.json file-plane route is wired through today.
|
||||
*/
|
||||
voyage_api_key?: string;
|
||||
/**
|
||||
* Cohere API key (v0.42.69.0). Cohere `rerank-v3.5` is the DEFAULT reranker
|
||||
* as of the ZeroEntropy sunset, so this key sits on the default retrieval
|
||||
* path — the exact situation that bit zeroentropy_api_key (v0.37 CDX2-5+6)
|
||||
* and voyage_api_key (#2662): accepted at the file plane but never threaded
|
||||
* into the gateway env, so daemon/launchd/MCP contexts with no process-env
|
||||
* export failed silently while config.json looked complete. Wired through:
|
||||
* file plane + COHERE_API_KEY env merge below -> buildGatewayConfig env dict
|
||||
* -> recipe reads COHERE_API_KEY.
|
||||
*/
|
||||
cohere_api_key?: string;
|
||||
/** Azure OpenAI (keyless/Entra). Non-secret endpoint + deployment + Entra opt-in,
|
||||
* folded into the gateway env so the azure-openai recipe works in any shell.
|
||||
* The bearer token is minted at request time via `az` — no secret stored here. */
|
||||
@@ -581,6 +592,7 @@ export function loadConfig(): GBrainConfig | null {
|
||||
...(process.env.ANTHROPIC_API_KEY ? { anthropic_api_key: process.env.ANTHROPIC_API_KEY } : {}),
|
||||
...(process.env.ZEROENTROPY_API_KEY ? { zeroentropy_api_key: process.env.ZEROENTROPY_API_KEY } : {}),
|
||||
...(process.env.OPENROUTER_API_KEY ? { openrouter_api_key: process.env.OPENROUTER_API_KEY } : {}),
|
||||
...(process.env.COHERE_API_KEY ? { cohere_api_key: process.env.COHERE_API_KEY } : {}),
|
||||
...(process.env.GBRAIN_EMBEDDING_MODEL ? { embedding_model: process.env.GBRAIN_EMBEDDING_MODEL } : {}),
|
||||
...(process.env.GBRAIN_EMBEDDING_DIMENSIONS ? { embedding_dimensions: parseInt(process.env.GBRAIN_EMBEDDING_DIMENSIONS, 10) } : {}),
|
||||
...(process.env.GBRAIN_EXPANSION_MODEL ? { expansion_model: process.env.GBRAIN_EXPANSION_MODEL } : {}),
|
||||
@@ -919,6 +931,7 @@ export const KNOWN_CONFIG_KEYS: readonly string[] = [
|
||||
'zeroentropy_api_key',
|
||||
'openrouter_api_key',
|
||||
'voyage_api_key',
|
||||
'cohere_api_key',
|
||||
'azure_openai_endpoint',
|
||||
'azure_openai_deployment',
|
||||
'azure_openai_use_entra',
|
||||
|
||||
@@ -41,6 +41,18 @@ export const EMBEDDING_PRICING: Record<string, EmbeddingPricing> = {
|
||||
// Reused here (not a separate rerank table) because budget-tracker.ts's
|
||||
// rerank-kind lookup falls back to this same table for paid providers.
|
||||
'zeroentropyai:zerank-2': { pricePerMTok: 0.025 },
|
||||
// Cohere rerankers — the DEFAULT reranker from v0.42.69.0. Cohere bills PER
|
||||
// SEARCH (one query + up to 100 docs), not per token, and the per-search
|
||||
// rate is client-side-rendered on cohere.com/pricing (third-party trackers
|
||||
// report $1 vs $2 per 1K searches). These are CONSERVATIVE pseudo-rates in
|
||||
// this table's $/1M-token unit so `--max-cost` callers get a bounded
|
||||
// over-estimate instead of a TX2 no_pricing hard-fail on the default path.
|
||||
// Derivation: balanced sends top_n_in=25 chunks ~= 10K estimated tokens
|
||||
// (tracker estimates chars/4), so 0.20/1M ~= $0.002/search — the higher
|
||||
// reported rate. See src/core/ai/recipes/cohere.ts for the full note.
|
||||
'cohere:rerank-v3.5': { pricePerMTok: 0.20 },
|
||||
'cohere:rerank-v4.0-fast': { pricePerMTok: 0.20 },
|
||||
'cohere:rerank-v4.0-pro': { pricePerMTok: 0.20 },
|
||||
// Mistral (https://mistral.ai/pricing/api/, verified 2026-07-19)
|
||||
'mistral:mistral-embed': { pricePerMTok: 0.10 },
|
||||
'mistral:mistral-embed-2312': { pricePerMTok: 0.10 },
|
||||
|
||||
@@ -72,7 +72,14 @@ import { hnswIndexExpected } from './vector-index.ts';
|
||||
/** v0.36.0.0 cutover target: ZeroEntropy zembed-1 at 1024d via Matryoshka. */
|
||||
export const ZE_TARGET_EMBEDDING_MODEL = 'zeroentropyai:zembed-1';
|
||||
export const ZE_TARGET_EMBEDDING_DIM = 1280;
|
||||
export const ZE_TARGET_RERANKER_MODEL = 'zeroentropyai:zerank-2';
|
||||
/**
|
||||
* The reranker this planner writes into user config when the ZE retrieval
|
||||
* upgrade is applied. NOT a ZeroEntropy model: ZE's hosted API shuts down
|
||||
* 2026-09-04 (#3390), so persisting `zeroentropyai:zerank-2` into a user's
|
||||
* config here would actively install a dead default. Tracks
|
||||
* DEFAULT_RERANKER_MODEL in src/core/ai/gateway.ts.
|
||||
*/
|
||||
export const ZE_TARGET_RERANKER_MODEL = 'cohere:rerank-v3.5';
|
||||
|
||||
/** Config keys (D12). */
|
||||
export const KEY_PROMPT_SHOWN = 'ze_switch_prompt_shown';
|
||||
|
||||
@@ -211,7 +211,11 @@ export function formatBanner(plan: RetrievalUpgradeState): string {
|
||||
lines.push(' • Fastest: 442ms vs OpenAI 973ms (2.2× faster)');
|
||||
lines.push(' • Cheapest: $0.05/M tokens vs OpenAI $0.13/M (2.6× cheaper)');
|
||||
lines.push(' (sale rate $0.025/M may be promotional, subject to change)');
|
||||
lines.push(' • zerank-2 reshuffles 60% of top-1 results (real value)');
|
||||
// v0.42.69.0: this flow now installs the CURRENT default reranker
|
||||
// (cohere:rerank-v3.5), not zerank-2 — ZE's hosted reranker sunsets
|
||||
// 2026-09-04. The 60%-reshuffle figure was measured on zerank-2, so it is
|
||||
// no longer the reranker this prompt actually enables; stated generically.
|
||||
lines.push(' • A cross-encoder reranker reshuffles ~60% of top-1 results');
|
||||
lines.push(' • Only 10–18% overlap between providers — they see different');
|
||||
lines.push(' things, so a pair compounds');
|
||||
lines.push('');
|
||||
|
||||
@@ -27,7 +27,7 @@ import { applyAutocut, type AutocutDecision } from './autocut.ts';
|
||||
import { buildRelationalArm } from './relational-recall.ts';
|
||||
import { loadConfigWithEngine } from '../config.ts';
|
||||
import { dedupResults } from './dedup.ts';
|
||||
import { applyReranker } from './rerank.ts';
|
||||
import { applyReranker, type RerankerOpts } from './rerank.ts';
|
||||
import { autoDetectDetail, classifyQuery, isAmbiguousModalityQuery } from './query-intent.ts';
|
||||
import { isTitlePhraseMatch } from './title-match.ts';
|
||||
import { normalizeAlias } from './alias-normalize.ts';
|
||||
@@ -1598,7 +1598,21 @@ export async function hybridSearch(
|
||||
model: resolvedMode.reranker_model,
|
||||
timeoutMs: resolvedMode.reranker_timeout_ms,
|
||||
};
|
||||
const reranked = rerankerOpts.enabled
|
||||
// #3421 — availability pre-gate, mirroring the embedding arm's
|
||||
// `isAvailable('embedding', …)` guard above. `balanced` is
|
||||
// DEFAULT_SEARCH_MODE and ships `reranker_enabled: true`, so without this an
|
||||
// install with no reranker key issues one doomed HTTP request per query and
|
||||
// burns the full reranker_timeout_ms before applyReranker fails open —
|
||||
// silently, on every search. Probe the RESOLVED per-call model (same reason
|
||||
// the embedding arm probes the resolved column's provider, not the global
|
||||
// default). `rerankerFn` is the test seam: when a caller injects a reranker
|
||||
// no provider auth is involved, so skip the probe (matches how isAvailable
|
||||
// treats an installed chat transport stub).
|
||||
const rerankerUsable =
|
||||
rerankerOpts.enabled &&
|
||||
((rerankerOpts as RerankerOpts).rerankerFn !== undefined ||
|
||||
isAvailable('reranker', rerankerOpts.model));
|
||||
const reranked = rerankerUsable
|
||||
? await applyReranker(query, deduped, rerankerOpts as any)
|
||||
: deduped;
|
||||
|
||||
|
||||
+26
-18
@@ -95,18 +95,21 @@ export interface ModeBundle {
|
||||
searchLimit: number;
|
||||
/**
|
||||
* v0.35.0.0+ — cross-encoder reranker. Off for conservative/balanced,
|
||||
* on for tokenmax. ZeroEntropy zerank-2 by default; can be overridden
|
||||
* on for tokenmax. Cohere rerank-v3.5 by default; can be overridden
|
||||
* via `search.reranker.model`. Slots between dedup and token-budget
|
||||
* enforcement in hybrid.ts; fail-open on any RerankError (audit-logged).
|
||||
* Cost anchor: ~$0.0003/query at tokenmax topNIn=30 × ~400 tokens/chunk
|
||||
* enforcement in hybrid.ts; skipped entirely when the provider is
|
||||
* unreachable (#3421), fail-open on any RerankError (audit-logged).
|
||||
* Cost anchor: ~$0.002/query — Cohere bills PER SEARCH (one query + up to
|
||||
* 100 docs), so the anchor is flat in topNIn, not per-token
|
||||
* (rounding error vs Opus, meaningful vs Haiku).
|
||||
*/
|
||||
reranker_enabled: boolean;
|
||||
/**
|
||||
* Provider:model for the reranker. Default `'zeroentropyai:zerank-2'`.
|
||||
* Other ZE rerankers (`zerank-1`, `zerank-1-small`) work via the same
|
||||
* recipe; future Cohere/Voyage rerankers drop in as new recipes
|
||||
* declaring `touchpoints.reranker`.
|
||||
* Provider:model for the reranker. Default `'cohere:rerank-v3.5'`.
|
||||
* `rerank-v4.0-fast` / `rerank-v4.0-pro` are same-recipe opt-ups. Other
|
||||
* providers (ZeroEntropy, DashScope, OpenRouter, llama-server-reranker,
|
||||
* or any vLLM instance via `provider_base_urls.cohere`) drop in as
|
||||
* recipes declaring `touchpoints.reranker`.
|
||||
*/
|
||||
reranker_model: string;
|
||||
/** Candidates to send upstream (default 30). The full result list always
|
||||
@@ -294,7 +297,7 @@ export const MODE_BUNDLES: Readonly<Record<SearchMode, Readonly<ModeBundle>>> =
|
||||
// v0.35.0.0+: reranker off — conservative is cost-sensitive; reranker
|
||||
// spend doesn't fit the tier's value prop.
|
||||
reranker_enabled: false,
|
||||
reranker_model: 'zeroentropyai:zerank-2',
|
||||
reranker_model: 'cohere:rerank-v3.5',
|
||||
reranker_top_n_in: 30,
|
||||
reranker_top_n_out: null,
|
||||
reranker_timeout_ms: 5000,
|
||||
@@ -336,15 +339,19 @@ export const MODE_BUNDLES: Readonly<Record<SearchMode, Readonly<ModeBundle>>> =
|
||||
expansion: false,
|
||||
searchLimit: 25,
|
||||
// v0.36.0.0 (D6): reranker flipped ON for `balanced` mode bundle. The
|
||||
// real-corpus benchmark shows zerank-2 reshuffles 60% of top-1 results
|
||||
// — the headline ZE quality story reaches the 80% of installs that
|
||||
// stay on `balanced`. Per-query rerank cost ~$0.025/M tokens, ~150ms
|
||||
// p50 added latency. Missing ZEROENTROPY_API_KEY is handled via
|
||||
// src/core/search/rerank.ts fail-open contract: log to audit JSONL,
|
||||
// return input order unchanged. Opt out with
|
||||
// real-corpus benchmark showed a cross-encoder reshuffling 60% of top-1
|
||||
// results — that quality story reaches the 80% of installs that stay on
|
||||
// `balanced`. ~150ms p50 added latency.
|
||||
//
|
||||
// v0.42.69.0: the model is `cohere:rerank-v3.5` (ZE's hosted reranker
|
||||
// sunsets 2026-09-04). A MISSING provider key is now a no-op, not a
|
||||
// failed request: hybrid.ts pre-gates on isAvailable('reranker', model)
|
||||
// before applyReranker (#3421), so no HTTP is issued and no timeout is
|
||||
// paid. applyReranker's fail-open contract (audit JSONL + input order
|
||||
// unchanged) still covers keys that exist but stop working. Opt out with
|
||||
// `gbrain config set search.reranker.enabled false`.
|
||||
reranker_enabled: true,
|
||||
reranker_model: 'zeroentropyai:zerank-2',
|
||||
reranker_model: 'cohere:rerank-v3.5',
|
||||
// v0.42.3.0 D4: topNIn = searchLimit (25) so the cross-encoder scores
|
||||
// every result the limit slice will return — no unscored tail for autocut
|
||||
// to wrongly drop (Codex #2). Was 30; tracking searchLimit is the
|
||||
@@ -396,10 +403,11 @@ export const MODE_BUNDLES: Readonly<Record<SearchMode, Readonly<ModeBundle>>> =
|
||||
// tokenmax is the high-cost-tolerant tier that already pays for LLM
|
||||
// expansion + 50-result payloads. Reranker is the natural capstone:
|
||||
// better ordering of a large candidate set is where rerankers earn
|
||||
// their fee. ~$0.0003/query at this shape; rounding error vs the
|
||||
// tier's $700/mo @ Opus pairing per CLAUDE.md cost matrix.
|
||||
// their fee. ~$0.002/query at this shape (Cohere bills per search, so
|
||||
// 50 docs costs the same as 25); rounding error vs the tier's $700/mo
|
||||
// @ Opus pairing per CLAUDE.md cost matrix.
|
||||
reranker_enabled: true,
|
||||
reranker_model: 'zeroentropyai:zerank-2',
|
||||
reranker_model: 'cohere:rerank-v3.5',
|
||||
// v0.42.3.0 D4: topNIn = searchLimit (50) so every returned result is
|
||||
// cross-encoder scored — closes the Codex #2 recall gap where autocut
|
||||
// would drop the deliberately-preserved un-reranked tail (results 31-50).
|
||||
|
||||
@@ -159,7 +159,10 @@ describe('model-resolver', () => {
|
||||
});
|
||||
|
||||
test('resolveRecipe throws AIConfigError for unknown provider', () => {
|
||||
expect(() => resolveRecipe('cohere:embed-v3')).toThrow(AIConfigError);
|
||||
// NB: 'cohere' used to be the stand-in for an unregistered provider here.
|
||||
// It is a real recipe as of v0.42.69.0 (the default reranker), so this now
|
||||
// needs a provider id that genuinely isn't in the registry.
|
||||
expect(() => resolveRecipe('nosuchprovider:embed-v3')).toThrow(AIConfigError);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
/**
|
||||
* v0.42.69.0 — Cohere reranker recipe (the new DEFAULT reranker).
|
||||
*
|
||||
* The load-bearing claim this file pins: gbrain's existing "ZeroEntropy wire
|
||||
* shape" IS the Cohere dialect (`top_n` request / `results[{index,
|
||||
* relevance_score}]` response), so Cohere rides `gateway.rerank()`'s native
|
||||
* path with NO adapter hooks — only the recipe-pluggable `path` override.
|
||||
* If that stops being true, these tests fail instead of production searches.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, afterEach, beforeEach } from 'bun:test';
|
||||
import {
|
||||
configureGateway,
|
||||
resetGateway,
|
||||
rerank,
|
||||
isAvailable,
|
||||
RerankError,
|
||||
__setRerankTransportForTests,
|
||||
} from '../../src/core/ai/gateway.ts';
|
||||
import { getRecipe } from '../../src/core/ai/recipes/index.ts';
|
||||
import { MODE_BUNDLES } from '../../src/core/search/mode.ts';
|
||||
import { lookupEmbeddingPrice } from '../../src/core/embedding-pricing.ts';
|
||||
|
||||
function configureCohere(model = 'cohere:rerank-v3.5'): void {
|
||||
configureGateway({
|
||||
reranker_model: model,
|
||||
env: { COHERE_API_KEY: 'co-test-key' },
|
||||
});
|
||||
}
|
||||
|
||||
function mockResp(json: unknown, status = 200): Response {
|
||||
return new Response(JSON.stringify(json), {
|
||||
status,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
__setRerankTransportForTests(null);
|
||||
resetGateway();
|
||||
});
|
||||
|
||||
describe('cohere recipe shape', () => {
|
||||
test('registered in the static recipe registry', () => {
|
||||
expect(getRecipe('cohere')?.id).toBe('cohere');
|
||||
});
|
||||
|
||||
test('reranker touchpoint: models, default, path, payload cap', () => {
|
||||
const tp = getRecipe('cohere')!.touchpoints.reranker!;
|
||||
expect(tp.models).toEqual(['rerank-v3.5', 'rerank-v4.0-fast', 'rerank-v4.0-pro']);
|
||||
expect(tp.default_model).toBe('rerank-v3.5');
|
||||
expect(tp.path).toBe('/rerank');
|
||||
expect(tp.max_payload_bytes).toBe(5_000_000);
|
||||
expect(tp.default_timeout_ms).toBe(5_000);
|
||||
});
|
||||
|
||||
test('auth is COHERE_API_KEY and base url is the v2 surface', () => {
|
||||
const r = getRecipe('cohere')!;
|
||||
expect(r.auth_env?.required).toEqual(['COHERE_API_KEY']);
|
||||
expect(r.base_url_default).toBe('https://api.cohere.com/v2');
|
||||
});
|
||||
|
||||
test('is the default reranker for every mode bundle that enables one', () => {
|
||||
for (const mode of ['conservative', 'balanced', 'tokenmax'] as const) {
|
||||
expect(MODE_BUNDLES[mode].reranker_model).toBe('cohere:rerank-v3.5');
|
||||
}
|
||||
});
|
||||
|
||||
test('priced in the table budget-tracker rerank lookups fall back to', () => {
|
||||
// Without this, `--max-cost` callers TX2 hard-fail on the DEFAULT path.
|
||||
for (const m of ['rerank-v3.5', 'rerank-v4.0-fast', 'rerank-v4.0-pro']) {
|
||||
const hit = lookupEmbeddingPrice(`cohere:${m}`);
|
||||
expect(hit.kind).toBe('known');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('cohere rides gateway.rerank() with no adapter hooks', () => {
|
||||
beforeEach(() => configureCohere());
|
||||
|
||||
test('URL is https://api.cohere.com/v2/rerank (no /v2/v2 doubling)', async () => {
|
||||
let capturedUrl = '';
|
||||
__setRerankTransportForTests(async (url) => {
|
||||
capturedUrl = url;
|
||||
return mockResp({ results: [{ index: 0, relevance_score: 0.9 }] });
|
||||
});
|
||||
await rerank({ query: 'q', documents: ['d'] });
|
||||
expect(capturedUrl).toBe('https://api.cohere.com/v2/rerank');
|
||||
});
|
||||
|
||||
test('request body is the Cohere dialect — top_n, not top_k', async () => {
|
||||
let captured: any = null;
|
||||
__setRerankTransportForTests(async (_url, init) => {
|
||||
captured = JSON.parse(init.body as string);
|
||||
return mockResp({ results: [{ index: 0, relevance_score: 0.9 }] });
|
||||
});
|
||||
await rerank({ query: 'q', documents: ['d1', 'd2'], topN: 2 });
|
||||
expect(captured).toEqual({
|
||||
model: 'rerank-v3.5',
|
||||
query: 'q',
|
||||
documents: ['d1', 'd2'],
|
||||
top_n: 2,
|
||||
});
|
||||
});
|
||||
|
||||
test('Bearer auth from COHERE_API_KEY', async () => {
|
||||
let authHeader = '';
|
||||
__setRerankTransportForTests(async (_url, init) => {
|
||||
authHeader = new Headers(init.headers as HeadersInit).get('authorization') ?? '';
|
||||
return mockResp({ results: [{ index: 0, relevance_score: 1 }] });
|
||||
});
|
||||
await rerank({ query: 'q', documents: ['d'] });
|
||||
expect(authHeader).toBe('Bearer co-test-key');
|
||||
});
|
||||
|
||||
test('response parsing: results[{index, relevance_score}] → RerankResult[]', async () => {
|
||||
__setRerankTransportForTests(async () =>
|
||||
mockResp({
|
||||
results: [
|
||||
{ index: 2, relevance_score: 0.99 },
|
||||
{ index: 0, relevance_score: 0.4 },
|
||||
],
|
||||
}),
|
||||
);
|
||||
const out = await rerank({ query: 'q', documents: ['a', 'b', 'c'] });
|
||||
expect(out).toEqual([
|
||||
{ index: 2, relevanceScore: 0.99 },
|
||||
{ index: 0, relevanceScore: 0.4 },
|
||||
]);
|
||||
});
|
||||
|
||||
test('v4 models are accepted by the allowlist; unknown ids are not', async () => {
|
||||
__setRerankTransportForTests(async () => mockResp({ results: [] }));
|
||||
configureCohere('cohere:rerank-v4.0-pro');
|
||||
await expect(rerank({ query: 'q', documents: ['d'] })).resolves.toEqual([]);
|
||||
configureCohere('cohere:rerank-v9-imaginary');
|
||||
await expect(rerank({ query: 'q', documents: ['d'] })).rejects.toBeInstanceOf(RerankError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('availability gate wiring', () => {
|
||||
test('isAvailable("reranker") is false without COHERE_API_KEY', () => {
|
||||
configureGateway({ reranker_model: 'cohere:rerank-v3.5', env: {} });
|
||||
expect(isAvailable('reranker')).toBe(false);
|
||||
expect(isAvailable('reranker', 'cohere:rerank-v3.5')).toBe(false);
|
||||
});
|
||||
|
||||
test('isAvailable("reranker") is true with the key set', () => {
|
||||
configureCohere();
|
||||
expect(isAvailable('reranker', 'cohere:rerank-v3.5')).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -19,8 +19,8 @@ describe('Mode bundle defaults (D6)', () => {
|
||||
expect(MODE_BUNDLES.balanced.reranker_enabled).toBe(true);
|
||||
});
|
||||
|
||||
test('balanced reranker model is zeroentropyai:zerank-2', () => {
|
||||
expect(MODE_BUNDLES.balanced.reranker_model).toBe('zeroentropyai:zerank-2');
|
||||
test('balanced reranker model is cohere:rerank-v3.5', () => {
|
||||
expect(MODE_BUNDLES.balanced.reranker_model).toBe('cohere:rerank-v3.5');
|
||||
});
|
||||
|
||||
test('conservative reranker stays off (cheap tier)', () => {
|
||||
|
||||
+1
-1
@@ -120,7 +120,7 @@ describe('doctor command', () => {
|
||||
} as any);
|
||||
expect(check.status).toBe('warn');
|
||||
expect(check.message).toContain('unknown');
|
||||
expect(check.message).toContain('ZEROENTROPY_API_KEY');
|
||||
expect(check.message).toContain('API_KEY');
|
||||
});
|
||||
} finally {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
/**
|
||||
* Cohere rerank live E2E tests.
|
||||
*
|
||||
* Real HTTP round-trip against `api.cohere.com/v2/rerank`. Gated on
|
||||
* `COHERE_API_KEY` — when absent every test skips gracefully so
|
||||
* `bun run test:e2e` stays green on machines without a Cohere account.
|
||||
*
|
||||
* WHY THIS FILE EXISTS: every other Cohere test in the repo asserts against
|
||||
* a mock response WE authored (`mockResp({ results: [...] })`). That proves
|
||||
* gbrain's parser handles the shape we imagined, not the shape Cohere
|
||||
* actually returns — so a wire-shape drift would keep CI green while the
|
||||
* default reranker silently broke on the query hot path. This file is the
|
||||
* only thing that can catch that, which matters because `rerank-v3.5` is
|
||||
* the DEFAULT reranker and `balanced` (the default search mode) has
|
||||
* reranking on.
|
||||
*
|
||||
* Pins (only meaningful when the env var is set):
|
||||
* - POST /v2/rerank returns a `results` array whose elements carry
|
||||
* `index` + `relevance_score`, and gateway.rerank() maps them to
|
||||
* RerankResult[] with `index` + `relevanceScore`.
|
||||
* - Ranking is semantically correct: the one relevant document outranks
|
||||
* the distractors (guards against an off-by-one in index mapping,
|
||||
* which a shape-only assertion would miss entirely).
|
||||
* - All three models named in the recipe allowlist actually exist and
|
||||
* respond (a typo'd model id otherwise surfaces only in production).
|
||||
*
|
||||
* Cost note: Cohere bills rerank PER SEARCH, not per token. Each test is
|
||||
* 1 search unit; the file is ~4 units total. The trial tier allows 1,000
|
||||
* calls/month with no payment method, so this is free to run.
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { configureGateway, resetGateway, rerank } from '../../src/core/ai/gateway.ts';
|
||||
|
||||
const KEY = process.env.COHERE_API_KEY;
|
||||
const RUN = Boolean(KEY);
|
||||
|
||||
const QUERY = 'what is the capital of France';
|
||||
// index 1 is the only relevant document; 0 and 2 are distractors.
|
||||
const DOCS = [
|
||||
'Bananas are yellow and grow in tropical climates.',
|
||||
'Paris is the capital and largest city of France.',
|
||||
'Berlin is the capital of Germany.',
|
||||
];
|
||||
const RELEVANT = 1;
|
||||
|
||||
describe.skipIf(!RUN)('Cohere rerank — live wire-shape verification', () => {
|
||||
beforeAll(() => {
|
||||
configureGateway({ env: { COHERE_API_KEY: KEY } } as never);
|
||||
});
|
||||
afterAll(() => {
|
||||
// Module-global gateway must not leak into sibling test files (#3066 class).
|
||||
resetGateway();
|
||||
});
|
||||
|
||||
test('rerank-v3.5 returns index + relevanceScore and ranks the relevant doc first', async () => {
|
||||
const out = await rerank({ query: QUERY, documents: DOCS, model: 'cohere:rerank-v3.5' });
|
||||
|
||||
expect(Array.isArray(out)).toBe(true);
|
||||
expect(out.length).toBe(DOCS.length);
|
||||
|
||||
for (const r of out) {
|
||||
expect(typeof r.index).toBe('number');
|
||||
expect(typeof r.relevanceScore).toBe('number');
|
||||
expect(r.index).toBeGreaterThanOrEqual(0);
|
||||
expect(r.index).toBeLessThan(DOCS.length);
|
||||
}
|
||||
|
||||
// Descending by score, and the relevant doc wins. This is the assertion a
|
||||
// mock cannot make honestly.
|
||||
expect(out[0].index).toBe(RELEVANT);
|
||||
for (let i = 1; i < out.length; i++) {
|
||||
expect(out[i - 1].relevanceScore).toBeGreaterThanOrEqual(out[i].relevanceScore);
|
||||
}
|
||||
|
||||
// Every input index appears exactly once (catches index-mapping bugs).
|
||||
expect([...out.map((r) => r.index)].sort()).toEqual([0, 1, 2]);
|
||||
}, 30_000);
|
||||
|
||||
test('the reachable recipe models exist and rank correctly', async () => {
|
||||
// rerank-v4.0-pro is DELIBERATELY excluded. Live-verified 2026-07-28 on a
|
||||
// fresh trial key: back-to-back calls with the same key returned
|
||||
// rerank-v3.5 HTTP 200 in 0.21s
|
||||
// rerank-v4.0-fast HTTP 200 in 0.20s
|
||||
// rerank-v4.0-pro no response at all, 20s+ (curl HTTP 000)
|
||||
// So -pro is not reachable on the trial tier — not account-wide throttling,
|
||||
// since its siblings answered instantly in the same second. Asserting it
|
||||
// here would make this test fail for every trial-tier contributor. The
|
||||
// recipe still lists it for paid keys; docs carry the caveat.
|
||||
let first = true;
|
||||
for (const model of ['cohere:rerank-v3.5', 'cohere:rerank-v4.0-fast']) {
|
||||
if (!first) await new Promise((r) => setTimeout(r, 8_000));
|
||||
first = false;
|
||||
const out = await rerank({ query: QUERY, documents: DOCS, model });
|
||||
expect(out.length, `${model} returned no results`).toBe(DOCS.length);
|
||||
expect(out[0].index, `${model} misranked`).toBe(RELEVANT);
|
||||
}
|
||||
}, 60_000);
|
||||
});
|
||||
@@ -35,9 +35,22 @@ import {
|
||||
|
||||
const API_KEY = process.env.ZEROENTROPY_API_KEY;
|
||||
|
||||
// Skip the entire file when env is absent. `describe.skipIf` exists in
|
||||
// modern bun:test; fall back to in-test guards for older runners.
|
||||
const skipAll = !API_KEY;
|
||||
/**
|
||||
* ZeroEntropy's hosted API shuts down 2026-09-04 (#3390). On and after that
|
||||
* date these tests can only fail — a stale key still in someone's shell would
|
||||
* turn `bun run test:e2e` red for a reason that has nothing to do with their
|
||||
* change. Skip past the sunset regardless of key presence.
|
||||
* `GBRAIN_ZE_SUNSET_OVERRIDE=1` forces the tests to run anyway (useful only if
|
||||
* ZE extends the date, or when replaying against a self-hosted endpoint).
|
||||
*/
|
||||
const ZE_SUNSET = Date.parse('2026-09-04T00:00:00Z');
|
||||
const sunsetPassed =
|
||||
Date.now() >= ZE_SUNSET && process.env.GBRAIN_ZE_SUNSET_OVERRIDE !== '1';
|
||||
|
||||
// Skip the entire file when env is absent (or the hosted API is gone).
|
||||
// `describe.skipIf` exists in modern bun:test; fall back to in-test guards
|
||||
// for older runners.
|
||||
const skipAll = !API_KEY || sunsetPassed;
|
||||
|
||||
beforeAll(() => {
|
||||
if (skipAll) return;
|
||||
@@ -56,7 +69,9 @@ afterAll(() => {
|
||||
describe('ZE live — embed round-trip', () => {
|
||||
test('embed(["text"]) returns Float32Array[2560]', async () => {
|
||||
if (skipAll) {
|
||||
console.warn('[skip] ZEROENTROPY_API_KEY not set');
|
||||
console.warn(sunsetPassed
|
||||
? '[skip] ZeroEntropy hosted API sunset (2026-09-04) — see #3390'
|
||||
: '[skip] ZEROENTROPY_API_KEY not set');
|
||||
return;
|
||||
}
|
||||
const [v] = await embed(['hello world']);
|
||||
@@ -70,7 +85,9 @@ describe('ZE live — embed round-trip', () => {
|
||||
|
||||
test('embedQuery("text") returns Float32Array[2560] (query side)', async () => {
|
||||
if (skipAll) {
|
||||
console.warn('[skip] ZEROENTROPY_API_KEY not set');
|
||||
console.warn(sunsetPassed
|
||||
? '[skip] ZeroEntropy hosted API sunset (2026-09-04) — see #3390'
|
||||
: '[skip] ZEROENTROPY_API_KEY not set');
|
||||
return;
|
||||
}
|
||||
const v = await embedQuery('what is foo');
|
||||
@@ -82,7 +99,9 @@ describe('ZE live — embed round-trip', () => {
|
||||
|
||||
test('embed batch of 3 returns 3 vectors in order', async () => {
|
||||
if (skipAll) {
|
||||
console.warn('[skip] ZEROENTROPY_API_KEY not set');
|
||||
console.warn(sunsetPassed
|
||||
? '[skip] ZeroEntropy hosted API sunset (2026-09-04) — see #3390'
|
||||
: '[skip] ZEROENTROPY_API_KEY not set');
|
||||
return;
|
||||
}
|
||||
const out = await embed(['one', 'two', 'three']);
|
||||
@@ -110,7 +129,9 @@ const ZE_TEST_TIMEOUT_MS = 30_000;
|
||||
describe('ZE live — rerank round-trip', () => {
|
||||
test('rerank({query, documents}) returns sorted RerankResult[]', async () => {
|
||||
if (skipAll) {
|
||||
console.warn('[skip] ZEROENTROPY_API_KEY not set');
|
||||
console.warn(sunsetPassed
|
||||
? '[skip] ZeroEntropy hosted API sunset (2026-09-04) — see #3390'
|
||||
: '[skip] ZEROENTROPY_API_KEY not set');
|
||||
return;
|
||||
}
|
||||
const out = await rerank({
|
||||
@@ -141,7 +162,9 @@ describe('ZE live — rerank round-trip', () => {
|
||||
|
||||
test('rerank with top_n=2 returns at most 2 results', async () => {
|
||||
if (skipAll) {
|
||||
console.warn('[skip] ZEROENTROPY_API_KEY not set');
|
||||
console.warn(sunsetPassed
|
||||
? '[skip] ZeroEntropy hosted API sunset (2026-09-04) — see #3390'
|
||||
: '[skip] ZEROENTROPY_API_KEY not set');
|
||||
return;
|
||||
}
|
||||
const out = await rerank({
|
||||
@@ -157,7 +180,9 @@ describe('ZE live — rerank round-trip', () => {
|
||||
describe('ZE live — flexible dims', () => {
|
||||
test('1280-dim embedding returns Float32Array[1280]', async () => {
|
||||
if (skipAll) {
|
||||
console.warn('[skip] ZEROENTROPY_API_KEY not set');
|
||||
console.warn(sunsetPassed
|
||||
? '[skip] ZeroEntropy hosted API sunset (2026-09-04) — see #3390'
|
||||
: '[skip] ZEROENTROPY_API_KEY not set');
|
||||
return;
|
||||
}
|
||||
resetGateway();
|
||||
|
||||
@@ -47,14 +47,14 @@ describe('resolveLiveRerankerModel — divergence fix', () => {
|
||||
expect(resolved).toBe('llama-server-reranker:qwen3-reranker-4b');
|
||||
});
|
||||
|
||||
test('returns the mode-bundle default when no override is set (balanced enables zerank-2)', async () => {
|
||||
test('returns the mode-bundle default when no override is set (balanced enables cohere rerank-v3.5)', async () => {
|
||||
// balanced mode bundle has reranker_enabled: true + reranker_model:
|
||||
// 'zeroentropyai:zerank-2' baked in. Pre-fix this case returned
|
||||
// 'cohere:rerank-v3.5' baked in. Pre-fix this case returned
|
||||
// undefined; post-fix doctor sees what search actually uses.
|
||||
configureGateway({ env: {} });
|
||||
const engine = makeEngineStub({});
|
||||
const resolved = await resolveLiveRerankerModel(engine);
|
||||
expect(resolved).toBe('zeroentropyai:zerank-2');
|
||||
expect(resolved).toBe('cohere:rerank-v3.5');
|
||||
});
|
||||
|
||||
test('returns undefined when reranker is explicitly disabled via config', async () => {
|
||||
@@ -93,7 +93,7 @@ describe('resolveLiveRerankerModel — divergence fix', () => {
|
||||
const resolved = await resolveLiveRerankerModel(engine);
|
||||
// balanced mode bundle is the safety fallback when search.mode is unset
|
||||
// (and here, every config read failed) — and balanced enables
|
||||
// zeroentropyai:zerank-2 by default.
|
||||
expect(resolved).toBe('zeroentropyai:zerank-2');
|
||||
// cohere:rerank-v3.5 by default.
|
||||
expect(resolved).toBe('cohere:rerank-v3.5');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -64,7 +64,7 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
|
||||
expansion: false,
|
||||
searchLimit: 10,
|
||||
reranker_enabled: false,
|
||||
reranker_model: 'zeroentropyai:zerank-2',
|
||||
reranker_model: 'cohere:rerank-v3.5',
|
||||
reranker_top_n_in: 30,
|
||||
reranker_top_n_out: null,
|
||||
reranker_timeout_ms: 5000,
|
||||
@@ -95,7 +95,7 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
|
||||
expansion: false,
|
||||
searchLimit: 25,
|
||||
reranker_enabled: true,
|
||||
reranker_model: 'zeroentropyai:zerank-2',
|
||||
reranker_model: 'cohere:rerank-v3.5',
|
||||
// v0.42.3.0 D4: topNIn = searchLimit (25), was 30.
|
||||
reranker_top_n_in: 25,
|
||||
reranker_top_n_out: null,
|
||||
@@ -125,7 +125,7 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
|
||||
expansion: true,
|
||||
searchLimit: 50,
|
||||
reranker_enabled: true,
|
||||
reranker_model: 'zeroentropyai:zerank-2',
|
||||
reranker_model: 'cohere:rerank-v3.5',
|
||||
// v0.42.3.0 D4: topNIn = searchLimit (50), was 30.
|
||||
reranker_top_n_in: 50,
|
||||
reranker_top_n_out: null,
|
||||
|
||||
@@ -28,6 +28,7 @@ import {
|
||||
configureGateway,
|
||||
resetGateway,
|
||||
__setEmbedTransportForTests,
|
||||
__setRerankTransportForTests,
|
||||
} from '../../src/core/ai/gateway.ts';
|
||||
import type { PageInput, SearchOpts } from '../../src/core/types.ts';
|
||||
import type { RerankInput, RerankResult } from '../../src/core/ai/gateway.ts';
|
||||
@@ -279,6 +280,56 @@ describe('hybridSearch — reranker enabled (reorder)', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('hybridSearch — reranker availability pre-gate (#3421)', () => {
|
||||
// `balanced` is DEFAULT_SEARCH_MODE and ships reranker_enabled=true, so
|
||||
// without a pre-gate every query on an install with no reranker key issues a
|
||||
// doomed HTTP request and burns the full reranker_timeout_ms, silently.
|
||||
// The gateway here is configured with OPENAI_API_KEY only — no COHERE_API_KEY
|
||||
// — so the default reranker is unavailable. Assert gateway.rerank() is never
|
||||
// reached: the transport stub must stay untouched.
|
||||
test('reranker enabled + no provider key → gateway.rerank is never called', async () => {
|
||||
let transportCalls = 0;
|
||||
__setRerankTransportForTests(async () => {
|
||||
transportCalls++;
|
||||
return new Response(JSON.stringify({ results: [] }), {
|
||||
status: 200,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
});
|
||||
});
|
||||
try {
|
||||
const baseline = await hybridSearch(engine, 'alpha keyword', { limit: 10 });
|
||||
// enabled=true, NO rerankerFn → the real gateway path would be taken.
|
||||
const out = await hybridSearch(engine, 'alpha keyword', {
|
||||
limit: 10,
|
||||
reranker: { enabled: true, topNIn: 30, topNOut: null },
|
||||
});
|
||||
expect(transportCalls).toBe(0);
|
||||
expect(out.map(r => r.slug)).toEqual(baseline.map(r => r.slug));
|
||||
} finally {
|
||||
__setRerankTransportForTests(null);
|
||||
}
|
||||
});
|
||||
|
||||
test('injected rerankerFn is exempt from the gate (test seam still works)', async () => {
|
||||
let called = 0;
|
||||
const out = await hybridSearch(engine, 'alpha keyword', {
|
||||
limit: 10,
|
||||
autocut: false,
|
||||
reranker: {
|
||||
enabled: true,
|
||||
topNIn: 30,
|
||||
topNOut: null,
|
||||
rerankerFn: async (input: RerankInput): Promise<RerankResult[]> => {
|
||||
called++;
|
||||
return input.documents.map((_, i) => ({ index: i, relevanceScore: 0.5 }));
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(called).toBe(1);
|
||||
expect(out.length).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('hybridSearch — fail-open contract end-to-end', () => {
|
||||
test('rerankerFn throws → results still come back (RRF order preserved)', async () => {
|
||||
const baseline = await hybridSearch(engine, 'alpha keyword', { limit: 10 });
|
||||
|
||||
Reference in New Issue
Block a user