fix(doctor,upgrade): warn brains pinned to a provider shutting down 2026-09-04 (#3548)

Wave-assembled from PR #3548 by @time-attack. Composition conflicts vs wave
item #3993 resolved: EmbedResult keeps both chunkless_pages_healed and the
PR's lock_skipped; KEY_FILES.md keeps the #3993 embed entry and takes this
PR's migrate-embeddings entry.

Co-Authored-By: Garry Tan <garrytan@gmail.com>

docs/ai-providers/zeroentropy.md

@@ -1,5 +1,23 @@
 # ZeroEntropy — zembed-1 + zerank-2
 
+> **Hosted API shutdown: 2026-09-04.** ZeroEntropy announced (2026-07-24)
+> that its hosted endpoints — `/models/embed` and `/models/rerank` — shut
+> down on that date. A brain still embedding through the hosted API loses
+> semantic retrieval entirely on that date: query embedding uses the same
+> endpoint, so **existing vectors become unqueryable**, not just new
+> content. Two fixes, either works:
+>
+> 1. **Self-host the same model** — zembed-1 weights are Apache-2.0. Serve
+>    them via `llama-server` or Ollama and point the config at the local
+>    endpoint. Keeps every existing vector; no re-embed at all.
+> 2. **Migrate to another provider** — `gbrain migrate embeddings --to
+>    <provider:model> --dim <N> --dry-run` (resumable; see
+>    [the migration guide](../guides/embedding-migration.md)). `gbrain
+>    doctor` (check `provider_sunset`) prints this command with your
+>    brain's actual `--dim` filled in.
+>
+> The hosted setup below remains accurate until the shutdown date.
+
 [ZeroEntropy](https://zeroentropy.dev) ships two specialized small models
 for retrieval pipelines:
 

docs/guides/embedding-migration.md

@@ -27,6 +27,37 @@ gbrain migrate embeddings --to voyage:voyage-3-large --yes
 declared width and is required for recipes that don't declare one (litellm,
 llama-server, and other bring-your-own-model providers).
 
+**Pick `--dim` = your brain's current column width when the target supports
+it.** A different width triggers the destructive schema transition (column +
+index rebuild across all three dim-pinned tables); the same width skips it
+entirely. `gbrain doctor` (check `provider_sunset`, for providers with an
+announced shutdown) prints the paste-ready command with your actual width
+already filled in — it reads the real `vector(N)` column, not the config
+value, which can drift.
+
+## How affected brains find out (provider sunsets)
+
+Two surfaces flag a brain whose embedding model (or reranker) is on a
+provider with an announced hosted-API shutdown, such as ZeroEntropy
+(2026-09-04):
+
+- **`gbrain doctor`** — the `provider_sunset` check warns on every run until
+  the brain is off the provider. After the shutdown date it escalates to
+  `fail` only when embedded vectors actually exist on the dead provider
+  (retrieval is genuinely down); a zero-vector brain whose config merely
+  resolves to the dead default stays `warn`, so doctor-as-CI-gate setups
+  don't start exiting 1 on the date. The reranker side resolves through the
+  same plane search actually reranks with (the mode bundle +
+  `search.reranker.*` overrides). The message carries the paste-ready
+  migration command with the brain's actual `--dim`. Accepted the risk?
+  `gbrain config set doctor.suppress_provider_sunset true` silences it.
+- **`gbrain upgrade`** — a one-shot banner (gated by
+  `ze_sunset_notice_shown`) with the same two fixes.
+
+Both state the full consequence: after the shutdown, **existing vectors
+become unqueryable** — query embedding uses the same endpoint as ingestion —
+not just new content.
+
 ## What it does, in order
 
 1. **Plan.** Counts every chunk not already in the target embedding space —
@@ -88,6 +119,12 @@ continues where it stopped. An in-flight marker (`embedding_migration.state`
 in DB config) records the target; it is cleared only when the backlog drains
 to zero.
 
+One caveat after a HARD kill (SIGKILL, crash, power loss — not Ctrl-C): the
+run's per-source single-flight embed lock is left behind, and an immediate
+re-run skips the re-embed and reports the migration as paused. The command
+says so explicitly (`lock_skipped` in `--json`); the lock expires on its own
+after at most 60 minutes, then the same re-run resumes normally.
+
 A page whose chunks straddle two stale batches is embedded correctly but not
 stamped by the embed loop (which only stamps all-or-nothing per batch), so the
 migration runs one reconcile pass after the drain that stamps every

docs/integrations/embedding-providers.md

@@ -23,7 +23,7 @@ The resolved provider + dimensions get persisted to `~/.gbrain/config.json` atom
 
 | Provider | env vars | default dims | cost ($/1M tokens) | local? | multimodal? |
 |---|---|---|---|---|---|
-| `zeroentropyai` | `ZEROENTROPY_API_KEY` | 2560 (Matryoshka to 1280/640/320/...) | 0.05 | no | no |
+| `zeroentropyai` (hosted API **shuts down 2026-09-04** — see note below) | `ZEROENTROPY_API_KEY` | 2560 (Matryoshka to 1280/640/320/...) | 0.05 | no | no |
 | `openai` | `OPENAI_API_KEY` | 1536 | 0.13 | no | no |
 | `openrouter` | `OPENROUTER_API_KEY` | 1536 | 0.02 | no | model-dependent |
 | `voyage` | `VOYAGE_API_KEY` | 1024 | 0.18 | no | yes (`voyage-multimodal-3`) |
@@ -42,6 +42,8 @@ The resolved provider + dimensions get persisted to `~/.gbrain/config.json` atom
 
 **Note on local providers.** Ollama and llama-server have no required API key, so they don't show up in env-detection auto-pick. Pick them explicitly with `--embedding-model ollama:<model>` to avoid silently routing to a daemon that may not be running.
 
+**Note on the ZeroEntropy hosted API.** ZeroEntropy announced (2026-07-24) that its hosted endpoints shut down on **2026-09-04**. A brain still embedding through the hosted API loses semantic retrieval entirely on that date — query embedding uses the same endpoint, so existing vectors become unqueryable, not just new content. Either self-host the Apache-2.0 zembed-1 weights via llama-server/Ollama (keeps every existing vector, no re-embed), or migrate with `gbrain migrate embeddings` — see [the migration guide](../guides/embedding-migration.md). `gbrain doctor` (check `provider_sunset`) flags affected brains and prints the paste-ready command with the brain's actual `--dim` filled in.
+
 ## If first import fails
 
 If `gbrain import` fails with `expected N dimensions, not M`, run `gbrain doctor`. The output will print the exact `gbrain config set ...` or `gbrain retrieval-upgrade` command to repair the mismatch. **You should not need to delete `~/.gbrain`.** The bug-class that historically forced `rm -rf` recoveries is closed as of v0.37.