Files
Garry TanandClaude Fable 5 f83449be28 test(e2e): live Cohere rerank verification + trial-tier caveat
Every other Cohere test asserts against a mock response we authored, so a
wire-shape drift would keep CI green while the DEFAULT reranker silently
broke on the query hot path. This adds the only test that can catch that.

Live-verified against api.cohere.com/v2/rerank on 2026-07-28:
- real response shape is {results:[{index, relevance_score}]} — matches the
  parser exactly, so the mocks were accidentally correct (now established)
- ranking is semantically right; gateway maps to {index, relevanceScore}
- measured latency ~0.2s, not the ~614ms third-party leaderboards claim
- rerank-v4.0-pro is UNREACHABLE on a trial key (no response, 20s+) while
  v3.5 and v4.0-fast answered in 0.2s the same second — excluded from the
  assertion and documented, since asserting it would fail for every
  trial-tier contributor

Gated on COHERE_API_KEY; skips cleanly (4 skip, 0 fail) without it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 11:09:30 -07:00
..