diff --git a/apps/server/docs/ai-context/langfuse-tracing.md b/apps/server/docs/ai-context/langfuse-tracing.md new file mode 100644 index 000000000..5d8f789d8 --- /dev/null +++ b/apps/server/docs/ai-context/langfuse-tracing.md @@ -0,0 +1,111 @@ +# Langfuse LLM-native 观测接入 + +逐条 prompt 级 trace + 评测 + 成本归因到用户/会话。阶段 1(chat completion + TTS speech)已实现并验证,见下「已实现」与 `verifications/langfuse-tracing.md`。 + +## 目标 + +1. 逐条 prompt trace：每次 `/api/v1/openai/chat/completions` 的 input messages、output、model 完整可读。 +2. 评测 eval：dataset、人工标注、LLM-as-judge。 +3. 成本归因：按 user 和按 conversation/session 切分 token 与成本。 + +## 为什么直接 Langfuse，不先用 Grafana 顶 + +Grafana 栈（Prometheus/Tempo/Loki）已经管好 ops 聚合层，继续不动：rate、latency、聚合 token 吞吐/消耗、按模型 flux、错误、fallback、上游健康。这层完整。 + +上面三个目标里 Grafana 栈的实际能力： + +- 逐条 prompt trace：Tempo 能塞 span，但 trace 是 head-sampled，采样比 < 1 直接丢 prompt；span 属性有体积上限，长 prompt 被截；没有把 prompt 当一等对象读/对比/标注的界面。 +- eval：零能力，dataset / 标注队列 / LLM-as-judge 全得自己造。 +- 按 user/session 成本：Prometheus 按 userId/sessionId 做 label 会高基数爆炸，做不了。按 user 的成本原料其实在 `llm_request_log` 表里，用 SQL 能查，跟 Grafana 无关。 + +这三件事里正文采集是最重的活，两边都得从零写，先做 Grafana 一点不省。真正能省的只有 eval 和 per-user/session 成本，而这俩在 Grafana 上等于手搓一个更差的 Langfuse，迁移时全扔。所以不走「先 Grafana 再 Langfuse」，直接 Langfuse 补 LLM-native 这一层。 + +边界划分： + +- Grafana 栈：ops 指标，不动。 +- Langfuse：逐条 prompt trace + 正文 + eval + user/session 成本归因。 + +## 选型：Langfuse v5 = OpenTelemetry SpanProcessor + +Langfuse v5 JS SDK 基于 OpenTelemetry，`@langfuse/otel` 的 `LangfuseSpanProcessor` 就是一个 OTel SpanProcessor。AIRI 这里没有把它挂到现有 NodeSDK 上，而是起一个独立 `NodeTracerProvider` 并通过 `setLangfuseTracerProvider()` 只给 `@langfuse/tracing` 使用。 + +- Grafana 出口：`OTLPTraceExporter` → Grafana Cloud Tempo。 +- Langfuse 出口：独立 provider 上的 `LangfuseSpanProcessor` → Langfuse Cloud。 +- 两者共享 OTel context/trace id，但不共享 SpanProcessor，避免 prompt/completion 正文进 Grafana Tempo。 + +不走「复用 OTLP exporter 指向 Langfuse」的原因：① 目标里有 eval，eval 只能走 Langfuse SDK/API，OTLP 解决不了；② 现有 span 用 `airi.gen_ai.*` 自定义 attribute key，Langfuse 不认，得改成它认的 generation 字段。一套 SDK 把 trace + 正文 + 成本 + eval 全包，不维护两条上报路。 + +需要的包：`@langfuse/tracing`、`@langfuse/otel`。 + +## 部署形态与脱敏决策 + +- 形态：Langfuse Cloud。 +- 脱敏：不脱敏。prompt/completion 正文全量出境到 Langfuse 托管，已确认可接受。 +- 凭据:`LANGFUSE_PUBLIC_KEY` / `LANGFUSE_SECRET_KEY` / `LANGFUSE_BASE_URL`(SDK 默认变量名,带下划线),走 Railway env secret,本地走 `.env.local`。instrumentation.ts 和网关 route 都直读 `process.env`,**不进 `env.ts` 的 valibot schema**。原因:instrumentation 是 preload(在 env 解析之前跑),且这是部署开关(类比 NODE_ENV)不是服务依赖,不需要进 DI。 + +## Session 来源决策 + +网关 `openai/v1/index.ts` 是无状态 OpenAI 兼容代理，本身没有 conversation/session 概念，每请求只有 `c.get('user')` 的 userId 和一个临时 `nanoid()` requestId。`chats` 表存在但跟这条代理路径零关联（那是另一套 `/chats` API）。 + +所以 session id 必须客户端传。决策：客户端传对话 id，网关读出来当 Langfuse `sessionId`。 + +落地点几乎免费：stage-ui 的 `packages/stage-ui/src/libs/providers/providers/official/shared.ts` 已有 `withCredentials()` fetch 包装器在每请求注入 `Authorization` header。在同一处加一个 `x-airi-session-id` header 即可，不用碰 `@xsai` 的 body 透传。stage-web 和 stage-tamagotchi 都复用这个 provider，改一处两端生效。 + +改端范围仅此一处。telegram-bot 自带 provider，不经 server 网关，不在范围内。 + +## 成本归因方案 + +- USD 真金成本：交给 Langfuse 按 model 定价表自动算（传 `usageDetails` 的 input/output tokens 即可）。风险见下。 +- flux 业务成本：作为自定义字段放 generation 的 `metadata`（flux 不是货币，不塞 `costDetails`）。 +- 这样业务成本（flux）和真金成本（USD）都在，按 user / session 都能切。 + +## 已实现(阶段 1:chat completion) + +### `apps/server/instrumentation.ts` + +- `langfuseEnabled = !!LANGFUSE_PUBLIC_KEY && !!LANGFUSE_SECRET_KEY`,与 `otlpEndpoint` **独立门控**。两者任一开启就启动 NodeSDK;只配一个不会让另一个静默 no-op。 +- OTLP 的 trace exporter / metric reader / log processor 仅在 `otlpEndpoint` 存在时挂(条件 spread),NodeSDK 的 `spanProcessors` 数组只装 OTLP 的 `BatchSpanProcessor`。 +- Langfuse 起独立 `NodeTracerProvider`(`langfuseProvider`),其上挂 `LangfuseSpanProcessor`(`exportMode: 'batched'`),再 `setLangfuseTracerProvider(langfuseProvider)` 让 `startObservation` 路由到它。 +- `shouldExportSpan`:只放行带 `langfuse.*` 属性的 span(SDK 创建的 generation 都带 `langfuse.observation.type` 等)。这是防御性兜底,独立 provider 本来就只见自己的 span。 +- 独立 provider 显式 `AlwaysOnSampler`:调低 `OTEL_TRACES_SAMPLING_RATIO` 给 Grafana 降量,不会丢 Langfuse generation,逐条 prompt 捕获保持完整。 +- SIGTERM:`Promise.all([sdk.shutdown(), langfuseProvider?.shutdown()])`。`sdk.shutdown()` 不会 drain 独立 provider,必须显式 shutdown,否则最后一批 generation 在部署重启时丢失。 + +### `apps/server/src/routes/openai/v1/index.ts`(`handleCompletion`) + +`langfuseEnabled` 门控(只读一次,非每请求):读 `process.env.LANGFUSE_TRACING_ACTIVE`,这个 sentinel 由 instrumentation.ts 在 `setLangfuseTracerProvider()` 成功**之后**才置 `'1'`。**禁用时不创建 generation** —— 这很关键:Langfuse 关闭时没调 `setLangfuseTracerProvider`,`startObservation` 会 fallback 到全局 provider,正文就会漏进 OTLP/Grafana。gate 绑定真实 provider 状态(单一真相在 instrumentation.ts),而不是在 route 里独立再判一次 key —— 避免将来改 instrumentation 的开关条件时两处 desync 导致正文漏到错误后端。 + +output(给 eval 用,要可读):非流式 = `responseBody`;流式 = 从 SSE delta 解析出的 assistant 正文(`extractSseDeltaText` 逐行解析 `choices[0].delta.content`,**不是**原始 `data: {...}` 框架),硬上界 1M 字符防止长输出 × 高并发占内存。 + +generation 形态: + +- `startObservation('chat.completion', { input: body.messages, model: requestModel, metadata: { requestId, stream } }, { asType: 'generation' })`。 +- 身份:`generation.otelSpan.setAttribute('langfuse.user.id', user.id)`;有 `x-airi-session-id` header 时再 set `langfuse.session.id`。这两个 compat 属性被平台提升为 trace 级,支撑按 user/session 归因。 +- output:非流式 = `responseBody`;流式 = 后台累积的 assistant 正文。 +- usageDetails:`{ input: promptTokens, output: completionTokens }`,复用计费已提取的 usage。 +- metadata:保留 `{ requestId, stream }`,完成时补 `{ fluxConsumed }`。 +- 生命周期:5 个退出分支各 `generation?.update(...)` + `generation?.end()` 恰好一次 —— router throw catch、`!response.ok`、流式 interrupted(finally)、流式 completed(finally)、非流式。错误分支标 `level: 'ERROR'` + `statusMessage`。流式 generation 在后台 async IIFE 的 finally 里结束,跟 `span.end()` 对齐,不在 response 返回时提前结束。 + +### `apps/server/src/routes/openai/v1/index.ts`(`handleTTS`) + +- `startTtsGeneration({ input: { text, voice, speed, responseFormat }, model, requestId, userId, sessionId })` 创建 `tts.speech` generation。 +- 不缓冲二进制 audio 到 Langfuse；成功 output 只记录 `{ contentType }`。 +- usageDetails 使用 `{ input: inputChars }`，flux 作为 metadata 记录。 +- router throw、上游非 2xx、billing/Redis failure 都会 `fail(...)` 并 end；成功在 `ttsMeter.accumulate()` 后 `succeed(...)`。 + +### `packages/stage-ui/src/libs/providers/providers/official/shared.ts` + +- `withCredentials()` 除 `Authorization` 外，会在 Pinia 已初始化且有 active chat session 时注入 `x-airi-session-id`。 +- stage-web / stage-tamagotchi / stage-pocket 复用 official provider 的请求都会带同一个会话 id；匿名或非 chat 上下文没有 active session 时自动退化为 user-only trace。 + +### `apps/server/src/libs/env.ts` + +未改 —— LANGFUSE_* 故意不进 valibot schema(见「部署形态与脱敏决策」)。 + +## 验证 + +见 [`verifications/langfuse-tracing.md`](./verifications/langfuse-tracing.md)。Langfuse Cloud(project airi)回读到 `chat.completion` GENERATION,完整带 input(messages)/ output / model / usageDetails / userId / sessionId / metadata,trace 与 generation 共享 traceId。当前代码路径的 typecheck、targeted Vitest、eslint 通过。 + +## 待办(后续阶段) + +- **staging 真实端到端**:`.env.local` 的 DB/Redis/OTLP 全指生产,本地起真实 server 会连生产。在 staging(指向非生产)起 server 发一次真实 chat 请求补全 HTTP 路径端到端。 +- **model 定价匹配**:网关 model 是解析后的路由名,能否命中 Langfuse 定价表算 USD 待实测;命不中就配自定义定价或传 `costDetails`。flux 已放 metadata。 diff --git a/apps/server/docs/ai-context/metrics-ownership.md b/apps/server/docs/ai-context/metrics-ownership.md index 3e9758ebe..17a77d8d6 100644 --- a/apps/server/docs/ai-context/metrics-ownership.md +++ b/apps/server/docs/ai-context/metrics-ownership.md @@ -11,7 +11,7 @@ | **System / API observability** | Grafana Cloud + Prometheus + OTel | 系统健康、延迟、错误率、SRE on-call 告警 | | **Product analytics** | PostHog Cloud | 用户行为、漏斗、retention、cohort、A/B、feature adoption | | **Financial truth source** | Postgres (`flux_transaction` / Stripe webhook 持久化) | 收入与扣费 ledger，任何展示都视作近似 | -| **LLM-native observability**（预留） | Langfuse / Helicone（未接入） | token cost、prompt eval、provider trace — 后续按需引入 | +| **LLM-native observability** | Langfuse Cloud（已接入：chat completion + TTS speech） | 逐条 prompt/completion trace、TTS text trace、token/字符用量、按 user/session 成本归因、eval。真实 staging HTTP E2E 与 model 定价匹配待做 | **业界没有权威的判定 framework**（参见下方"参考来源"），这份文档落实成项目内的可执行规则。 @@ -47,7 +47,8 @@ | HTTP / WS / DB / Stripe webhook **计数** | **Grafana**（OTel counter） | 系统事件，PostHog 看不到 | | 用户去重 DAU / WAU / retention | **PostHog** | 需要 distinctId 去重，session table 计数不准 | | 收入展示（MRR / ARR / churn revenue） | **Postgres → 两边展示** | 真相在 Postgres，Grafana 取系统侧切片（panel-30），PostHog 取用户维度切片 | -| LLM token / cost | **Grafana**（短期） | 后续若引入 Langfuse 则迁过去 | +| LLM token / cost（聚合速率、按模型） | **Grafana**（OTel counter） | 系统侧聚合，SRE 视角 | +| LLM 逐条 prompt / completion / TTS text / eval / 按 user-session 成本 | **Langfuse** | 已接入 chat completion + TTS speech，正文级 trace + eval | | 用户行为漏斗各步骤 | **PostHog**（必须） | 第一步通常是前端事件，Grafana 拿不到 | ### Better Auth session table 与活跃用户 diff --git a/apps/server/docs/ai-context/verifications/langfuse-tracing.md b/apps/server/docs/ai-context/verifications/langfuse-tracing.md new file mode 100644 index 000000000..9bd32a35a --- /dev/null +++ b/apps/server/docs/ai-context/verifications/langfuse-tracing.md @@ -0,0 +1,120 @@ +# Verification: Langfuse LLM-native tracing + +## 场景 + +用户路径:客户端发 `POST /api/v1/openai/chat/completions` → 网关 `handleCompletion` 在处理时创建 Langfuse generation(input messages / output / model / token usage / userId / sessionId)→ 数据到达 Langfuse Cloud,可在逐条 prompt trace、eval、按用户/会话成本归因里查询。 + +## 为什么不用真实 server 起 + +`apps/server/.env.local` 的 `DATABASE_URL`(Neon)、`REDIS_URL`(Upstash)、`OTEL_EXPORTER_OTLP_ENDPOINT`(Grafana)全部指向**生产**实例。本地 `pnpm dev` 会连生产库并把 OTLP span 打到生产 Grafana,污染线上可观测性数据,也有触发真实计费的风险。因此用隔离 smoke 脚本复刻 `instrumentation.ts` + `handleCompletion` 的完全相同 wiring(独立 `NodeTracerProvider` + `LangfuseSpanProcessor` + `shouldExportSpan` langfuse.* 过滤 + `setLangfuseTracerProvider` + `startObservation(asType:'generation')` + `langfuse.user.id`/`langfuse.session.id` 属性),只验证 Langfuse 导出链路,不碰生产 DB/Redis/Grafana。生产路径的 generation 代码与 smoke 同形,typecheck 保证编译一致。 + +## 命令 + +```sh +# 1. 凭据有效性 +curl -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" https://us.cloud.langfuse.com/api/public/projects +# 2. smoke 脚本复刻 wiring,发一条 chat.completion generation 后 forceFlush + shutdown +pnpm exec dotenvx run -f .env.local -- tsx # 临时脚本,验证后已删 +# 3. 回读 Langfuse Cloud +curl -u "$PK:$SK" https://us.cloud.langfuse.com/api/public/traces?limit=20 +curl -u "$PK:$SK" https://us.cloud.langfuse.com/api/public/observations?limit=20 +``` + +类型 / 静态检查: + +```sh +pnpm -F @proj-airi/server typecheck # tsc --noEmit,0 错误 +pnpm exec eslint apps/server/instrumentation.ts apps/server/src/routes/openai/v1/index.ts # 0 warning/error +``` + +## 预期 + +- 凭据 API 返回 200。 +- smoke 无报错,forceFlush + shutdown 成功。 +- 回读 traces 出现 `name=chat.completion`,带 `userId` / `sessionId`(证明 `langfuse.user.id`/`langfuse.session.id` 提升为 trace 级归因)。 +- 回读 observations 出现 `type=GENERATION`,带 `model` / `input`(messages 数组)/ `output` / `usageDetails`。 + +## 实际 + +- typecheck 0 错误;eslint 0 输出。 +- 凭据:`status=200`,project `name=Airi` id `cmajdtoua06h2ad07yp1qf1nk`。 +- 直接 ingestion API POST(文档化 batch 格式)返回 `status=207`,两 event 均 `201 created` —— 写入端点 + 凭据 + 格式全部正确。 +- 回读(摄取 lag 约 90s 后,新项目首批较慢): + +``` +TRACE name=chat.completion user=smoke-user session=smoke-session (× 多条 smoke run) +TRACE name=direct-ingest-test user=direct-user +OBS_COUNT=20 +OBS name=chat.completion type=GENERATION model=smoke-test-model + in=[{"role":"user","content":"ping from airi langfuse smoke (...)"}] + out="pong" + usage={"input":5,"output":1} +``` + +input messages / output / model / usageDetails / userId / sessionId 全部落到 Langfuse Cloud。`shouldExportSpan` langfuse.* 过滤未拒绝 generation span(smoke 用的就是该过滤,trace 正常出现)。 + +## 已知限制 + +- 未经真实 `handleCompletion` HTTP 请求验证(需生产隔离环境 + auth token + 配置好的 LLM_ROUTER_CONFIG 模型)。smoke 复刻同一 SDK 调用形态 + typecheck 编译一致 + lint 通过,作为当前可得的最强 fresh evidence。下一次在 staging(DB/Redis/Grafana 指向非生产)起真实 server 发一次 chat 请求即可补全端到端。 + +## codex review 后复测 + +codex 独立 review 报 8 项,修了 4 项(详见下方代码改动)。修复后复测: + +- typecheck `tsc --noEmit` 0 错误;eslint(instrumentation.ts + openai/v1/index.ts)0 输出。 +- SSE 解析纯逻辑单测(`extractSseDeltaText` + chunk 边界组装)4 case 全 PASS:简单多 delta、内容跨 chunk 断行、usage-only/空行忽略、malformed line 降级。 +- 带 `AlwaysOnSampler`(F4 修复)的 live smoke 再次回读成功:Langfuse Cloud observation `model=verify-model`,`input=[{role:user,content:...}]`,`output="Hello"`,`usageDetails={input:5,output:2,total:6}` —— 确认 provider sampler 改动没破坏导出。 + +修复项: +- F4(sampler,真 bug):generation 的 parent 是 `@hono/otel` HTTP span,默认 ParentBased sampler 会让 `OTEL_TRACES_SAMPLING_RATIO<1` 时连带丢 Langfuse generation。改 langfuseProvider 显式 `AlwaysOnSampler`,Langfuse 捕获与 Grafana head-sampling 解耦。 +- F6(非流式 generation 泄漏,真 bug):`response.json()` 解析失败时 span + generation 都不 end。加 try/catch 在抛出前关闭两者。 +- F7(流式 output 是原始 SSE,体验):改 `extractSseDeltaText` 逐行解析出 assistant 正文,不再存 `data:` 框架。 +- F9(流式 fullText 内存,真 bug):加 1M 上界；后续复审发现单个超大 delta 仍会越界,已改成按剩余容量 slice 的硬上限。 +- F5(gate)：codex 判 non-issue，仍主动改成 instrumentation 置的 `LANGFUSE_TRACING_ACTIVE` sentinel(单一真相,防 enable 条件 desync 漏 PII)。 +- 拒绝/记录:F1(隔离)/F3(shutdown)/F5/F7-gate codex 判 non-issue,确认;F2(traceId 关联)经核对实为「继承 Hono span traceId,OTLP 开启时可关联」,已修正注释与文档(此前误述为不关联);F10(input base64 无 cap)记为已知限制,低优先。 + +## 抽象重构后复测(llm-tracing 深模块) + +把 Langfuse 逻辑从 transport 层 `openai/v1/index.ts` 抽到 `services/domain/llm-tracing/index.ts`(gate / SDK / SSE 解析 / 生命周期全部隐藏,route 只调 `startChatGeneration` → `appendStreamChunk` / `succeed` / `fail`)。复测: + +- `pnpm -F @proj-airi/server typecheck`:0 错误。 +- `pnpm exec vitest run .../llm-tracing/index.test.ts`:**Tests 9 passed (9)** —— disabled no-op、创建参数、session 有无、非流式 output、流式跨 chunk 组装、malformed SSE 忽略、fail ERROR、幂等 end。纯逻辑单测,按 Iron Law 即该模块的 fresh evidence。 +- `pnpm exec eslint`(instrumentation + route + 模块 + 测试):0 输出。 +- 端到端行为不变:route 改的只是调用形态,generation 字段映射与之前 smoke 回读到的一致(input/output/model/usageDetails/userId/sessionId)。 + +## 收尾复测(TTS + client session + hard cap) + +补齐: + +- `startTtsGeneration` + `/api/v1/audio/speech` route:记录 `tts.speech` generation,不缓冲二进制 audio,只记录 input text/voice/speed/format、contentType、input char usage、flux metadata。 +- `packages/stage-ui/src/libs/providers/providers/official/shared.ts`:official provider fetch 自动带 `x-airi-session-id`(Pinia active chat session 存在时)。 +- 流式 output hard cap:单个超大 SSE delta 也只追加剩余容量。 + +复测: + +- `pnpm -F @proj-airi/server typecheck`:0 错误。 +- `pnpm exec vitest run apps/server/src/services/domain/llm-tracing/index.test.ts apps/server/src/services/domain/llm-router/tests/router.test.ts apps/server/src/routes/openai/v1/route.test.ts`:3 files / 77 tests passed。 +- `pnpm exec eslint apps/server/instrumentation.ts apps/server/src/routes/openai/v1/index.ts apps/server/src/services/domain/llm-tracing/index.ts apps/server/src/services/domain/llm-tracing/index.test.ts apps/server/src/services/domain/llm-router/router.ts apps/server/src/services/domain/llm-router/tests/router.test.ts packages/stage-ui/src/libs/providers/providers/official/shared.ts`:0 输出。 +- `pnpm -F @proj-airi/stage-ui typecheck`:0 错误。 + +Langfuse 隔离 live smoke(覆盖 `NODE_ENV=codex-langfuse-smoke`, `OTEL_SERVICE_NAME=server-codex-langfuse-smoke`, `SERVER_INSTANCE_ID=codex-langfuse-smoke`, 且清空 `OTEL_EXPORTER_OTLP_ENDPOINT`/`OTEL_EXPORTER_OTLP_HEADERS`)已补跑,只复用 `.env.local` 的 Langfuse keys,不连 DB/Redis/Grafana OTLP: + +- 写入命令:`pnpm exec dotenvx run -f .env.local --ignore=MISSING_ENV_FILE -- tsx --import ./instrumentation.ts scripts/langfuse-smoke.ts` +- 启动日志确认:`OpenTelemetry initialized — OTLP: off, Langfuse: https://us.cloud.langfuse.com` +- run id:`codex-langfuse-1780140569403` +- 回读 traces: + - `tts.speech`, `userId=codex-langfuse-smoke-user`, `sessionId=codex-langfuse-1780140569403-session`, `metadata.requestId=codex-langfuse-1780140569403-tts` + - `chat.completion`, `userId=codex-langfuse-smoke-user`, `sessionId=codex-langfuse-1780140569403-session`, `metadata.requestId=codex-langfuse-1780140569403-chat` +- 回读 observations: + - `tts.speech`, `type=GENERATION`, `model=codex-smoke-tts-model`, `usageDetails={input:38,total:38}`, `output.contentType=audio/mpeg` + - `chat.completion`, `type=GENERATION`, `model=codex-smoke-chat-model`, `usageDetails={input:4,output:5,total:9}`, `output="hello from chat smoke"` +- 两条回读记录的 `resourceAttributes` 均为 `service.name=server-codex-langfuse-smoke`, `service.namespace=airi`, `service.instance.id=codex-langfuse-smoke`, `deployment.environment=codex-langfuse-smoke`。 + +仍未做真实 server HTTP E2E:本地 `.env.local` 里的 DB/Redis 仍指向生产实例。当前已验证的是同一 `instrumentation.ts` + `llm-tracing` generation SDK 写入链路;真实 HTTP 请求还需要 staging DB/Redis/router/auth token 后补跑。 + +## 环境 + +- base commit: `dc1037f34`(本次改动未提交,工作树状态) +- Langfuse: us.cloud.langfuse.com,project Airi +- SDK: `@langfuse/tracing` + `@langfuse/otel` 5.4.0,`@opentelemetry/api` 1.9.1 +- 最后验证日期:2026-05-30 diff --git a/apps/server/instrumentation.ts b/apps/server/instrumentation.ts index ac57a7df7..78025937e 100644 --- a/apps/server/instrumentation.ts +++ b/apps/server/instrumentation.ts @@ -26,6 +26,8 @@ import process, { env, exit } from 'node:process' import { randomUUID } from 'node:crypto' +import { LangfuseSpanProcessor } from '@langfuse/otel' +import { setLangfuseTracerProvider } from '@langfuse/tracing' import { diag, DiagConsoleLogger, DiagLogLevel } from '@opentelemetry/api' import { OTLPLogExporter } from '@opentelemetry/exporter-logs-otlp-proto' import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-proto' @@ -39,7 +41,7 @@ import { resourceFromAttributes } from '@opentelemetry/resources' import { BatchLogRecordProcessor } from '@opentelemetry/sdk-logs' import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics' import { NodeSDK } from '@opentelemetry/sdk-node' -import { BatchSpanProcessor, ParentBasedSampler, TraceIdRatioBasedSampler } from '@opentelemetry/sdk-trace-node' +import { AlwaysOnSampler, BatchSpanProcessor, NodeTracerProvider, ParentBasedSampler, TraceIdRatioBasedSampler } from '@opentelemetry/sdk-trace-node' import { ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION } from '@opentelemetry/semantic-conventions' // NOTICE: @@ -60,8 +62,15 @@ if (!env.OTEL_SEMCONV_STABILITY_OPT_IN) console.info(`[otel-preload] OTEL_SEMCONV_STABILITY_OPT_IN=${env.OTEL_SEMCONV_STABILITY_OPT_IN}`) const otlpEndpoint = env.OTEL_EXPORTER_OTLP_ENDPOINT -if (!otlpEndpoint) { - console.info('[otel-preload] OpenTelemetry disabled (set OTEL_EXPORTER_OTLP_ENDPOINT to enable)') +// Langfuse LLM-native tracing is an INDEPENDENT exporter from the OTLP +// (Grafana) pipeline. Gate the two separately so setting only Langfuse keys +// does not silently no-op just because OTEL_EXPORTER_OTLP_ENDPOINT is unset +// (and vice versa). Both ride the same NodeSDK tracer provider; Langfuse adds +// a second SpanProcessor that exports only its own observation spans (see +// shouldExportSpan below), so the OTLP/Grafana trace stream is untouched. +const langfuseEnabled = !!env.LANGFUSE_PUBLIC_KEY && !!env.LANGFUSE_SECRET_KEY +if (!otlpEndpoint && !langfuseEnabled) { + console.info('[otel-preload] OpenTelemetry disabled (set OTEL_EXPORTER_OTLP_ENDPOINT and/or LANGFUSE_PUBLIC_KEY+LANGFUSE_SECRET_KEY to enable)') } else { if (env.OTEL_DEBUG === 'true') @@ -115,27 +124,92 @@ else { 'deployment.environment': env.NODE_ENV || 'development', }) + // Traces fan out to independent processors. OTLP (Grafana Tempo) sees every + // sampled span; Langfuse sees ONLY the LLM observation spans created via + // @langfuse/tracing. Each processor is added only when its backend is + // configured, so enabling one without the other is safe. + const spanProcessors = [] + if (otlpEndpoint) { + spanProcessors.push(new BatchSpanProcessor(new OTLPTraceExporter({ + url: `${otlpEndpoint}/v1/traces`, + headers, + }))) + } + // Langfuse runs on its OWN TracerProvider, isolated from the global NodeSDK + // provider above. Why isolation instead of a second SpanProcessor on the + // shared provider: the OTLP BatchSpanProcessor has no per-span filter, so a + // shared provider would also ship every Langfuse generation span — prompt and + // completion text included — to Grafana Tempo. An isolated provider keeps the + // LLM observation spans (and their text) exclusively on the Langfuse export + // path. + // + // sampler: AlwaysOnSampler is REQUIRED, not cosmetic. The gateway handler runs + // inside the @hono/otel HTTP server span's context, so startObservation picks + // that span up as the generation's parent (OTel context is global, shared + // across providers). A NodeTracerProvider with no explicit sampler defaults to + // ParentBased, which would inherit that parent's sampling decision — so + // lowering OTEL_TRACES_SAMPLING_RATIO for Grafana would silently drop Langfuse + // generations whenever the parent HTTP span is sampled out. AlwaysOnSampler + // decouples Langfuse capture from Grafana's head-sampling: every generation is + // recorded and exported regardless of the parent decision. Trace id is still + // inherited from the active parent, so when OTLP is on the generation shares + // the request's trace id and stays correlatable with the Grafana trace. + let langfuseProvider: NodeTracerProvider | null = null + if (langfuseEnabled) { + langfuseProvider = new NodeTracerProvider({ + resource, + sampler: new AlwaysOnSampler(), + spanProcessors: [new LangfuseSpanProcessor({ + publicKey: env.LANGFUSE_PUBLIC_KEY, + secretKey: env.LANGFUSE_SECRET_KEY, + baseUrl: env.LANGFUSE_BASE_URL, + // Railway is long-running → batched. Flushed via + // langfuseProvider.shutdown() in the SIGTERM handler below. + exportMode: 'batched', + // Full override of Langfuse's default filter. Export ONLY spans the + // @langfuse/tracing SDK created (they carry `langfuse.*` attributes + // such as `langfuse.observation.type`). This provider should only ever + // see those anyway; the predicate guards against stray + // context-propagated spans landing in Langfuse Cloud. + shouldExportSpan: ({ otelSpan }) => + Object.keys(otelSpan.attributes).some(key => key.startsWith('langfuse.')), + })], + }) + setLangfuseTracerProvider(langfuseProvider) + // Single source of truth for "the isolated Langfuse provider is actually + // wired". The chat route gates generation creation on THIS, not on a second + // independent key check — if setLangfuseTracerProvider was never called, + // startObservation would fall back to the GLOBAL provider and leak prompt + // text to the OTLP/Grafana exporter. Setting the sentinel only here ties the + // route's gate to the real provider state. + env.LANGFUSE_TRACING_ACTIVE = '1' + } + const sdk = new NodeSDK({ resource, sampler: new ParentBasedSampler({ root: new TraceIdRatioBasedSampler(samplingRatio), }), - spanProcessors: [new BatchSpanProcessor(new OTLPTraceExporter({ - url: `${otlpEndpoint}/v1/traces`, - headers, - }))], - metricReaders: [new PeriodicExportingMetricReader({ - exporter: new OTLPMetricExporter({ - url: `${otlpEndpoint}/v1/metrics`, - headers, - }), - exportIntervalMillis: 15_000, - exportTimeoutMillis: 10_000, - })], - logRecordProcessors: [new BatchLogRecordProcessor(new OTLPLogExporter({ - url: `${otlpEndpoint}/v1/logs`, - headers, - }))], + spanProcessors, + // Metrics and logs are OTLP-only (Langfuse is traces-only). Omit the + // readers entirely when OTLP is off so NodeSDK doesn't build exporters + // pointed at an empty URL. + ...(otlpEndpoint + ? { + metricReaders: [new PeriodicExportingMetricReader({ + exporter: new OTLPMetricExporter({ + url: `${otlpEndpoint}/v1/metrics`, + headers, + }), + exportIntervalMillis: 15_000, + exportTimeoutMillis: 10_000, + })], + logRecordProcessors: [new BatchLogRecordProcessor(new OTLPLogExporter({ + url: `${otlpEndpoint}/v1/logs`, + headers, + }))], + } + : {}), instrumentations: [ // Inbound HTTP is instrumented by @hono/otel inside the Hono pipeline // (it sees Hono's matched route pattern; auto-instrumentation can't). @@ -162,7 +236,7 @@ else { }) sdk.start() - console.info(`[otel-preload] OpenTelemetry initialized, exporting to ${otlpEndpoint}, sampling ratio: ${samplingRatio}`) + console.info(`[otel-preload] OpenTelemetry initialized — OTLP: ${otlpEndpoint || 'off'}, Langfuse: ${langfuseEnabled ? (env.LANGFUSE_BASE_URL || 'https://cloud.langfuse.com') : 'off'}, sampling ratio: ${samplingRatio}`) // Graceful shutdown — flush pending exports before exit. Idempotent. let shuttingDown = false @@ -171,7 +245,13 @@ else { return shuttingDown = true try { - await sdk.shutdown() + // Shut down both providers. The Langfuse provider is separate from + // NodeSDK, so sdk.shutdown() does NOT drain it — flush it explicitly or + // the last batch of generations is lost on deploy/restart. + await Promise.all([ + sdk.shutdown(), + langfuseProvider?.shutdown(), + ]) console.info('[otel-preload] OpenTelemetry shut down successfully') } catch (err) { diff --git a/apps/server/otel/grafana/dashboards/airi-server-overview-cloud.json b/apps/server/otel/grafana/dashboards/airi-server-overview-cloud.json index aa7a57851..ed23f2a5a 100644 --- a/apps/server/otel/grafana/dashboards/airi-server-overview-cloud.json +++ b/apps/server/otel/grafana/dashboards/airi-server-overview-cloud.json @@ -1548,7 +1548,7 @@ } } }, - "panel-72": { + "panel-66": { "kind": "Panel", "spec": { "data": { @@ -1566,8 +1566,8 @@ "group": "prometheus", "kind": "DataQuery", "spec": { - "expr": "sum by (gen_ai_request_model) (rate(airi_billing_flux_consumed_total{service_name=~\"$service\", deployment_environment=~\"$env\", gen_ai_request_model!=\"\"}[$__rate_interval]))", - "legendFormat": "{{gen_ai_request_model}}" + "expr": "sum by (provider) (rate(gen_ai_client_operation_count_total{service_name=~\"$service\", deployment_environment=~\"$env\", provider!=\"\"}[$__rate_interval]))", + "legendFormat": "{{provider}}" }, "version": "v0" }, @@ -1579,10 +1579,358 @@ "transformations": [] } }, - "description": "Normal flux debited per LLM request, by model (flux/sec). The billed-usage counterpart to panel-43 (⚠ Flux Unbilled): together they show what was charged vs what leaked. A model trending up here without matching request-rate growth means per-call cost rose.", - "id": 72, + "description": "Outbound request rate to each upstream provider (chat + tts), as our gateway sees it. The RPM / 调用次数 screens on the provider consoles, unified. provider = upstream hostname the router used.", + "id": 66, "links": [], - "title": "LLM Flux Consumed by Model", + "title": "Requests/s by Provider", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "smooth", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "reqps" + }, + "overrides": [] + }, + "options": { + "annotations": { + "clustering": -1, + "multiLane": false + }, + "legend": { + "calcs": [ + "lastNotNull", + "max" + ], + "displayMode": "table", + "placement": "right", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + } + }, + "version": "13.0.0-23630096546" + } + } + }, + "panel-67": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "grafanacloud-projairi-prom" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "expr": "histogram_quantile(0.95, sum by (le, provider) (rate(gen_ai_client_operation_duration_seconds_bucket{service_name=~\"$service\", deployment_environment=~\"$env\", provider!=\"\"}[$__rate_interval])))", + "legendFormat": "{{provider}}" + }, + "version": "v0" + }, + "refId": "A" + } + } + ], + "queryOptions": {}, + "transformations": [] + } + }, + "description": "P95 upstream call duration per provider (chat + tts), across models. Mirrors each provider console's 调用时长 p95/p99 panel — but here every provider is on one axis.", + "id": 67, + "links": [], + "title": "Provider Latency P95", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "smooth", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "s" + }, + "overrides": [] + }, + "options": { + "annotations": { + "clustering": -1, + "multiLane": false + }, + "legend": { + "calcs": [ + "lastNotNull", + "max" + ], + "displayMode": "table", + "placement": "right", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + } + }, + "version": "13.0.0-23630096546" + } + } + }, + "panel-68": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "grafanacloud-projairi-prom" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "expr": "100 * sum by (provider) (rate(gen_ai_client_operation_count_total{service_name=~\"$service\", deployment_environment=~\"$env\", provider!=\"\", http_response_status_code=~\"4..|5..\"}[$__rate_interval])) / clamp_min(sum by (provider) (rate(gen_ai_client_operation_count_total{service_name=~\"$service\", deployment_environment=~\"$env\", provider!=\"\"}[$__rate_interval])), 1)", + "legendFormat": "{{provider}}" + }, + "version": "v0" + }, + "refId": "A" + } + } + ], + "queryOptions": {}, + "transformations": [] + } + }, + "description": "4xx + 5xx ÷ all requests per provider, our side of the call. Matches each provider 失败率 panel. Pair with Upstream Errors by Status Code (LLM Router Health) to see which codes drive it.", + "id": 68, + "links": [], + "title": "Provider Failure %", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "smooth", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "percent" + }, + "overrides": [] + }, + "options": { + "annotations": { + "clustering": -1, + "multiLane": false + }, + "legend": { + "calcs": [ + "lastNotNull", + "max" + ], + "displayMode": "table", + "placement": "right", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + } + }, + "version": "13.0.0-23630096546" + } + } + }, + "panel-69": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "grafanacloud-projairi-prom" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "expr": "sum by (model) (rate(airi_billing_tts_chars_total{service_name=~\"$service\", deployment_environment=~\"$env\"}[$__rate_interval]))", + "legendFormat": "{{model}}" + }, + "version": "v0" + }, + "refId": "A" + } + } + ], + "queryOptions": {}, + "transformations": [] + } + }, + "description": "Billed TTS characters per second by model (from `airi.billing.tts.chars`). The 用量统计「字数」screen on the TTS consoles (豆包 / 阿里), unified. Integrate over the range for a window total.", + "id": 69, + "links": [], + "title": "TTS Characters/s by Model", "vizConfig": { "group": "timeseries", "kind": "VizConfig", @@ -3507,10 +3855,10 @@ "spec": { "element": { "kind": "ElementReference", - "name": "panel-72" + "name": "panel-11" }, "height": 8, - "width": 13, + "width": 12, "x": 0, "y": 0 } @@ -3523,8 +3871,35 @@ "name": "panel-21" }, "height": 8, - "width": 11, - "x": 13, + "width": 12, + "x": 12, + "y": 0 + } + } + ] + } + }, + "title": "LLM Gateway" + } + }, + { + "kind": "RowsLayoutRow", + "spec": { + "collapse": false, + "layout": { + "kind": "GridLayout", + "spec": { + "items": [ + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-66" + }, + "height": 7, + "width": 6, + "x": 0, "y": 0 } }, @@ -3533,18 +3908,44 @@ "spec": { "element": { "kind": "ElementReference", - "name": "panel-11" + "name": "panel-67" }, - "height": 8, - "width": 24, - "x": 0, - "y": 8 + "height": 7, + "width": 6, + "x": 6, + "y": 0 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-68" + }, + "height": 7, + "width": 6, + "x": 12, + "y": 0 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-69" + }, + "height": 7, + "width": 6, + "x": 18, + "y": 0 } } ] } }, - "title": "LLM Gateway" + "title": "Provider Upstreams" } }, { @@ -3850,7 +4251,7 @@ "grafana-cloud" ], "timeSettings": { - "autoRefresh": "", + "autoRefresh": "30s", "autoRefreshIntervals": [ "5s", "10s", diff --git a/apps/server/otel/grafana/dashboards/build.ts b/apps/server/otel/grafana/dashboards/build.ts index eb9e8c9e3..e4169c13b 100644 --- a/apps/server/otel/grafana/dashboards/build.ts +++ b/apps/server/otel/grafana/dashboards/build.ts @@ -571,7 +571,7 @@ elements['panel-94'] = timeseriesPanel( { unit: 'short' }, ) -// --- Row 3: LLM Gateway — request mix, latency, billed usage --------------- +// --- Row 3: LLM Gateway — request mix + latency ---------------------------- elements['panel-11'] = timeseriesPanel( 11, 'LLM Request Rate by Model', @@ -594,13 +594,53 @@ elements['panel-21'] = timeseriesPanel( { unit: 's' }, ) -elements['panel-72'] = timeseriesPanel( - 72, - 'LLM Flux Consumed by Model', - 'Normal flux debited per LLM request, by model (flux/sec). The billed-usage counterpart to panel-43 (⚠ Flux Unbilled): together they show what was charged vs what leaked. A model trending up here without matching request-rate growth means per-call cost rose.', +// --- Row: Provider Upstreams — our gateway's view of each upstream so the +// per-provider consoles (OpenRouter / Volcengine 豆包 / DashScope 阿里) don't +// have to be checked one by one. `provider` is the upstream the router +// actually used (winning upstream on success, last-tried on exhaustion); +// it's the URL hostname, so legends read e.g. `openrouter.ai`, +// `dashscope.aliyuncs.com`. Note: provider-only truths (real $ spend, account +// quota / balance) are NOT here — those need the provider billing APIs. +elements['panel-66'] = timeseriesPanel( + 66, + 'Requests/s by Provider', + 'Outbound request rate to each upstream provider (chat + tts), as our gateway sees it. The RPM / 调用次数 screens on the provider consoles, unified. provider = upstream hostname the router used.', [query( - `sum by (gen_ai_request_model) (rate(airi_billing_flux_consumed_total{${SERVICE_FILTER}, gen_ai_request_model!=""}[$__rate_interval]))`, - '{{gen_ai_request_model}}', + `sum by (provider) (rate(gen_ai_client_operation_count_total{${SERVICE_FILTER}, provider!=""}[$__rate_interval]))`, + '{{provider}}', + )], + { unit: 'reqps' }, +) + +elements['panel-67'] = timeseriesPanel( + 67, + 'Provider Latency P95', + 'P95 upstream call duration per provider (chat + tts), across models. Mirrors each provider console\'s 调用时长 p95/p99 panel — but here every provider is on one axis.', + [query( + `histogram_quantile(0.95, sum by (le, provider) (rate(gen_ai_client_operation_duration_seconds_bucket{${SERVICE_FILTER}, provider!=""}[$__rate_interval])))`, + '{{provider}}', + )], + { unit: 's' }, +) + +elements['panel-68'] = timeseriesPanel( + 68, + 'Provider Failure %', + '4xx + 5xx ÷ all requests per provider, our side of the call. Matches each provider 失败率 panel. Pair with Upstream Errors by Status Code (LLM Router Health) to see which codes drive it.', + [query( + `100 * sum by (provider) (rate(gen_ai_client_operation_count_total{${SERVICE_FILTER}, provider!="", http_response_status_code=~"4..|5.."}[$__rate_interval])) / clamp_min(sum by (provider) (rate(gen_ai_client_operation_count_total{${SERVICE_FILTER}, provider!=""}[$__rate_interval])), 1)`, + '{{provider}}', + )], + { unit: 'percent' }, +) + +elements['panel-69'] = timeseriesPanel( + 69, + 'TTS Characters/s by Model', + 'Billed TTS characters per second by model (from `airi.billing.tts.chars`). The 用量统计「字数」screen on the TTS consoles (豆包 / 阿里), unified. Integrate over the range for a window total.', + [query( + `sum by (model) (rate(airi_billing_tts_chars_total{${SERVICE_FILTER}}[$__rate_interval]))`, + '{{model}}', )], { unit: 'short' }, ) @@ -821,11 +861,18 @@ const rows = [ item('panel-16', 0, 8, 7, 11), item('panel-20', 7, 8, 17, 11), ]), - // Row 4: LLM gateway — billed flux + latency side by side, request mix below. + // Row 4: LLM gateway — request mix + latency side by side. row('LLM Gateway', [ - item('panel-72', 0, 0, 13, 8), - item('panel-21', 13, 0, 11, 8), - item('panel-11', 0, 8, 24, 8), + item('panel-11', 0, 0, 12, 8), + item('panel-21', 12, 0, 12, 8), + ]), + // Row 5: Provider Upstreams — per-provider rollup so the vendor consoles + // don't have to be opened one by one. Four wide, one screen line. + row('Provider Upstreams', [ + item('panel-66', 0, 0, 6, 7), + item('panel-67', 6, 0, 6, 7), + item('panel-68', 12, 0, 6, 7), + item('panel-69', 18, 0, 6, 7), ]), // Row 4: token totals + throughput + the two revenue/quality alert stats. row('LLM Tokens & Quality', [ @@ -930,12 +977,13 @@ const variables = [ * heatmap, live WS trend: "is anything broken right now?" * 2. User Engagement — rolling DAU/WAU/MAU from user.last_seen_at * 3. HTTP — error breakdown by route, request ranking, latency by route - * 4. LLM Gateway — billed flux, latency (TTFB + end-to-end), request mix - * 5. LLM Tokens & Quality — token totals/throughput, revenue-leak alerts - * 6. LLM Router Health — key/decrypt/fallback "wake someone up" signals - * 7. Business — Stripe / Flux money flow - * 8. Infrastructure (collapsed) — DB / runtime health for triage - * 9. Logs — Loki for live debugging + * 4. LLM Gateway — per-model request rate + latency (TTFB + end-to-end) + * 5. Provider Upstreams — per-provider rate/latency/failure + TTS chars + * 6. LLM Tokens & Quality — token totals/throughput, revenue-leak alerts + * 7. LLM Router Health — key/decrypt/fallback "wake someone up" signals + * 8. Business — Stripe / Flux money flow + * 9. Infrastructure (collapsed) — DB / runtime health for triage + * 10. Logs — Loki for live debugging * * One metric, one panel: we deliberately do not duplicate a metric across * stat/trend/bar/pie forms. Counter conventions: rate() for "now" trends, @@ -973,7 +1021,7 @@ const dashboard = { preload: false, tags: ['airi', 'observability', 'grafana-cloud'], timeSettings: { - autoRefresh: '', + autoRefresh: '30s', autoRefreshIntervals: ['5s', '10s', '30s', '1m', '5m', '15m', '30m', '1h', '2h', '1d'], fiscalYearStartMonth: 0, from: 'now-1h', diff --git a/apps/server/package.json b/apps/server/package.json index 52d17b423..093cb112d 100644 --- a/apps/server/package.json +++ b/apps/server/package.json @@ -24,6 +24,8 @@ "@hono/node-server": "^1.19.14", "@hono/node-ws": "catalog:", "@hono/otel": "^1.1.2", + "@langfuse/otel": "catalog:", + "@langfuse/tracing": "catalog:", "@moeru/eventa": "catalog:", "@moeru/std": "catalog:", "@opentelemetry/api": "^1.9.1", diff --git a/apps/server/src/routes/openai/v1/index.ts b/apps/server/src/routes/openai/v1/index.ts index 144c1fd6d..bfc04c745 100644 --- a/apps/server/src/routes/openai/v1/index.ts +++ b/apps/server/src/routes/openai/v1/index.ts @@ -7,7 +7,7 @@ import type { UsageInfo } from '../../../services/domain/billing/billing' import type { BillingService } from '../../../services/domain/billing/billing-service' import type { FluxMeter } from '../../../services/domain/billing/flux-meter' import type { FluxService } from '../../../services/domain/flux' -import type { LlmRouterService } from '../../../services/domain/llm-router' +import type { LlmRouteContext, LlmRouterService } from '../../../services/domain/llm-router' import type { RequestLogService } from '../../../services/domain/request-log' import type { HonoEnv } from '../../../types/hono' @@ -20,6 +20,7 @@ import { configGuard } from '../../../middlewares/config-guard' import { rateLimiter } from '../../../middlewares/rate-limit' import { captureSafe } from '../../../services/adapters/posthog' import { calculateFluxFromUsage, extractUsageFromBody } from '../../../services/domain/billing/billing' +import { startChatGeneration, startTtsGeneration } from '../../../services/domain/llm-tracing' import { createBadGatewayError, createBadRequestError, createPaymentRequiredError, createServiceUnavailableError } from '../../../utils/error' import { nanoid } from '../../../utils/id' import { @@ -51,12 +52,18 @@ function buildSafeResponseHeaders(response: Response): Headers { return headers } -function getLlmMetricAttributes(opts: { model: string, type: string, status: number }): Record { +function getLlmMetricAttributes(opts: { model: string, type: string, status: number, provider: string }): Record { + // `provider` is the upstream the router actually used (winning upstream on + // success, last-tried on exhaustion), so per-provider rollups in Grafana + // line up with each vendor's own console. Same label name as the gateway + // error counters (`airi_gen_ai_gateway_upstream_errors{provider}`) so the + // two can be compared/joined. if (opts.type === 'chat') { return { [GEN_AI_ATTR_REQUEST_MODEL]: opts.model, [GEN_AI_ATTR_OPERATION_NAME]: 'chat', 'http.response.status_code': opts.status, + 'provider': opts.provider, } } @@ -64,9 +71,18 @@ function getLlmMetricAttributes(opts: { model: string, type: string, status: num [GEN_AI_ATTR_REQUEST_MODEL]: opts.model, [AIRI_ATTR_GEN_AI_OPERATION_KIND]: opts.type, 'http.response.status_code': opts.status, + 'provider': opts.provider, } } +// Fresh per-request context handed to `llmRouter.route` / `routeTts` so the +// router can report back which upstream it used (for the `provider` metric +// label). Must be created per request — never shared — because the route +// closures live at factory scope across concurrent requests. +function newRouteContext(): LlmRouteContext { + return { provider: 'unknown', triedUpstreams: 0, triedKeys: 0, lastStatus: null } +} + export function createV1Routes( fluxService: FluxService, billingService: BillingService, @@ -83,7 +99,7 @@ export function createV1Routes( // TODO: Extract this compat route into smaller facades/modules. // It currently mixes auth, rate limiting, proxying, billing, telemetry, and event publishing in one transport layer entrypoint. - function recordMetrics(opts: { model: string, status: number, type: string, durationMs: number, fluxConsumed: number, promptTokens?: number, completionTokens?: number }) { + function recordMetrics(opts: { model: string, status: number, type: string, provider: string, durationMs: number, fluxConsumed: number, promptTokens?: number, completionTokens?: number }) { if (!genAi) return const attrs = getLlmMetricAttributes(opts) @@ -162,6 +178,20 @@ export function createV1Routes( }, }) + // Langfuse LLM-native generation: per-request prompt/completion record + // (input/output/model/usage) powering prompt trace, eval, and per-user/ + // session cost. The llm-tracing module hides the enable gate, SDK shape, SSE + // assembly, and lifecycle — this is a no-op handle when tracing is off, and + // succeed/fail are idempotent, so every exit branch below can close it. + const generationTrace = startChatGeneration({ + input: body.messages, + model: requestModel, + requestId, + stream, + userId: user.id, + sessionId: c.req.header('x-airi-session-id'), + }) + const startedAt = Date.now() // Router throws ApiError (502/503/504/400) on full exhaustion or unknown @@ -174,15 +204,17 @@ export function createV1Routes( // fluxConsumed: 0 while real cost was incurred — a silent revenue leak. // Source: codex review 2026-05-15 HIGH #1. const clientAbort = c.req.raw.signal + const routeCtx = newRouteContext() let response: Response try { response = await context.with(trace.setSpan(context.active(), span), () => - llmRouter.route({ modelName: requestModel, body, headers: {}, abortSignal: clientAbort })) + llmRouter.route({ modelName: requestModel, body, headers: {}, abortSignal: clientAbort }, routeCtx)) } catch (err) { span.setStatus({ code: SpanStatusCode.ERROR, message: 'Router exhausted or unknown model' }) span.end() - recordMetrics({ model: requestModel, status: 502, type: 'chat', durationMs: Date.now() - startedAt, fluxConsumed: 0 }) + generationTrace.fail('Router exhausted or unknown model') + recordMetrics({ model: requestModel, status: 502, type: 'chat', provider: routeCtx.provider, durationMs: Date.now() - startedAt, fluxConsumed: 0 }) throw err } @@ -192,7 +224,8 @@ export function createV1Routes( if (!response.ok) { span.setStatus({ code: SpanStatusCode.ERROR, message: `Gateway ${response.status}` }) span.end() - recordMetrics({ model: requestModel, status: response.status, type: 'chat', durationMs, fluxConsumed: 0 }) + generationTrace.fail(`Gateway ${response.status}`) + recordMetrics({ model: requestModel, status: response.status, type: 'chat', provider: routeCtx.provider, durationMs, fluxConsumed: 0 }) // Emit server-side so funnels see real HTTP status — the client only // ever observes "stream closed" and cannot tell 401 / 429 / 5xx apart. void captureSafe(posthog ?? null, { @@ -249,11 +282,15 @@ export function createV1Routes( genAi?.firstTokenDuration.record((firstChunkAt - startedAt) / 1000, { [GEN_AI_ATTR_REQUEST_MODEL]: requestModel, [GEN_AI_ATTR_OPERATION_NAME]: 'chat', + provider: routeCtx.provider, }) } await writer.write(value) const text = decoder.decode(value, { stream: true }) tailBuffer = (tailBuffer + text).slice(-2048) + // Accumulate the assistant completion for the Langfuse trace output + // (no-op when tracing is off). Module owns SSE parsing + the cap. + generationTrace.appendStreamChunk(text) } } catch (err) { @@ -280,7 +317,8 @@ export function createV1Routes( finally { if (streamInterrupted) { span.end() - recordMetrics({ model: requestModel, status: response.status, type: 'chat', durationMs, fluxConsumed: 0 }) + generationTrace.fail('Gateway stream interrupted') + recordMetrics({ model: requestModel, status: response.status, type: 'chat', provider: routeCtx.provider, durationMs, fluxConsumed: 0 }) } else if (streamCompleted) { try { @@ -309,7 +347,14 @@ export function createV1Routes( [AIRI_ATTR_BILLING_FLUX_CONSUMED]: fluxConsumed, }) span.end() - recordMetrics({ model: requestModel, status: response.status, type: 'chat', durationMs, fluxConsumed, ...usage }) + // Streaming output comes from appendStreamChunk above, so succeed + // omits it and the module uses the assembled assistant text. + generationTrace.succeed({ + promptTokens: usage.promptTokens, + completionTokens: usage.completionTokens, + fluxConsumed, + }) + recordMetrics({ model: requestModel, status: response.status, type: 'chat', provider: routeCtx.provider, durationMs, fluxConsumed, ...usage }) // Debit flux via DB transaction (source of truth) // NOTICE: streaming response is already sent, so we cannot reject on failure. @@ -406,8 +451,21 @@ export function createV1Routes( }) } - // Non-streaming: parse response, bill, then return - const responseBody = await response.json() + // Non-streaming: parse response, bill, then return. + // Parse failure (malformed upstream JSON) must close both span and the + // Langfuse generation before bubbling up — otherwise the trace leaks. + // Mirrors the error-branch shape used above (router throw / !response.ok). + let responseBody + try { + responseBody = await response.json() + } + catch (err) { + span.setStatus({ code: SpanStatusCode.ERROR, message: 'Failed to parse upstream response body' }) + span.end() + generationTrace.fail('Failed to parse upstream response body') + recordMetrics({ model: requestModel, status: response.status, type: 'chat', provider: routeCtx.provider, durationMs, fluxConsumed: 0 }) + throw err + } const usage = extractUsageFromBody(responseBody) const fluxConsumed = calculateFluxFromUsage(usage, fluxPer1kTokens, fallbackRate) @@ -417,7 +475,13 @@ export function createV1Routes( [AIRI_ATTR_BILLING_FLUX_CONSUMED]: fluxConsumed, }) span.end() - recordMetrics({ model: requestModel, status: response.status, type: 'chat', durationMs, fluxConsumed, ...usage }) + generationTrace.succeed({ + output: responseBody, + promptTokens: usage.promptTokens, + completionTokens: usage.completionTokens, + fluxConsumed, + }) + recordMetrics({ model: requestModel, status: response.status, type: 'chat', provider: routeCtx.provider, durationMs, fluxConsumed, ...usage }) // Debit flux via DB transaction (source of truth). // The upstream call has already happened (cost incurred), so partial @@ -526,6 +590,13 @@ export function createV1Routes( speed: typeof body.speed === 'number' ? body.speed : undefined, responseFormat: typeof body.response_format === 'string' ? body.response_format : undefined, } + const generationTrace = startTtsGeneration({ + input: ttsInput, + model: requestModel, + requestId, + userId: user.id, + sessionId: c.req.header('x-airi-session-id'), + }) const span = tracer.startSpan('llm.gateway.tts', { attributes: { @@ -536,15 +607,17 @@ export function createV1Routes( const startedAt = Date.now() + const routeCtx = newRouteContext() let response: Response try { response = await context.with(trace.setSpan(context.active(), span), () => - llmRouter.routeTts({ modelName: requestModel, input: ttsInput, abortSignal: c.req.raw.signal })) + llmRouter.routeTts({ modelName: requestModel, input: ttsInput, abortSignal: c.req.raw.signal }, routeCtx)) } catch (err) { span.setStatus({ code: SpanStatusCode.ERROR, message: 'TTS router exhausted or unknown model' }) span.end() - recordMetrics({ model: requestModel, status: 502, type: 'tts', durationMs: Date.now() - startedAt, fluxConsumed: 0 }) + generationTrace.fail('TTS router exhausted or unknown model') + recordMetrics({ model: requestModel, status: 502, type: 'tts', provider: routeCtx.provider, durationMs: Date.now() - startedAt, fluxConsumed: 0 }) throw err } @@ -554,7 +627,8 @@ export function createV1Routes( if (!response.ok) { span.setStatus({ code: SpanStatusCode.ERROR, message: `Gateway ${response.status}` }) span.end() - recordMetrics({ model: requestModel, status: response.status, type: 'tts', durationMs, fluxConsumed: 0 }) + generationTrace.fail(`Gateway ${response.status}`) + recordMetrics({ model: requestModel, status: response.status, type: 'tts', provider: routeCtx.provider, durationMs, fluxConsumed: 0 }) logger.withFields({ requestId, userId: user.id, model: requestModel, status: response.status, durationMs }) .warn('tts speech delivered with upstream error status') return new Response(response.body, { @@ -583,11 +657,20 @@ export function createV1Routes( }) fluxConsumed = result.fluxDebited span.setAttribute(AIRI_ATTR_BILLING_FLUX_CONSUMED, fluxConsumed) + generationTrace.succeed({ + inputChars: inputText.length, + fluxConsumed, + output: { contentType: response.headers.get('content-type') }, + }) + } + catch (err) { + generationTrace.fail('TTS billing failed') + throw err } finally { span.end() } - recordMetrics({ model: requestModel, status: response.status, type: 'tts', durationMs, fluxConsumed }) + recordMetrics({ model: requestModel, status: response.status, type: 'tts', provider: routeCtx.provider, durationMs, fluxConsumed }) recordRequestLog({ userId: user.id, diff --git a/apps/server/src/services/domain/llm-router/index.ts b/apps/server/src/services/domain/llm-router/index.ts index 50c92f585..64098d8d8 100644 --- a/apps/server/src/services/domain/llm-router/index.ts +++ b/apps/server/src/services/domain/llm-router/index.ts @@ -3,4 +3,4 @@ export { createConfigSyncSubscriber } from './config-sync-subscriber' export { createLlmRouterService } from './router' export type { LlmRouterService } from './router' -export type { LlmModel, TtsModel, TtsUpstream } from './types' +export type { LlmModel, LlmRouteContext, TtsModel, TtsUpstream } from './types' diff --git a/apps/server/src/services/domain/llm-router/router.ts b/apps/server/src/services/domain/llm-router/router.ts index 321a726ef..93a7661b8 100644 --- a/apps/server/src/services/domain/llm-router/router.ts +++ b/apps/server/src/services/domain/llm-router/router.ts @@ -7,7 +7,7 @@ import type { GatewayMetrics } from '../../../otel' import type { EnvelopeCrypto } from '../../../utils/envelope-crypto' import type { ConfigKVService } from '../../adapters/config-kv' import type { TtsAdapterId, TtsInput } from '../../adapters/tts/types' -import type { LlmRouteRequest, LlmUpstream, TtsUpstream } from './types' +import type { LlmRouteContext, LlmRouteRequest, LlmUpstream, TtsUpstream } from './types' import { Buffer as NodeBuffer } from 'node:buffer' @@ -325,7 +325,7 @@ export function createLlmRouterService(options: CreateLlmRouterServiceOptions) { return { kind: 'exhausted', failures } } - async function route(req: LlmRouteRequest): Promise { + async function route(req: LlmRouteRequest, ctx?: LlmRouteContext): Promise { // Honor pre-flight cancellation before any work. if (req.abortSignal?.aborted) throw req.abortSignal.reason ?? new Error('aborted') @@ -348,6 +348,11 @@ export function createLlmRouterService(options: CreateLlmRouterServiceOptions) { const upstream = slice.model.upstreams[i] const provider = deriveProviderTag(upstream.baseURL) triedUpstreams += 1 + // Surface the current upstream so the caller can label success metrics + // by provider. On `ok` this holds the winning provider; on full + // exhaustion it holds the last one tried. + if (ctx) + ctx.provider = provider const perAttemptTimeoutMs = upstream.timeoutMs ?? defaults.perAttemptTimeoutMs ?? 30000 @@ -534,7 +539,7 @@ export function createLlmRouterService(options: CreateLlmRouterServiceOptions) { return { kind: 'exhausted', failures } } - async function routeTts(req: { modelName: string, input: TtsInput, abortSignal?: AbortSignal }): Promise { + async function routeTts(req: { modelName: string, input: TtsInput, abortSignal?: AbortSignal }, ctx?: LlmRouteContext): Promise { if (req.abortSignal?.aborted) throw req.abortSignal.reason ?? new Error('aborted') @@ -560,6 +565,10 @@ export function createLlmRouterService(options: CreateLlmRouterServiceOptions) { const upstream = slice.model.upstreams[i] const providerTag = deriveProviderTag(upstream.baseURL) triedUpstreams += 1 + // Surface the current upstream so the caller can label success metrics + // by provider (winning provider on `ok`, last-tried on exhaustion). + if (ctx) + ctx.provider = providerTag // tts upstream schema has no per-upstream timeoutMs (see ttsUpstreamSchema); // we use the defaults bucket alone here. diff --git a/apps/server/src/services/domain/llm-router/tests/router.test.ts b/apps/server/src/services/domain/llm-router/tests/router.test.ts index 3cfda3ca6..95b4a9f6f 100644 --- a/apps/server/src/services/domain/llm-router/tests/router.test.ts +++ b/apps/server/src/services/domain/llm-router/tests/router.test.ts @@ -156,6 +156,61 @@ describe('createLlmRouterService', () => { expect((metrics.keyExhaustedCount.add as ReturnType).mock.calls.length).toBe(0) }) + it('reports the winning upstream via ctx.provider (happy path)', async () => { + // ROOT CAUSE: + // + // The success-path gen_ai metrics (operation count/duration/tokens) were + // labelled by model only, so a per-provider rollup in Grafana was + // impossible — the route layer never learned which upstream served the + // request. We thread an out-param `ctx` the router fills with the upstream + // it used so those metrics can carry a `provider` label. + const { config, crypto } = makeConfig({ upstreams: [{ baseURL: 'https://up.example/v1', keyIds: ['kA1'] }] }) + const fetchImpl = vi.fn(async () => happyResponse({ ok: 1 })) + const router = createLlmRouterService({ + configKV: makeConfigKV(config), + envelopeCrypto: crypto, + gatewayMetrics: null, + fetchImpl, + redis: makeRedisStub(), + }) + + const ctx = { provider: 'unknown', triedUpstreams: 0, triedKeys: 0, lastStatus: null } + const res = await router.route({ modelName: 'openai/gpt-5-mini', body: { messages: [] } }, ctx) + expect(res.status).toBe(200) + // deriveProviderTag = URL hostname. + expect(ctx.provider).toBe('up.example') + }) + + it('ctx.provider reflects the upstream that actually succeeded after fallback', async () => { + // ROOT CAUSE: + // + // With a fallback chain the winning provider is whichever upstream finally + // returned 200, not the first one tried. ctx.provider must be the winner + // (up-b), else per-provider success metrics would mis-attribute the request + // to the failing upstream. + const { config, crypto } = makeConfig({ + upstreams: [ + { baseURL: 'https://up-a.example/v1', keyIds: ['kA1'] }, + { baseURL: 'https://up-b.example/v1', keyIds: ['kB1'] }, + ], + }) + const fetchImpl = vi.fn() + .mockResolvedValueOnce(failResponse(401)) + .mockResolvedValueOnce(happyResponse({ ok: 1 })) + const router = createLlmRouterService({ + configKV: makeConfigKV(config), + envelopeCrypto: crypto, + gatewayMetrics: makeMetrics(), + fetchImpl, + redis: makeRedisStub(), + }) + + const ctx = { provider: 'unknown', triedUpstreams: 0, triedKeys: 0, lastStatus: null } + const res = await router.route({ modelName: 'openai/gpt-5-mini', body: {} }, ctx) + expect(res.status).toBe(200) + expect(ctx.provider).toBe('up-b.example') + }) + it('happy path injects Bearer + model + url correctly', async () => { const { config, crypto } = makeConfig({ upstreams: [{ baseURL: 'https://up.example/v1/', keyIds: ['kA1'] }] }) const fetchImpl: typeof fetch = vi.fn(async () => happyResponse({ ok: 1 })) as unknown as typeof fetch diff --git a/apps/server/src/services/domain/llm-tracing/index.test.ts b/apps/server/src/services/domain/llm-tracing/index.test.ts new file mode 100644 index 000000000..8668b11de --- /dev/null +++ b/apps/server/src/services/domain/llm-tracing/index.test.ts @@ -0,0 +1,206 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +import { startChatGeneration, startTtsGeneration } from '.' + +// Mock the Langfuse SDK so tests assert what the module sends to it without a +// real exporter. `startObservation` returns a stub generation whose methods are +// spies; `otelSpan.setAttribute` captures trace-identity attributes. +const generationStub = { + otelSpan: { setAttribute: vi.fn() }, + update: vi.fn(), + end: vi.fn(), +} +const startObservation = vi.fn((_name: string, _attributes: unknown, _options: unknown) => generationStub) +vi.mock('@langfuse/tracing', () => ({ + startObservation: (name: string, attributes: unknown, options: unknown) => startObservation(name, attributes, options), +})) + +const BASE_INPUT = { + input: [{ role: 'user', content: 'hi' }], + model: 'openai/gpt-5-mini', + requestId: 'req-1', + stream: false, + userId: 'user-1', +} + +beforeEach(() => { + startObservation.mockClear() + generationStub.otelSpan.setAttribute.mockClear() + generationStub.update.mockClear() + generationStub.end.mockClear() +}) + +afterEach(() => { + vi.unstubAllEnvs() +}) + +describe('startChatGeneration', () => { + describe('when LANGFUSE_TRACING_ACTIVE is not "1"', () => { + it('returns a no-op trace and never calls the SDK', () => { + // @example disabled deployment: no env set + const trace = startChatGeneration(BASE_INPUT) + trace.appendStreamChunk('data: {"choices":[{"delta":{"content":"x"}}]}\n') + trace.succeed({ output: 'x', promptTokens: 1, completionTokens: 1 }) + trace.fail('should be ignored') + + expect(startObservation).not.toHaveBeenCalled() + expect(generationStub.update).not.toHaveBeenCalled() + expect(generationStub.end).not.toHaveBeenCalled() + }) + }) + + describe('when LANGFUSE_TRACING_ACTIVE is "1"', () => { + beforeEach(() => { + vi.stubEnv('LANGFUSE_TRACING_ACTIVE', '1') + }) + + it('creates a generation with input/model/metadata and trace identity', () => { + // @example a request with a client conversation id + startChatGeneration({ ...BASE_INPUT, sessionId: 'sess-9', stream: true }) + + expect(startObservation).toHaveBeenCalledWith( + 'chat.completion', + { + input: BASE_INPUT.input, + model: BASE_INPUT.model, + metadata: { requestId: 'req-1', stream: true }, + }, + { asType: 'generation' }, + ) + expect(generationStub.otelSpan.setAttribute).toHaveBeenCalledWith('langfuse.user.id', 'user-1') + expect(generationStub.otelSpan.setAttribute).toHaveBeenCalledWith('langfuse.session.id', 'sess-9') + }) + + it('omits session attribute when no sessionId is supplied', () => { + // @example a request without x-airi-session-id → user-only attribution + startChatGeneration(BASE_INPUT) + + expect(generationStub.otelSpan.setAttribute).toHaveBeenCalledWith('langfuse.user.id', 'user-1') + expect(generationStub.otelSpan.setAttribute).not.toHaveBeenCalledWith('langfuse.session.id', expect.anything()) + }) + + it('records explicit output + usage + flux on succeed (non-streaming)', () => { + // @example non-streaming completion passes the parsed response body + const trace = startChatGeneration(BASE_INPUT) + trace.succeed({ output: { ok: true }, promptTokens: 12, completionTokens: 34, fluxConsumed: 5 }) + + expect(generationStub.update).toHaveBeenCalledWith({ + output: { ok: true }, + usageDetails: { input: 12, output: 34 }, + metadata: { requestId: 'req-1', stream: false, fluxConsumed: 5 }, + }) + expect(generationStub.end).toHaveBeenCalledTimes(1) + }) + + it('assembles streamed assistant text across chunk boundaries for succeed output', () => { + // @example a delta whose JSON is split mid-key across two chunks + const trace = startChatGeneration({ ...BASE_INPUT, stream: true }) + trace.appendStreamChunk('data: {"choices":[{"delta":{"role":"assistant"}}]}\n') + trace.appendStreamChunk('data: {"choices":[{"delta":{"con') + trace.appendStreamChunk('tent":"Hel"}}]}\ndata: {"choices":[{"delta":{"content":"lo"}}]}\n') + trace.appendStreamChunk('data: [DONE]\n') + trace.succeed({ promptTokens: 2, completionTokens: 1, fluxConsumed: 1 }) + + expect(generationStub.update).toHaveBeenCalledWith({ + output: 'Hello', + usageDetails: { input: 2, output: 1 }, + metadata: { requestId: 'req-1', stream: true, fluxConsumed: 1 }, + }) + }) + + it('ignores malformed and non-text SSE lines when assembling output', () => { + // @example usage-only chunk, blank line, and broken JSON contribute nothing + const trace = startChatGeneration({ ...BASE_INPUT, stream: true }) + trace.appendStreamChunk('\n') + trace.appendStreamChunk('data: {bad json\n') + trace.appendStreamChunk('data: {"choices":[],"usage":{"prompt_tokens":5}}\n') + trace.appendStreamChunk('data: {"choices":[{"delta":{"content":"A"}}]}\n') + trace.succeed({}) + + expect(generationStub.update).toHaveBeenCalledWith( + expect.objectContaining({ output: 'A' }), + ) + }) + + it('records ERROR level + message on fail', () => { + // @example router exhaustion / upstream non-2xx + const trace = startChatGeneration(BASE_INPUT) + trace.fail('Gateway 502') + + expect(generationStub.update).toHaveBeenCalledWith({ + level: 'ERROR', + statusMessage: 'Gateway 502', + metadata: { requestId: 'req-1', stream: false }, + }) + expect(generationStub.end).toHaveBeenCalledTimes(1) + }) + + it('ends only once even if succeed/fail are called repeatedly', () => { + // @example defensive: overlapping transport exit branches + const trace = startChatGeneration(BASE_INPUT) + trace.succeed({ output: 'first' }) + trace.succeed({ output: 'second' }) + trace.fail('late failure') + + expect(generationStub.end).toHaveBeenCalledTimes(1) + expect(generationStub.update).toHaveBeenCalledTimes(1) + expect(generationStub.update).toHaveBeenCalledWith(expect.objectContaining({ output: 'first' })) + }) + + it('hard-caps streamed assistant text even when one SSE delta exceeds the remaining space', () => { + // @example a single very large provider delta should not overflow the buffer cap + const trace = startChatGeneration({ ...BASE_INPUT, stream: true }) + trace.appendStreamChunk(`data: ${JSON.stringify({ choices: [{ delta: { content: 'x'.repeat(1_100_000) } }] })}\n`) + trace.succeed({}) + + const output = generationStub.update.mock.calls[0][0].output + expect(output).toHaveLength(1_000_000) + }) + + it('creates a TTS generation and records character usage without buffering audio', () => { + // @example /audio/speech request: text in, content-type metadata out + const trace = startTtsGeneration({ + input: { text: 'hello', voice: 'alloy', responseFormat: 'mp3' }, + model: 'tts-1', + requestId: 'tts-1', + userId: 'user-1', + sessionId: 'sess-1', + }) + trace.succeed({ + inputChars: 5, + fluxConsumed: 2, + output: { contentType: 'audio/mpeg' }, + }) + + expect(startObservation).toHaveBeenCalledWith( + 'tts.speech', + { + input: { text: 'hello', voice: 'alloy', responseFormat: 'mp3' }, + model: 'tts-1', + metadata: { + requestId: 'tts-1', + inputChars: 5, + voice: 'alloy', + speed: undefined, + responseFormat: 'mp3', + }, + }, + { asType: 'generation' }, + ) + expect(generationStub.update).toHaveBeenCalledWith({ + output: { contentType: 'audio/mpeg' }, + usageDetails: { input: 5 }, + metadata: { + requestId: 'tts-1', + inputChars: 5, + voice: 'alloy', + speed: undefined, + responseFormat: 'mp3', + fluxConsumed: 2, + }, + }) + expect(generationStub.otelSpan.setAttribute).toHaveBeenCalledWith('langfuse.session.id', 'sess-1') + expect(generationStub.end).toHaveBeenCalledTimes(1) + }) + }) +}) diff --git a/apps/server/src/services/domain/llm-tracing/index.ts b/apps/server/src/services/domain/llm-tracing/index.ts new file mode 100644 index 000000000..f67dbf861 --- /dev/null +++ b/apps/server/src/services/domain/llm-tracing/index.ts @@ -0,0 +1,331 @@ +import process from 'node:process' + +import { startObservation } from '@langfuse/tracing' + +/** + * Upper bound on the assistant text buffered for a streaming generation's + * `output`. Without a cap a pathological long completion under concurrency would + * pin `N × completion_size` in memory; Langfuse truncates large payloads + * server-side anyway, so a generous cap loses nothing useful. + */ +const STREAM_OUTPUT_CHAR_CAP = 1_000_000 + +/** + * Whether per-request Langfuse generations should be created. + * + * Gated on the `LANGFUSE_TRACING_ACTIVE` sentinel that `instrumentation.ts` sets + * ONLY after `setLangfuseTracerProvider()` succeeds — not on a raw key check. + * Why: if the isolated Langfuse provider is not actually wired, `startObservation` + * falls back to the GLOBAL OTel TracerProvider, which would ship prompt/completion + * text to the OTLP/Grafana exporter. Binding to the real provider state (single + * source of truth in instrumentation.ts) keeps a future change to the enable + * condition there from silently desyncing this gate and leaking PII to the wrong + * backend. Read per call (cheap; the value is process-constant after the preload + * sets it) so the boundary stays self-contained and trivially testable. + */ +function tracingActive(): boolean { + return process.env.LANGFUSE_TRACING_ACTIVE === '1' +} + +/** + * Extracts the assistant text delta from a single OpenAI streaming SSE line. + * + * Before: + * - `data: {"choices":[{"delta":{"content":"Hel"}}]}` + * + * After: + * - `"Hel"` + * + * Returns `''` for lines that carry no assistant text — `[DONE]`, role-only + * deltas, usage-only chunks, blank/comment lines, or malformed JSON. The upstream + * SSE is an external boundary, so per-line JSON parsing is tolerated and a parse + * failure degrades to "this line added no text" rather than aborting capture: + * output is best-effort trace data, not billing. + */ +function extractSseDeltaText(sseLine: string): string { + const trimmed = sseLine.trimStart() + if (!trimmed.startsWith('data:')) + return '' + const payload = trimmed.slice(5).trim() + if (!payload || payload === '[DONE]') + return '' + try { + const json = JSON.parse(payload) + const content = json?.choices?.[0]?.delta?.content + return typeof content === 'string' ? content : '' + } + catch { + return '' + } +} + +/** Parameters identifying a request a Langfuse generation traces. */ +interface GenerationInput { + /** Provider-domain input payload, recorded verbatim as trace input. */ + input: unknown + /** Resolved upstream model id (after `auto` aliases are replaced). */ + model: string + /** Correlation id shared with billing / request-log rows. */ + requestId: string + /** Generation name shown in Langfuse. */ + name: string + /** Extra observation metadata. */ + metadata?: Record + /** Billing/identity owner of the request. Lifted to trace-level `userId`. */ + userId: string + /** Client-supplied conversation id (`x-airi-session-id`). Absent → user-only attribution. */ + sessionId?: string +} + +/** Parameters identifying the chat request a generation traces. */ +export interface ChatGenerationInput extends Omit { + /** OpenAI chat `messages` array (the prompt), recorded verbatim as trace input. */ + input: unknown + /** Whether the response is streamed (affects how output is captured). */ + stream: boolean +} + +/** Parameters identifying the TTS request a generation traces. */ +export interface TtsGenerationInput extends Omit { + /** Adapter-neutral TTS request payload, recorded as trace input. */ + input: { + text: string + voice?: string + speed?: number + responseFormat?: string + } +} + +/** Terminal usage/cost figures recorded when a generation completes successfully. */ +interface GenerationResult { + /** + * Explicit completion to record. Omit for streaming requests to use the + * assistant text assembled from the streamed SSE deltas. + */ + output?: unknown + /** Usage dimensions for Langfuse. For chat this is token counts; for TTS this is character count. */ + usageDetails?: Record + /** AIRI business cost (flux). Stored in generation metadata, not `costDetails`. */ + fluxConsumed?: number + /** Additional terminal metadata to merge with request metadata. */ + metadata?: Record +} + +/** Terminal usage/cost figures recorded when a chat generation completes successfully. */ +export interface ChatGenerationResult { + /** + * Explicit completion to record. Omit for streaming requests to use the + * assistant text assembled from the streamed SSE deltas. + */ + output?: unknown + promptTokens?: number + completionTokens?: number + /** AIRI business cost (flux). Stored in generation metadata, not `costDetails`. */ + fluxConsumed?: number +} + +/** Terminal usage/cost figures recorded when a TTS generation completes successfully. */ +export interface TtsGenerationResult { + /** Output metadata only; binary audio is not buffered into Langfuse. */ + output?: unknown + /** Input character count charged by the TTS flux meter. */ + inputChars: number + /** AIRI business cost (flux). Stored in generation metadata, not `costDetails`. */ + fluxConsumed?: number + /** Additional terminal metadata to merge with request metadata. */ + metadata?: Record +} + +/** + * Lifecycle handle for one chat completion's Langfuse generation. + * + * Hides whether Langfuse is enabled (no-op when off), the SDK call shape, the + * trace field mapping, and the streamed-output assembly. The owning route only + * drives the domain lifecycle: feed stream chunks, then end with success or + * failure exactly once (subsequent calls are ignored, so every transport exit + * branch can call defensively without double-ending). + */ +export interface ChatGenerationTrace { + /** + * Feed one decoded chunk of streamed SSE text. Accumulates the assistant + * completion for the trace `output`, bounded by the char cap. No-op for + * non-streaming requests (which pass `output` to {@link ChatGenerationTrace.succeed}). + */ + appendStreamChunk: (decodedChunk: string) => void + /** Record a successful completion with usage/cost and end the generation. */ + succeed: (result: ChatGenerationResult) => void + /** Record a failure (`level: ERROR` + message) and end the generation. */ + fail: (statusMessage: string) => void +} + +/** Lifecycle handle for one TTS Langfuse generation. */ +export interface TtsGenerationTrace { + /** Record a successful speech generation with character usage/cost and end the generation. */ + succeed: (result: TtsGenerationResult) => void + /** Record a failure (`level: ERROR` + message) and end the generation. */ + fail: (statusMessage: string) => void +} + +const NOOP_CHAT_TRACE: ChatGenerationTrace = { + appendStreamChunk() {}, + succeed() {}, + fail() {}, +} + +const NOOP_TTS_TRACE: TtsGenerationTrace = { + succeed() {}, + fail() {}, +} + +function startGeneration(input: GenerationInput): { + succeed: (result: GenerationResult) => void + fail: (statusMessage: string) => void +} | null { + if (!tracingActive()) + return null + + const baseMetadata = { requestId: input.requestId, ...input.metadata } + const generation = startObservation(input.name, { + input: input.input, + model: input.model, + metadata: baseMetadata, + }, { asType: 'generation' }) + // Trace-level identity via Langfuse compat attributes, lifted to the trace by + // the platform — enables per-user / per-session cost attribution. + generation.otelSpan.setAttribute('langfuse.user.id', input.userId) + if (input.sessionId) + generation.otelSpan.setAttribute('langfuse.session.id', input.sessionId) + + let ended = false + + return { + succeed(result) { + if (ended) + return + ended = true + generation.update({ + output: result.output, + usageDetails: result.usageDetails, + metadata: { ...baseMetadata, ...result.metadata, fluxConsumed: result.fluxConsumed ?? 0 }, + }) + generation.end() + }, + fail(statusMessage) { + if (ended) + return + ended = true + generation.update({ level: 'ERROR', statusMessage, metadata: baseMetadata }) + generation.end() + }, + } +} + +/** + * Starts a Langfuse generation for a chat completion, or a no-op handle when + * Langfuse tracing is disabled. + * + * Use when: + * - Entering a chat completion handler that should be traced for prompt/eval/cost. + * + * Expects: + * - Called once per request. `instrumentation.ts` has already wired the isolated + * Langfuse TracerProvider when tracing is active. + * + * Returns: + * - A {@link ChatGenerationTrace} whose `succeed`/`fail` are idempotent; the + * first call ends the generation and later calls are ignored. + */ +export function startChatGeneration(input: ChatGenerationInput): ChatGenerationTrace { + const generation = startGeneration({ + input: input.input, + model: input.model, + requestId: input.requestId, + name: 'chat.completion', + metadata: { stream: input.stream }, + userId: input.userId, + sessionId: input.sessionId, + }) + if (!generation) + return NOOP_CHAT_TRACE + + let assistantText = '' + let sseLineBuffer = '' + + return { + appendStreamChunk(decodedChunk) { + if (assistantText.length >= STREAM_OUTPUT_CHAR_CAP) + return + // Split on newlines, parse complete lines, keep the partial trailing line + // for the next chunk so a delta split across a chunk boundary still parses. + sseLineBuffer += decodedChunk + const lines = sseLineBuffer.split('\n') + sseLineBuffer = lines.pop() ?? '' + for (const line of lines) { + const deltaText = extractSseDeltaText(line) + const remainingChars = STREAM_OUTPUT_CHAR_CAP - assistantText.length + if (remainingChars <= 0) + break + assistantText += deltaText.slice(0, remainingChars) + if (assistantText.length >= STREAM_OUTPUT_CHAR_CAP) + break + } + }, + succeed(result) { + generation.succeed({ + output: result.output ?? assistantText, + usageDetails: { input: result.promptTokens ?? 0, output: result.completionTokens ?? 0 }, + fluxConsumed: result.fluxConsumed, + }) + }, + fail(statusMessage) { + generation.fail(statusMessage) + }, + } +} + +/** + * Starts a Langfuse generation for a TTS request, or a no-op handle when + * Langfuse tracing is disabled. + * + * Use when: + * - Entering the OpenAI-compatible `/audio/speech` handler. + * + * Expects: + * - Binary audio is not buffered into Langfuse; callers pass output metadata + * such as content type instead. + * + * Returns: + * - A {@link TtsGenerationTrace} whose `succeed`/`fail` are idempotent. + */ +export function startTtsGeneration(input: TtsGenerationInput): TtsGenerationTrace { + const generation = startGeneration({ + input: input.input, + model: input.model, + requestId: input.requestId, + name: 'tts.speech', + metadata: { + inputChars: input.input.text.length, + voice: input.input.voice, + speed: input.input.speed, + responseFormat: input.input.responseFormat, + }, + userId: input.userId, + sessionId: input.sessionId, + }) + if (!generation) + return NOOP_TTS_TRACE + + return { + succeed(result) { + generation.succeed({ + output: result.output, + usageDetails: { input: result.inputChars }, + fluxConsumed: result.fluxConsumed, + metadata: { inputChars: result.inputChars, ...result.metadata }, + }) + }, + fail(statusMessage) { + generation.fail(statusMessage) + }, + } +} diff --git a/packages/stage-ui/src/libs/providers/providers/official/shared.ts b/packages/stage-ui/src/libs/providers/providers/official/shared.ts index c56f51995..ee2021a3b 100644 --- a/packages/stage-ui/src/libs/providers/providers/official/shared.ts +++ b/packages/stage-ui/src/libs/providers/providers/official/shared.ts @@ -1,7 +1,9 @@ import { createOpenAI } from '@xsai-ext/providers/create' +import { getActivePinia } from 'pinia' import { getAuthToken } from '../../../../libs/auth' import { SERVER_URL } from '../../../../libs/server' +import { useChatSessionStore } from '../../../../stores/chat/session-store' export const OFFICIAL_ICON = 'i-solar:star-bold-duotone' @@ -12,6 +14,9 @@ export function withCredentials() { if (token) { headers.set('Authorization', `Bearer ${token}`) } + const chatSession = getActivePinia() ? useChatSessionStore() : null + if (chatSession?.activeSessionId) + headers.set('x-airi-session-id', chatSession.activeSessionId) return globalThis.fetch(input, { ...init, headers, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index caf8428c3..54352b746 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -75,6 +75,12 @@ catalogs: '@intlify/core': specifier: ^11.3.2 version: 11.3.2 + '@langfuse/otel': + specifier: ^5.4.0 + version: 5.4.0 + '@langfuse/tracing': + specifier: ^5.4.0 + version: 5.4.0 '@modelcontextprotocol/sdk': specifier: ^1.29.0 version: 1.29.0 @@ -650,6 +656,12 @@ importers: '@hono/otel': specifier: ^1.1.2 version: 1.1.2(hono@4.11.3) + '@langfuse/otel': + specifier: 'catalog:' + version: 5.4.0(@opentelemetry/api@1.9.1)(@opentelemetry/core@2.7.0(@opentelemetry/api@1.9.1))(@opentelemetry/exporter-trace-otlp-http@0.215.0(@opentelemetry/api@1.9.1))(@opentelemetry/sdk-trace-base@2.7.0(@opentelemetry/api@1.9.1)) + '@langfuse/tracing': + specifier: 'catalog:' + version: 5.4.0(@opentelemetry/api@1.9.1) '@moeru/eventa': specifier: 'catalog:' version: 1.0.0-beta.5(electron@41.2.1)(h3@2.0.1-rc.20(crossws@0.4.5(srvx@0.11.15))) @@ -1483,7 +1495,7 @@ importers: version: 3.0.2(electron@41.2.1) '@electron-toolkit/tsconfig': specifier: ^2.0.0 - version: 2.0.0(@types/node@24.12.2) + version: 2.0.0(@types/node@25.6.0) '@electron-toolkit/utils': specifier: ^4.0.0 version: 4.0.0(electron@41.2.1) @@ -1522,7 +1534,7 @@ importers: version: 3.1.0 '@intlify/unplugin-vue-i18n': specifier: ^11.0.7 - version: 11.0.7(@vue/compiler-dom@3.5.32)(eslint@10.2.1(jiti@2.6.1))(rollup@4.60.1)(typescript@5.9.3)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-i18n@11.3.2(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)) + version: 11.0.7(@vue/compiler-dom@3.5.32)(eslint@10.2.1(jiti@2.6.1))(rollup@4.60.1)(typescript@5.9.3)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-i18n@11.3.2(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)) '@modelcontextprotocol/sdk': specifier: 'catalog:' version: 1.29.0(@cfworker/json-schema@4.1.1)(zod@4.3.6) @@ -1558,10 +1570,10 @@ importers: version: link:../../packages/ui-transitions '@proj-airi/unplugin-fetch': specifier: 'catalog:' - version: 0.2.3(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) + version: 0.2.3(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) '@proj-airi/unplugin-live2d-sdk': specifier: ^0.1.7 - version: 0.1.7(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) + version: 0.1.7(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) '@types/audioworklet': specifier: 'catalog:' version: 0.0.97 @@ -1588,7 +1600,7 @@ importers: version: 2.10.3 '@vitejs/plugin-vue': specifier: ^6.0.6 - version: 6.0.6(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue@3.5.32(typescript@5.9.3)) + version: 6.0.6(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue@3.5.32(typescript@5.9.3)) '@vue-macros/volar': specifier: ^3.1.2 version: 3.1.2(typescript@5.9.3)(vue-tsc@3.2.6(typescript@5.9.3))(vue@3.5.32(typescript@5.9.3)) @@ -1621,7 +1633,7 @@ importers: version: 6.8.3 electron-vite: specifier: ^5.0.0 - version: 5.0.0(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) + version: 5.0.0(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) get-port-please: specifier: 'catalog:' version: 3.2.0 @@ -1642,31 +1654,31 @@ importers: version: 2.2.6 unocss-preset-scrollbar: specifier: ^4.0.0 - version: 4.0.0(unocss@66.6.8(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))) + version: 4.0.0(unocss@66.6.8(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))) unplugin-info: specifier: ^1.3.2 - version: 1.3.2(esbuild@0.27.2)(rollup@4.60.1)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) + version: 1.3.2(esbuild@0.27.2)(rollup@4.60.1)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) unplugin-yaml: specifier: ^4.1.0 - version: 4.1.0(esbuild@0.27.2)(rolldown@1.0.0-rc.16)(rollup@4.60.1)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) + version: 4.1.0(@nuxt/kit@3.20.2(magicast@0.5.2))(esbuild@0.27.2)(rolldown@1.0.0-rc.16)(rollup@4.60.1)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) vite: specifier: 'catalog:' - version: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) + version: 8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) vite-bundle-visualizer: specifier: ^1.2.1 version: 1.2.1(rolldown@1.0.0-rc.16)(rollup@4.60.1) vite-plugin-mkcert: specifier: 'catalog:' - version: 2.0.0(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) + version: 2.0.0(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) vite-plugin-vue-devtools: specifier: ^8.1.1 - version: 8.1.1(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue@3.5.32(typescript@5.9.3)) + version: 8.1.1(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue@3.5.32(typescript@5.9.3)) vite-plugin-vue-layouts: specifier: ^0.11.0 - version: 0.11.0(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-router@5.0.4(@pinia/colada@1.2.1(pinia@3.0.4(typescript@5.9.3)(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)))(@vue/compiler-sfc@3.5.32)(pinia@3.0.4(typescript@5.9.3)(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)) + version: 0.11.0(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-router@5.0.4(@pinia/colada@1.2.1(pinia@3.0.4(typescript@5.9.3)(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)))(@vue/compiler-sfc@3.5.32)(pinia@3.0.4(typescript@5.9.3)(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)) vue-macros: specifier: ^3.1.2 - version: 3.1.2(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(@vueuse/core@14.2.1(vue@3.5.32(typescript@5.9.3)))(esbuild@0.27.2)(rolldown@1.0.0-rc.16)(rollup@4.60.1)(typescript@5.9.3)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-tsc@3.2.6(typescript@5.9.3))(vue@3.5.32(typescript@5.9.3)) + version: 3.1.2(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(@vueuse/core@14.2.1(vue@3.5.32(typescript@5.9.3)))(esbuild@0.27.2)(rolldown@1.0.0-rc.16)(rollup@4.60.1)(typescript@5.9.3)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-tsc@3.2.6(typescript@5.9.3))(vue@3.5.32(typescript@5.9.3)) vue-tsc: specifier: ^3.2.6 version: 3.2.6(typescript@5.9.3) @@ -6910,6 +6922,26 @@ packages: '@kwsites/promise-deferred@1.1.1': resolution: {integrity: sha512-GaHYm+c0O9MjZRu0ongGBRbinu8gVAMd2UZjji6jVmqKtZluZnptXGWhz1E8j8D2HJ3f/yMxKAUC0b+57wncIw==} + '@langfuse/core@5.4.0': + resolution: {integrity: sha512-E9gggJOpIZONUiN9V+FNC8Lz8GNb4RQj2HArtdJDtQ6MVEXqLERwTMO+DUx9+9Hf8nX7CaVTa7KIdmgYwMk7Ew==} + peerDependencies: + '@opentelemetry/api': ^1.9.0 + + '@langfuse/otel@5.4.0': + resolution: {integrity: sha512-oC6YDspXdUTznNEmzNcy+rD5/Z3Zgt2VGCX2bpaEpACzQNyhq/0eF2BZ/5zvOevCG9RwbZiA2K025kS2ef+Dag==} + engines: {node: '>=20'} + peerDependencies: + '@opentelemetry/api': ^1.9.0 + '@opentelemetry/core': ^2.0.1 + '@opentelemetry/exporter-trace-otlp-http': '>=0.202.0 <1.0.0' + '@opentelemetry/sdk-trace-base': ^2.0.1 + + '@langfuse/tracing@5.4.0': + resolution: {integrity: sha512-dUptGzKk2HxE+p7761VdSVK3rtyXRAIUuKamYGzFSEJGxmBe3gzz/injyKKMldH561RVnIADmIbwmPMrTU1DBg==} + engines: {node: '>=20'} + peerDependencies: + '@opentelemetry/api': ^1.9.0 + '@laplace.live/event-bridge-sdk@1.0.22': resolution: {integrity: sha512-15A668PA1vWyS8coj5lI/IA9zqDKPGJO2v+J2YROBpOWuDXFhLCaMS0mNd3IYGT8NkmgU3l9GXg2y333r8VY2g==} @@ -19966,9 +19998,9 @@ snapshots: dependencies: electron: 41.2.1 - '@electron-toolkit/tsconfig@2.0.0(@types/node@24.12.2)': + '@electron-toolkit/tsconfig@2.0.0(@types/node@25.6.0)': dependencies: - '@types/node': 24.12.2 + '@types/node': 25.6.0 '@electron-toolkit/utils@4.0.0(electron@41.2.1)': dependencies: @@ -20880,31 +20912,6 @@ snapshots: - supports-color - typescript - '@intlify/unplugin-vue-i18n@11.0.7(@vue/compiler-dom@3.5.32)(eslint@10.2.1(jiti@2.6.1))(rollup@4.60.1)(typescript@5.9.3)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-i18n@11.3.2(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3))': - dependencies: - '@eslint-community/eslint-utils': 4.9.1(eslint@10.2.1(jiti@2.6.1)) - '@intlify/bundle-utils': 11.0.7(vue-i18n@11.3.2(vue@3.5.32(typescript@5.9.3))) - '@intlify/shared': 11.3.2 - '@intlify/vue-i18n-extensions': 8.0.0(@intlify/shared@11.3.2)(@vue/compiler-dom@3.5.32)(vue-i18n@11.3.2(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)) - '@rollup/pluginutils': 5.3.0(rollup@4.60.1) - '@typescript-eslint/scope-manager': 8.58.1 - '@typescript-eslint/typescript-estree': 8.58.1(typescript@5.9.3) - debug: 4.4.3(supports-color@10.2.2) - fast-glob: 3.3.3 - pathe: 2.0.3 - picocolors: 1.1.1 - unplugin: 2.3.11 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - vue: 3.5.32(typescript@5.9.3) - optionalDependencies: - vue-i18n: 11.3.2(vue@3.5.32(typescript@5.9.3)) - transitivePeerDependencies: - - '@vue/compiler-dom' - - eslint - - rollup - - supports-color - - typescript - '@intlify/unplugin-vue-i18n@11.0.7(@vue/compiler-dom@3.5.32)(eslint@10.2.1(jiti@2.6.1))(rollup@4.60.1)(typescript@5.9.3)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-i18n@11.3.2(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3))': dependencies: '@eslint-community/eslint-utils': 4.9.1(eslint@10.2.1(jiti@2.6.1)) @@ -21134,6 +21141,23 @@ snapshots: '@kwsites/promise-deferred@1.1.1': {} + '@langfuse/core@5.4.0(@opentelemetry/api@1.9.1)': + dependencies: + '@opentelemetry/api': 1.9.1 + + '@langfuse/otel@5.4.0(@opentelemetry/api@1.9.1)(@opentelemetry/core@2.7.0(@opentelemetry/api@1.9.1))(@opentelemetry/exporter-trace-otlp-http@0.215.0(@opentelemetry/api@1.9.1))(@opentelemetry/sdk-trace-base@2.7.0(@opentelemetry/api@1.9.1))': + dependencies: + '@langfuse/core': 5.4.0(@opentelemetry/api@1.9.1) + '@opentelemetry/api': 1.9.1 + '@opentelemetry/core': 2.7.0(@opentelemetry/api@1.9.1) + '@opentelemetry/exporter-trace-otlp-http': 0.215.0(@opentelemetry/api@1.9.1) + '@opentelemetry/sdk-trace-base': 2.7.0(@opentelemetry/api@1.9.1) + + '@langfuse/tracing@5.4.0(@opentelemetry/api@1.9.1)': + dependencies: + '@langfuse/core': 5.4.0(@opentelemetry/api@1.9.1) + '@opentelemetry/api': 1.9.1 + '@laplace.live/event-bridge-sdk@1.0.22(typescript@5.9.3)': dependencies: '@laplace.live/event-types': 2.0.14(typescript@5.9.3) @@ -22886,35 +22910,11 @@ snapshots: transitivePeerDependencies: - magicast - '@proj-airi/unplugin-fetch@0.2.3(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))': - dependencies: - ofetch: 1.5.1 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - '@proj-airi/unplugin-fetch@0.2.3(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))': dependencies: ofetch: 1.5.1 vite: 8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - '@proj-airi/unplugin-live2d-sdk@0.1.7(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)': - dependencies: - ofetch: 1.5.1 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - yauzl: 3.3.0 - transitivePeerDependencies: - - '@types/node' - - '@vitejs/devtools' - - esbuild - - jiti - - less - - sass - - sass-embedded - - stylus - - sugarss - - terser - - tsx - - yaml - '@proj-airi/unplugin-live2d-sdk@0.1.7(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)': dependencies: ofetch: 1.5.1 @@ -23815,7 +23815,6 @@ snapshots: '@types/node@25.6.0': dependencies: undici-types: 7.19.2 - optional: true '@types/nprogress@0.2.3': {} @@ -24481,12 +24480,6 @@ snapshots: vite: 6.4.2(@types/node@25.6.0)(jiti@2.6.1)(less@4.6.4)(lightningcss@1.32.0)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) vue: 3.5.32(typescript@5.9.3) - '@vitejs/plugin-vue@6.0.6(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue@3.5.32(typescript@5.9.3))': - dependencies: - '@rolldown/pluginutils': 1.0.0-rc.13 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - vue: 3.5.32(typescript@5.9.3) - '@vitejs/plugin-vue@6.0.6(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue@3.5.32(typescript@5.9.3))': dependencies: '@rolldown/pluginutils': 1.0.0-rc.13 @@ -24567,9 +24560,9 @@ snapshots: obug: 2.1.1 std-env: 4.1.0 tinyrainbow: 3.1.0 - vitest: 4.1.4(@opentelemetry/api@1.9.1)(@types/node@24.12.2)(@vitest/browser-playwright@4.1.4)(@vitest/coverage-v8@4.1.4)(jsdom@29.1.1(@noble/hashes@2.0.1)(canvas@3.2.3))(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) + vitest: 4.1.4(@opentelemetry/api@1.9.1)(@types/node@25.6.0)(@vitest/browser-playwright@4.1.4)(@vitest/coverage-v8@4.1.4)(jsdom@29.1.1(@noble/hashes@2.0.1)(canvas@3.2.3))(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) optionalDependencies: - '@vitest/browser': 4.1.4(bufferutil@4.1.0)(utf-8-validate@5.0.10)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vitest@4.1.4) + '@vitest/browser': 4.1.4(bufferutil@4.1.0)(utf-8-validate@5.0.10)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vitest@4.1.4) '@vitest/eslint-plugin@1.6.15(@typescript-eslint/eslint-plugin@8.58.1(@typescript-eslint/parser@8.58.1(eslint@10.2.1(jiti@2.6.1))(typescript@5.9.3))(eslint@10.2.1(jiti@2.6.1))(typescript@5.9.3))(eslint@10.2.1(jiti@2.6.1))(typescript@5.9.3)(vitest@4.1.4)': dependencies: @@ -24789,15 +24782,6 @@ snapshots: transitivePeerDependencies: - vue - '@vue-macros/devtools@3.1.2(typescript@5.9.3)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))': - dependencies: - sirv: 3.0.2 - vue: 3.5.32(typescript@5.9.3) - optionalDependencies: - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - transitivePeerDependencies: - - typescript - '@vue-macros/devtools@3.1.2(typescript@5.9.3)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))': dependencies: sirv: 3.0.2 @@ -26939,7 +26923,7 @@ snapshots: transitivePeerDependencies: - supports-color - electron-vite@5.0.0(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): + electron-vite@5.0.0(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): dependencies: '@babel/core': 7.29.0 '@babel/plugin-transform-arrow-functions': 7.27.1(@babel/core@7.29.0) @@ -26947,7 +26931,7 @@ snapshots: esbuild: 0.25.12 magic-string: 0.30.21 picocolors: 1.1.1 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) + vite: 8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) transitivePeerDependencies: - supports-color @@ -32863,8 +32847,7 @@ snapshots: undici-types@7.16.0: {} - undici-types@7.19.2: - optional: true + undici-types@7.19.2: {} undici@6.24.1: {} @@ -32976,6 +32959,11 @@ snapshots: '@unocss/preset-mini': 66.6.8 unocss: 66.6.8(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) + unocss-preset-scrollbar@4.0.0(unocss@66.6.8(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))): + dependencies: + '@unocss/preset-mini': 66.6.8 + unocss: 66.6.8(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) + unocss@66.6.8(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(vite@6.4.2(@types/node@25.6.0)(jiti@2.6.1)(less@4.6.4)(lightningcss@1.32.0)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): dependencies: '@unocss/cli': 66.6.8 @@ -33072,14 +33060,6 @@ snapshots: unplugin: 2.3.11 vite: 8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - unplugin-combine@2.3.0(esbuild@0.27.2)(rolldown@1.0.0-rc.16)(rollup@4.60.1)(unplugin@2.3.11)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): - optionalDependencies: - esbuild: 0.27.2 - rolldown: 1.0.0-rc.16 - rollup: 4.60.1 - unplugin: 2.3.11 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - unplugin-combine@2.3.0(esbuild@0.27.2)(rolldown@1.0.0-rc.16)(rollup@4.60.1)(unplugin@2.3.11)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): optionalDependencies: esbuild: 0.27.2 @@ -33114,19 +33094,6 @@ snapshots: transitivePeerDependencies: - supports-color - unplugin-info@1.3.2(esbuild@0.27.2)(rollup@4.60.1)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): - dependencies: - ci-info: 4.4.0 - git-url-parse: 16.1.0 - simple-git: 3.36.0 - unplugin: 2.3.11 - optionalDependencies: - esbuild: 0.27.2 - rollup: 4.60.1 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - transitivePeerDependencies: - - supports-color - unplugin-info@1.3.2(esbuild@0.27.2)(rollup@4.60.1)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): dependencies: ci-info: 4.4.0 @@ -33236,17 +33203,6 @@ snapshots: rollup: 4.60.1 vite: 6.4.2(@types/node@25.6.0)(jiti@2.6.1)(less@4.6.4)(lightningcss@1.32.0)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - unplugin-yaml@4.1.0(esbuild@0.27.2)(rolldown@1.0.0-rc.16)(rollup@4.60.1)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): - dependencies: - '@rollup/pluginutils': 5.3.0(rollup@4.60.1) - unplugin: 3.0.0 - yaml: 2.8.3 - optionalDependencies: - esbuild: 0.27.2 - rolldown: 1.0.0-rc.16 - rollup: 4.60.1 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - unplugin@2.3.11: dependencies: '@jridgewell/remapping': 2.3.5 @@ -33478,22 +33434,12 @@ snapshots: - rollup - supports-color - vite-dev-rpc@1.1.0(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): - dependencies: - birpc: 2.9.0 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - vite-hot-client: 2.1.0(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) - vite-dev-rpc@1.1.0(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): dependencies: birpc: 2.9.0 vite: 8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) vite-hot-client: 2.1.0(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) - vite-hot-client@2.1.0(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): - dependencies: - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - vite-hot-client@2.1.0(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): dependencies: vite: 8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) @@ -33539,21 +33485,6 @@ snapshots: - tsx - yaml - vite-plugin-inspect@11.3.3(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): - dependencies: - ansis: 4.2.0 - debug: 4.4.3(supports-color@10.2.2) - error-stack-parser-es: 1.0.5 - ohash: 2.0.11 - open: 10.2.0 - perfect-debounce: 2.1.0 - sirv: 3.0.2 - unplugin-utils: 0.3.1 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - vite-dev-rpc: 1.1.0(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) - transitivePeerDependencies: - - supports-color - vite-plugin-inspect@11.3.3(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): dependencies: ansis: 4.2.0 @@ -33585,13 +33516,6 @@ snapshots: - typescript - ws - vite-plugin-mkcert@2.0.0(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): - dependencies: - debug: 4.4.3(supports-color@10.2.2) - supports-color: 10.2.2 - undici: 8.1.0 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - vite-plugin-mkcert@2.0.0(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): dependencies: debug: 4.4.3(supports-color@10.2.2) @@ -33610,20 +33534,6 @@ snapshots: transitivePeerDependencies: - supports-color - vite-plugin-vue-devtools@8.1.1(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue@3.5.32(typescript@5.9.3)): - dependencies: - '@vue/devtools-core': 8.1.1(vue@3.5.32(typescript@5.9.3)) - '@vue/devtools-kit': 8.1.1 - '@vue/devtools-shared': 8.1.1 - sirv: 3.0.2 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - vite-plugin-inspect: 11.3.3(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) - vite-plugin-vue-inspector: 5.3.2(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) - transitivePeerDependencies: - - '@nuxt/kit' - - supports-color - - vue - vite-plugin-vue-devtools@8.1.1(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue@3.5.32(typescript@5.9.3)): dependencies: '@vue/devtools-core': 8.1.1(vue@3.5.32(typescript@5.9.3)) @@ -33638,21 +33548,6 @@ snapshots: - supports-color - vue - vite-plugin-vue-inspector@5.3.2(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): - dependencies: - '@babel/core': 7.29.0 - '@babel/plugin-proposal-decorators': 7.28.0(@babel/core@7.29.0) - '@babel/plugin-syntax-import-attributes': 7.28.6(@babel/core@7.29.0) - '@babel/plugin-syntax-import-meta': 7.10.4(@babel/core@7.29.0) - '@babel/plugin-transform-typescript': 7.28.5(@babel/core@7.29.0) - '@vue/babel-plugin-jsx': 1.5.0(@babel/core@7.29.0) - '@vue/compiler-dom': 3.5.32 - kolorist: 1.8.0 - magic-string: 0.30.21 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - transitivePeerDependencies: - - supports-color - vite-plugin-vue-inspector@5.3.2(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)): dependencies: '@babel/core': 7.29.0 @@ -33668,16 +33563,6 @@ snapshots: transitivePeerDependencies: - supports-color - vite-plugin-vue-layouts@0.11.0(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-router@5.0.4(@pinia/colada@1.2.1(pinia@3.0.4(typescript@5.9.3)(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)))(@vue/compiler-sfc@3.5.32)(pinia@3.0.4(typescript@5.9.3)(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)): - dependencies: - debug: 4.4.3(supports-color@10.2.2) - fast-glob: 3.3.3 - vite: 8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3) - vue: 3.5.32(typescript@5.9.3) - vue-router: 5.0.4(@pinia/colada@1.2.1(pinia@3.0.4(typescript@5.9.3)(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)))(@vue/compiler-sfc@3.5.32)(pinia@3.0.4(typescript@5.9.3)(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)) - transitivePeerDependencies: - - supports-color - vite-plugin-vue-layouts@0.11.0(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-router@5.0.4(@pinia/colada@1.2.1(pinia@3.0.4(typescript@5.9.3)(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)))(@vue/compiler-sfc@3.5.32)(pinia@3.0.4(typescript@5.9.3)(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)): dependencies: debug: 4.4.3(supports-color@10.2.2) @@ -33944,54 +33829,6 @@ snapshots: - vue-tsc - webpack - vue-macros@3.1.2(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(@vueuse/core@14.2.1(vue@3.5.32(typescript@5.9.3)))(esbuild@0.27.2)(rolldown@1.0.0-rc.16)(rollup@4.60.1)(typescript@5.9.3)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-tsc@3.2.6(typescript@5.9.3))(vue@3.5.32(typescript@5.9.3)): - dependencies: - '@vue-macros/better-define': 3.1.2(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/boolean-prop': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/chain-call': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/common': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/config': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/define-emit': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/define-models': 3.1.2(@vueuse/core@14.2.1(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/define-prop': 3.1.2(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/define-props': 3.1.2(@vue-macros/reactivity-transform@3.1.2(vue@3.5.32(typescript@5.9.3)))(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/define-props-refs': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/define-render': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/define-slots': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/define-stylex': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/devtools': 3.1.2(typescript@5.9.3)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) - '@vue-macros/export-expose': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/export-props': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/export-render': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/hoist-static': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/jsx-directive': 3.1.2(typescript@5.9.3) - '@vue-macros/named-template': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/reactivity-transform': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/script-lang': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/setup-block': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/setup-component': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/setup-sfc': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/short-bind': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/short-emits': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/short-vmodel': 3.1.2(vue@3.5.32(typescript@5.9.3)) - '@vue-macros/volar': 3.1.2(typescript@5.9.3)(vue-tsc@3.2.6(typescript@5.9.3))(vue@3.5.32(typescript@5.9.3)) - unplugin: 2.3.11 - unplugin-combine: 2.3.0(esbuild@0.27.2)(rolldown@1.0.0-rc.16)(rollup@4.60.1)(unplugin@2.3.11)(vite@8.0.8(@types/node@24.12.2)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3)) - unplugin-vue-define-options: 3.1.2(vue@3.5.32(typescript@5.9.3)) - vue: 3.5.32(typescript@5.9.3) - transitivePeerDependencies: - - '@emnapi/core' - - '@emnapi/runtime' - - '@rspack/core' - - '@vueuse/core' - - esbuild - - rolldown - - rollup - - typescript - - vite - - vue-tsc - - webpack - vue-macros@3.1.2(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(@vueuse/core@14.2.1(vue@3.5.32(typescript@5.9.3)))(esbuild@0.27.2)(rolldown@1.0.0-rc.16)(rollup@4.60.1)(typescript@5.9.3)(vite@8.0.8(@types/node@25.6.0)(esbuild@0.27.2)(jiti@2.6.1)(less@4.6.4)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))(vue-tsc@3.2.6(typescript@5.9.3))(vue@3.5.32(typescript@5.9.3)): dependencies: '@vue-macros/better-define': 3.1.2(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(vue@3.5.32(typescript@5.9.3)) diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 41d4e1556..04522bed2 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -53,6 +53,8 @@ catalog: '@iconify-json/ph': ^1.2.2 '@iconify-json/tabler': ^1.2.33 '@intlify/core': ^11.3.2 + '@langfuse/otel': ^5.4.0 + '@langfuse/tracing': ^5.4.0 '@modelcontextprotocol/sdk': ^1.29.0 '@moeru/eslint-config': 0.1.0-beta.15 '@moeru/eventa': 1.0.0-beta.5