mirror of
https://github.com/RightNow-AI/openfang.git
synced 2026-08-14 08:52:02 +00:00
guatoc-ecohubandMiguel Guerrero
92f7e996de
Adds an optional `audio_base_url` field to `MediaConfig` that overrides the hardcoded provider URLs in `media_understanding::transcribe_audio`, allowing the same OpenAI-compatible multipart wire format to be sent to a local Whisper service (speaches, faster-whisper-server, LM Studio, etc.) instead of api.openai.com / api.groq.com. Closes #1051. ## Why Self-hosted, sovereignty-conscious, or rate-limited deployments often need to route audio transcription to a local Whisper backend while keeping `media_transcribe` / `speech_to_text` working as native tools (no helper scripts, no shell_exec workarounds). Today the URLs in `media_understanding.rs:118-128` are literal `&'static str` so neither `OPENAI_BASE_URL` nor `provider_urls` (which the LLM drivers do respect) is read for audio. The same problem existed for embeddings and was already addressable via `provider_urls`, so this change keeps the pattern symmetric for media at the simplest possible surface area. ## Wire format The endpoint shape and Authorization header remain identical: POST <audio_base_url>/v1/audio/transcriptions Authorization: Bearer $<provider>_API_KEY Content-Type: multipart/form-data fields: file (binary), model, response_format=text This means **any OpenAI-compatible Whisper server is drop-in** (Speaches, faster-whisper-server, LM Studio's Whisper server, etc.). Local servers typically accept any non-empty bearer string, so users can keep `OPENAI_API_KEY=anything` for the auth header. ## Configuration ```toml [media] audio_provider = "openai" audio_base_url = "http://127.0.0.1:8000" # → POST http://127.0.0.1:8000/v1/audio/transcriptions ``` Or for Groq-compatible local servers: ```toml [media] audio_provider = "groq" audio_base_url = "http://127.0.0.1:9000" # → POST http://127.0.0.1:9000/v1/audio/transcriptions ``` Trailing slash on the user-supplied base is stripped to avoid double slashes in the final URL. ## Backward compatibility - `MediaConfig` already uses `#[serde(default)]`, so existing configs without `audio_base_url` deserialize as `None` and behave exactly as before (cloud provider URLs). - `Default` impl extended; `audio_base_url: None`. - `parakeet-mlx` provider path unaffected (it's a separate code branch). - No new dependencies, no breaking changes to public API. ## Tests - `test_media_config_default` extended to assert `audio_base_url.is_none()`. - `test_media_config_audio_base_url_serde_roundtrip` — set + JSON roundtrip. - `test_media_config_backward_compat_no_audio_base_url` — legacy JSON parses with the new field as None. - `test_audio_base_url_override_logic` — pure-function test that exercises the URL building branch (default URLs preserved when unset, override applied for both providers, trailing-slash strip). The runtime branch in `transcribe_audio` was kept as a straight `if Some/else default` rather than a helper function to minimize the diff and keep the patch obviously safe to review. ## Operational note This change does not affect anyone running the cloud provider URLs out of the box. The override is opt-in via a single optional config field. Useful for users like myself running a local Speaches container behind a reverse proxy and a chat-only LLM key (z.ai Coding Plan) that can't satisfy openai.com's audio endpoint. Linked: #1051 (Configurable STT/TTS/image URLs and local backends). Co-authored-by: Miguel Guerrero <kortux@gmail.com>