Voice UX improvements

- Add text_utils.py: Clean markdown/hashtags/URLs before TTS - No more 'hash hash' or 'asterisk asterisk' - Strips code blocks, links, emojis - Converts bullets to spoken form - Markdown rendering in UI: Display formatted text while speaking - Code blocks, bold, italic, links styled - Faster TTS: Switch to eleven_turbo_v2_5 (~2x faster than v1) - UI styling for markdown elements
2026-08-14 09:02:28 +00:00 · 2026-02-01 15:34:55 -05:00
parent 1c6d083e5c
commit 0f29ecf188
5 changed files with 244 additions and 6 deletions
@@ -0,0 +1,90 @@
+# OpenClaw Voice: Talk to Your AI Like You Talk to Alexa
+
+**Free. Open-source. Self-hosted. No subscriptions.**
+
+---
+
+## The Problem
+
+Voice AI is everywhere—Alexa, Siri, Google Assistant—but they're all locked ecosystems. Want to talk to YOUR AI? The one you've customized? The one with YOUR context?
+
+You've got two options:
+1. **Pay $0.08-0.15/minute** for hosted voice AI (ElevenLabs Agents, Retell.ai)
+2. **Spend 3 days** configuring WebRTC, Whisper, VAD, and a dozen other acronyms
+
+Neither is great.
+
+## The Solution
+
+**OpenClaw Voice** is a browser-based voice interface you can self-host in 5 minutes.
+
+- 🎤 **Local STT** — Whisper runs on YOUR machine. Your voice never leaves your server.
+- 🔊 **Premium TTS** — ElevenLabs integration for natural speech.
+- 🌐 **Works in any browser** — Desktop, mobile, no app install.
+- 🔌 **Connect any AI** — OpenAI, Claude, or your own custom agent.
+- 🏠 **100% self-hosted** — Your data stays yours.
+
+## How It Works
+
+```
+Browser → WebSocket → [Whisper STT] → [Your AI] → [ElevenLabs TTS] → Browser
+```
+
+That's it. Your voice gets transcribed locally, sent to your AI, and the response comes back as speech.
+
+## Quick Start
+
+If you're technical (or have an AI assistant that is):
+
+```bash
+git clone https://github.com/Purple-Horizons/openclaw-voice.git
+cd openclaw-voice
+python3 -m venv .venv && source .venv/bin/activate
+pip install -r requirements.txt
+ELEVENLABS_API_KEY="your-key" OPENAI_API_KEY="your-key" \
+  python -m src.server.main
+```
+
+Open http://localhost:8765. Start talking.
+
+For mobile (HTTPS required), use Tailscale Funnel:
+```bash
+tailscale funnel 8765
+```
+
+## For OpenClaw Users
+
+If you're already running OpenClaw, you can connect Voice directly to your agent. Same context, same memory, same tools—just voice.
+
+Add to your `openclaw.json`:
+```json
+{
+  "gateway": {
+    "http": {
+      "endpoints": {
+        "chatCompletions": { "enabled": true }
+      }
+    }
+  }
+}
+```
+
+Now your voice conversations route through your full agent.
+
+## Why Open Source?
+
+Because voice AI shouldn't be a subscription. 
+
+The models exist. The tools exist. The only thing missing was someone putting them together in a way that doesn't require a PhD in audio engineering.
+
+OpenClaw Voice is MIT licensed. Fork it. Modify it. Run it on a Raspberry Pi. We don't care. Just build cool stuff.
+
+## Links
+
+- 🦞 **Website:** [openclawvoice.com](https://openclawvoice.com)
+- 📦 **GitHub:** [github.com/Purple-Horizons/openclaw-voice](https://github.com/Purple-Horizons/openclaw-voice)
+- 🔧 **OpenClaw:** [openclaw.ai](https://openclaw.ai)
+
+---
+
+*Built by [Purple Horizons](https://purplehorizons.io) in Miami. Part of the OpenClaw ecosystem.*
@@ -184,6 +184,31 @@
            color: #4fc3f7;
        }
        
+        /* Markdown styling */
+        .transcript-box code {
+            background: rgba(255, 255, 255, 0.1);
+            padding: 2px 6px;
+            border-radius: 4px;
+            font-family: 'SF Mono', Monaco, monospace;
+            font-size: 0.9em;
+        }
+        
+        .transcript-box code.block {
+            display: block;
+            padding: 10px;
+            margin: 8px 0;
+            background: rgba(0, 0, 0, 0.3);
+        }
+        
+        .transcript-box a {
+            color: #ff6b35;
+            text-decoration: none;
+        }
+        
+        .transcript-box a:hover {
+            text-decoration: underline;
+        }
+        
        .controls {
            display: flex;
            gap: 10px;
@@ -383,11 +408,38 @@
        function addTranscript(speaker, text, className) {
            const p = document.createElement('p');
            p.className = className;
-            p.innerHTML = `<strong>${speaker}:</strong> ${text}`;
+            // Render markdown for assistant responses
+            const renderedText = className === 'assistant' ? renderMarkdown(text) : escapeHtml(text);
+            p.innerHTML = `<strong>${speaker}:</strong> ${renderedText}`;
            transcriptEl.appendChild(p);
            transcriptEl.scrollTop = transcriptEl.scrollHeight;
        }
        
+        function escapeHtml(text) {
+            const div = document.createElement('div');
+            div.textContent = text;
+            return div.innerHTML;
+        }
+        
+        function renderMarkdown(text) {
+            // Simple markdown renderer
+            return text
+                // Code blocks
+                .replace(/```[\s\S]*?```/g, '<code class="block">code</code>')
+                // Inline code
+                .replace(/`([^`]+)`/g, '<code>$1</code>')
+                // Bold
+                .replace(/\*\*([^*]+)\*\*/g, '<strong>$1</strong>')
+                // Italic
+                .replace(/\*([^*]+)\*/g, '<em>$1</em>')
+                // Headers (convert to bold)
+                .replace(/^#{1,3}\s+(.+)$/gm, '<strong>$1</strong>')
+                // Links
+                .replace(/\[([^\]]+)\]\(([^)]+)\)/g, '<a href="$2" target="_blank">$1</a>')
+                // Line breaks
+                .replace(/\n/g, '<br>');
+        }
+        
        // Audio recording
        async function startRecording() {
            if (isRecording) return;
@@ -28,6 +28,7 @@ from .tts import ChatterboxTTS
 from .backend import AIBackend
 from .vad import VoiceActivityDetector
 from .auth import token_manager, load_keys_from_env, APIKey
+from .text_utils import clean_for_speech


 class Settings(BaseSettings):
@@ -282,15 +283,20 @@ async def websocket_endpoint(websocket: WebSocket):
                        logger.debug("Getting AI response...")
                        response_text = await backend.chat(transcript)
                        
+                        # Send original text for display (with markdown)
                        await websocket.send_json({
                            "type": "response_text",
                            "text": response_text,
                        })
                        logger.info(f"Response: {response_text}")
                        
-                        # Generate speech
+                        # Clean text for TTS (remove markdown, hashtags, etc.)
+                        speech_text = clean_for_speech(response_text)
+                        logger.debug(f"Speech text: {speech_text}")
+                        
+                        # Generate speech from cleaned text
                        logger.debug("Generating speech...")
-                        audio_response = await tts.synthesize(response_text)
+                        audio_response = await tts.synthesize(speech_text)
                        
                        # Send audio back
                        audio_b64 = base64.b64encode(audio_response.tobytes()).decode()
@@ -298,7 +304,7 @@ async def websocket_endpoint(websocket: WebSocket):
                            "type": "audio_response",
                            "data": audio_b64,
                            "sample_rate": 24000,  # TTS output rate
-                            "text": response_text,
+                            "text": response_text,  # Original for display
                        })
                
                audio_buffer = []
@@ -0,0 +1,90 @@
+"""
+Text utilities for voice-friendly output.
+
+Cleans AI responses for TTS (removes markdown, hashtags, etc.)
+while preserving original for display.
+"""
+
+import re
+
+
+def clean_for_speech(text: str) -> str:
+    """
+    Clean text for TTS rendering.
+    
+    Removes:
+    - Markdown formatting (**, *, #, ```, etc.)
+    - Hashtags (#word)
+    - URLs
+    - Emojis
+    - Multiple spaces/newlines
+    
+    Converts:
+    - Bullet points to spoken equivalents
+    - Numbers with context
+    """
+    if not text:
+        return text
+    
+    # Remove code blocks first (``` ... ```)
+    text = re.sub(r'```[\s\S]*?```', ' code block omitted ', text)
+    
+    # Remove inline code (`...`)
+    text = re.sub(r'`([^`]+)`', r'\1', text)
+    
+    # Remove markdown headers (# ## ###)
+    text = re.sub(r'^#{1,6}\s*', '', text, flags=re.MULTILINE)
+    
+    # Remove bold/italic markers
+    text = re.sub(r'\*\*([^*]+)\*\*', r'\1', text)  # **bold**
+    text = re.sub(r'\*([^*]+)\*', r'\1', text)       # *italic*
+    text = re.sub(r'__([^_]+)__', r'\1', text)       # __bold__
+    text = re.sub(r'_([^_]+)_', r'\1', text)         # _italic_
+    
+    # Remove hashtags (but keep the word)
+    text = re.sub(r'#(\w+)', r'\1', text)
+    
+    # Remove URLs
+    text = re.sub(r'https?://\S+', '', text)
+    
+    # Remove markdown links [text](url) -> text
+    text = re.sub(r'\[([^\]]+)\]\([^)]+\)', r'\1', text)
+    
+    # Remove common emojis (keep some expressive ones?)
+    # For now, remove most technical emojis
+    text = re.sub(r'[🔗📦📁💻🖥️⚡🔧🛠️📝✅❌⚠️🚀🎯💡🔍📊📈📉🗂️📋]', '', text)
+    
+    # Convert bullet points to spoken form
+    text = re.sub(r'^\s*[-•]\s*', 'Next, ', text, flags=re.MULTILINE)
+    text = re.sub(r'^\s*\d+\.\s*', '', text, flags=re.MULTILINE)  # Remove numbered lists
+    
+    # Clean up multiple newlines
+    text = re.sub(r'\n{2,}', '. ', text)
+    text = re.sub(r'\n', ' ', text)
+    
+    # Clean up multiple spaces
+    text = re.sub(r'\s{2,}', ' ', text)
+    
+    # Remove leading/trailing whitespace
+    text = text.strip()
+    
+    # Don't end with "Next, "
+    if text.endswith('Next,'):
+        text = text[:-5].strip()
+    
+    return text
+
+
+def estimate_speech_duration(text: str, wpm: int = 150) -> float:
+    """
+    Estimate speech duration in seconds.
+    
+    Args:
+        text: Text to speak
+        wpm: Words per minute (default 150 for natural speech)
+    
+    Returns:
+        Estimated duration in seconds
+    """
+    word_count = len(text.split())
+    return (word_count / wpm) * 60
@@ -106,11 +106,11 @@ class ChatterboxTTS:
        """Synchronous synthesis."""
        if self._backend == "elevenlabs":
            try:
-                # Generate audio with ElevenLabs
+                # Generate audio with ElevenLabs (turbo model for speed)
                audio_generator = self._elevenlabs_client.text_to_speech.convert(
                    voice_id=self.voice_id,
                    text=text,
-                    model_id="eleven_monolingual_v1",
+                    model_id="eleven_turbo_v2_5",  # Fastest model (~2x faster)
                    output_format="pcm_24000",  # 24kHz PCM (matches server expectation)
                )
                # Collect all chunks