Loading Accelerated Logic…
GA Release

The Multi-Agent WebGPU Engine & Interactions API are now generally available.

Part 14: Voice Assistant & Audio Interaction

Use speech dictation, native Read Aloud, Kokoro 82M local voiceovers, and the full-screen Voice Interface.

Category: Getting Started • Read Time: 22 min read • Updated: 2026-11-10

1. Three Different Audio Features

Dictation, quick read-aloud, and generated voiceovers use different browser APIs and have different privacy and output behavior:

🎙️ Speech-to-Text Input (Dictation)

Dictate prompts hands-free using browser Web Speech Recognition. Transcribes spoken words directly into the composer input field.

🔊 Text-to-Speech Output (Read Aloud)

Listen to AI responses rendered via native browser Speech Synthesis voices with animated audio equalizer indicators.

✨ Local Kokoro Voiceovers

Generate downloadable WAV speech on your device with Kokoro 82M, then use it in chat or the Decks video timeline.

2. Speech-to-Text Voice Input

Click the Microphone icon in the composer when the input field is empty (or to append text).

Voice Dictation Workflow:

  1. Click the Microphone icon to open the full-screen Voice Interface.
  2. Grant browser microphone permissions when prompted.
  3. Speak naturally; live transcripts update in real time.
  4. Click Stop or pause briefly to complete dictation and insert text into the composer.

3. Text-to-Speech Read Aloud

Hover over any assistant message and click Read Aloud (Volume icon).

  • Cleaned Audio Output: System tags (<thinking>, <image_gen>) and code block syntax markers are stripped before synthesis.
  • Visual Equalizer: Animated soundwave bars indicate active speech synthesis. Click Stop Aloud anytime to silence playback.
  • Native OS Voices: Uses default system voices (e.g. Samantha/Alex on macOS, David/Zira on Windows).

4. Create a Voiceover from Chat

In chat, open the + menu and choose Voiceover, or ask for a voiceover in your own prompt. After the assistant finishes its reply, Kokoro creates a downloadable WAV card locally in your browser. Use Local Voice Studio when you want to paste and synthesize a script without asking the chat assistant. The first use downloads about 100 MB of model files; later sessions can reuse the browser cache.

  1. Choose an American or British English voice from the voiceover card.
  2. Preview, download, or regenerate the WAV in your browser.
  3. For a standalone script, open Local Voice Studio; for a video, insert speech at the playhead in Decks Studio.

The script is synthesized on your device. Model weights are downloaded from Hugging Face; the voice generator does not send the script to an AI API.

5. Full-Screen Voice Interface Modal

The full-screen Voice Interface modal features an animated microphone orb with pulsating radial waves, displaying live interim speech hypotheses for extended hands-free prompt composition.

6. Sandboxed Iframe Security & Audio Fallbacks

⚠️ "Read Aloud is blocked by sandboxed iframe" Notice

When Accelerated Logic AI runs inside a restricted preview iframe without explicit audio policy permissions, browsers block window.speechSynthesis.

Resolution: Click "Open in New Tab" at the top right of the workspace preview to run in a top-level browser context where Web Speech APIs operate unimpeded.

7. Hands-Free Developer Workflows

Dictate core feature requirements, allow multi-agent teams to generate application code, and listen to architectural explanations via Read Aloud while inspecting the live Web Sandbox preview pane.

8. Privacy & Data Handling

Speech Input (STT)

Processed by standard browser speech services (e.g. Google Cloud Speech in Chrome).

Speech Output (TTS)

Read Aloud uses browser speech voices. Chat voiceover cards and Local Voice Studio use Kokoro model files downloaded to the browser, then generate audio on-device. Chat prompts and replies sent through a connected cloud model are governed by that selected provider.

9. Voice Troubleshooting

Microphone Denied? Click the address bar lock icon → Site Settings → Microphone → Allow.

No Sound on Read Aloud? Ensure system audio is unmuted and verify that the current tab is not muted in the browser tab bar.

10. Choosing Between Dictation, Read Aloud, Chat Voiceovers, and Local Voice Studio

Feature What it does Where audio comes from Reach for it when
Dictation (mic icon)Speech-to-composer text inputBrowser speech servicesHands-free prompting, walking-and-thinking ideation
Read AloudSpeaks an assistant replyOS/browser native TTS voicesQuick auditory scan of a long answer; accessibility while multitasking
Chat Voiceover cardDownloadable WAV of Kokoro speech appended to a replyKokoro 82M on-deviceNarration/voiceover for short replies without leaving chat
Local Voice Studio (/local-tts)Full script → WAV pipeline with voice pickerKokoro 82M on-device1–12,000 character scripts, downloads, deck timeline insertion

Dictation and Read Aloud use browser/OS speech services (quality varies by platform); the Kokoro paths synthesize on-device after a one-time model download and never ship your script to an AI API.

11. Dictation Accuracy Playbook

  • Quiet room beats hardware. A cheap headset mic in a silent room outperforms a studio mic next to a fan. Background noise is the #1 dictation killer.
  • Speak punctuation explicitly for structure: "new paragraph", "comma", "question mark" — browser engines honor spoken punctuation commands far more reliably than they infer sentence boundaries.
  • Natural pace, not telegraph speech. Deliberate slow monotone confuses segmenters; speak at normal conversational speed with light emphasis.
  • Code is the weak spot. Dictated identifiers and brackets are error-prone — dictate the prose, then type the code tokens yourself, or paste code from the clipboard.
  • Always proof before send. Live interim transcripts can silently swap near-homophones (their/there, to/two); glance at the composer before hitting Enter.
  • Resume behavior: dictation sessions end on pause or Stop — text lands in the composer, not auto-sent, so you stay in charge of what goes out.

12. Where Kokoro Voiceovers Pay for Their Download

Async code review narration

Ask the assistant to summarize a PR diff, then request a voiceover of the summary for a teammate who prefers listening over reading — WAV card attaches to the same reply.

Deck narration track

Generate the script in chat, open Local Voice Studio, synthesize per-slide voiceovers, and drop each WAV at the timeline playhead in Decks (Part 21).

Accessibility cross-checks

Listening to your own marketplace text/docs read aloud catches rhythm and repetition problems your eyes skip over.

First-use reality: the ~100 MB Kokoro download happens once per browser profile. If storage was cleared (Part 1), the next synthesis re-downloads. English only; 24 kHz mono output WAV.

13. Audio Permissions & Playback — Troubleshooting Matrix

Symptom Likely cause Fix
Mic prompt never appearsPermission previously denied, or OS-level mic blockAddress bar lock → Site settings → Microphone → Allow; then OS privacy settings
Dictation Starts then instantly stopsAnother app holds the device, or speech service unreachableClose other mic apps; check browser speech service status; retry in a fresh tab
Read Aloud silentMuted tab, no OS voices installed, autoplay policyUnmute tab; install/select a system voice; click the button in a direct user gesture
Kokoro card never appears after replyFirst-time model download still running, or script >12,000 charsWatch network/progress; split long scripts; check DevTools console for model-load errors
WAV downloads but plays silentCorrupt download or zero-length generationRegenerate; check the script actually reached the card (empty input yields empty WAV)
Everything works outside iframes but not insideSandbox blocks media APIsOpen in New Tab — the standing fix for sandboxed previews (Sections 6 & 9)