A reading list for builders, designers, and PMs trying to understand the shift from click-based UX to voice + intent-driven interaction.
The thesis in one line
Voice isn't just replacing typing — it's replacing clicking. Every dropdown, drag, and multi-step flow is collapsing into one sentence. The visuals stay. The way you operate them changes.
Start here (10-minute reads)
If you only have time for two, read these:
- OpenAI — Introducing gpt-realtime and Realtime API updates for production voice agents — the launch post that made voice agents production-ready.
- Google — Build real-time conversational agents with Gemini 3.1 Flash Live — Google's bet on voice + vision as one input.
Official documentation by provider
OpenAI
The infrastructure for voice agents that reason, call tools, and switch languages mid-sentence.
- Realtime and audio overview — start here for the mental model.
- Voice agents guide — building voice agents with the OpenAI Agents SDK.
- Audio and speech concepts — STT, TTS, speech-to-speech explained.
- Realtime models prompting guide — how to prompt voice agents differently from text ones.
- openai-realtime-agents on GitHub — working code patterns for multi-agent voice systems.
Google (Gemini Live)
Real-time voice + vision + screen-share in one session. 70+ languages. The model that talks while watching what you're doing.
- Gemini Live API overview — the main docs.
- Live API capabilities guide — affective dialog, proactive audio, VAD, tool use.
- gemini-live-api-examples on GitHub — sample apps.
Nvidia
The model layer underneath everything else. Nemotron speech models, full-duplex conversational AI, and PersonaPlex (listens and speaks simultaneously, like humans).
- Nvidia Riva — speech AI SDK — ASR, TTS, translation pipelines.
- Conversational AI overview — how Nemotron models fit together.
- Nemotron Voice Agent Blueprint — production-ready voice agent reference architecture.
- PersonaPlex research blog — full-duplex speech-to-speech, listens and speaks at the same time. Solves the "when to interrupt" problem.
- PersonaPlex on Hugging Face — open weights.
The generative UI layer
Voice is the input. Generative UI is what happens on screen when you speak. The agent assembles the UI around your intent.
- A2UI — Google's Agent-to-User Interface protocol — open protocol for agents to send UI structure (not HTML, not code) that your app renders natively.
- Introducing A2UI — Google Developers blog — the why and how.
- Prefab on GitHub — open-source generative UI framework for the Python/MCP ecosystem.
- The Developer's Guide to Generative UI in 2026 (CopilotKit) — the three patterns (static, declarative, open-ended) explained well.
Voice for non-English languages
The piece most US-centric coverage misses. Voice unlocks software for users who never typed comfortably in English.
- Wispr Flow — wisprflow.ai — dictates in Hinglish, Hindi, Tamil, Telugu. 4x faster than typing for knowledge workers.
- OpenAI Realtime translation — realtime translation docs — translates live speech across 70+ input languages.
- Gemini Live multilingual — supports 70 languages with native code-switching mid-sentence.
Where voice meets the pointer
Pointer + speech as one input. You don't have to describe what you're looking at — you point.
- DeepMind Magic Pointer — search "Google Magic Pointer demo" for the latest videos. Point at this, say do that, the cursor understands what you mean because it sees what you're pointing at.
Builders worth following
People shipping voice-first interfaces, not just talking about them.
- Farza (@FarzaTV on X) — building voice-enabled experiences and posting raw demos.
- ElevenLabs — voice infrastructure powering most consumer voice features you've used in the last 12 months. elevenlabs.io/docs.
- Daily.co / Pipecat — open-source voice agent framework. Building Voice Agents with Nvidia Open Models is a great hands-on read.
How to actually think about this as a designer / PM
Three questions to ask of every product you ship:
- What's the one-sentence version of the user's current 5-click flow? That's your voice command.
- What visuals does the user still need to see while operating the product by voice? Those stay. Everything else collapses.
- What's the user's first language? If your interface only works for English typers, you've already excluded the next 500M users.
What I'm building toward
Designing for voice-first interaction is the next 18 months of work for most product teams. The teams shipping for it now will look prescient. Everyone else will be playing catch-up. If you're working on something in this space, I'd love to see it. — @deepika.builds
