How do voice AI agents work?

Voice AI agents handle phone conversations by chaining three systems: speech-to-text (STT) converts the caller's voice to text, an LLM processes the text and generates a response, and text-to-speech (TTS) converts that response back to natural-sounding voice.

The pipeline runs in real-time: microphone input → STT (50-200ms) → LLM reasoning (200-500ms) → TTS synthesis (100-300ms) → speaker output. Total latency: 350ms-1 second. Human conversational pauses are 200-500ms, so the agent feels natural when latency is under 500ms.

Platforms: Vapi (developer-first, good API), Retell AI (lowest latency, natural interruption handling), Bland AI (high-volume outbound calling), and custom builds using Twilio + Deepgram + OpenAI/Anthropic. Custom gives full control but costs 3-5× more to build.

Capabilities in 2026: natural turn-taking (agents detect when callers interrupt and pause appropriately), emotion detection (adjusts tone based on caller frustration), tool use during calls (look up account information, schedule appointments, process payments), and multi-language support (switch languages mid-call).

Cost: $0.05-$0.15 per minute of conversation. At 1,000 calls/month averaging 3 minutes, that's $150-$450/month in API costs plus platform fees.

Use cases with proven ROI: appointment scheduling (healthcare, dental, salons), lead qualification (real estate, insurance), customer service (order status, returns), and outbound campaigns (surveys, reminders, reactivation).