Uploaded March 2026 | Updated September 2026, 1 hour ago
Build a real-time AI voice assistant using Twilio Voice and Google Gemini. In this step-by-step tutorial, you’ll connect live phone calls to Gemini using Twilio ConversationRelay, FastAPI, and WebSockets — with incredibly low latency.
🚀 Full Source Code: github.com/rishabkumar7/twilio-cr-gemini-python
In this video, you’ll build a production-ready voice AI assistant that can answer real phone calls and maintain conversation context.
When someone calls your Twilio number:
• Twilio hits your /twiml endpoint
• TwiML opens a WebSocket connection
• Twilio streams speech transcripts in real time
• Your server sends prompts to Gemini
• Gemini generates a response
• Twilio converts it back to speech instantly
The result: a fully conversational AI assistant powered by Gemini 3 Flash.
What you’ll learn:
• How Twilio ConversationRelay enables low-latency AI calls
• How to generate and securely manage a Gemini API key
• How to configure a Twilio phone number for voice webhooks
• How to build a FastAPI WebSocket server
• How to maintain per-caller session memory using Call SID
• How to prevent memory leaks in high-volume call systems
Timestamps:
00:00 – Introduction & architecture overview
00:35 – End-to-end call flow explained
01:09 – Required tools (Twilio, Gemini, Enro/ngrok)
01:45 – Setting up FastAPI & dependencies
02:17 – Environment variables & system prompt
03:35 – Session management with Call SID
04:12 – TwiML & ConversationRelay breakdown
05:35 – WebSocket event handling
06:12 – Maintaining conversation context
06:56 – Cleanup & session teardown
07:36 – Exposing your server with Enro
08:56 – Configuring your Twilio number
10:11 – Live demo call with Gemini
11:17 – Context awareness & latency discussion
11:53 – Next steps & enhancements
Tech Stack:
Python
FastAPI
WebSockets
Twilio Voice
TwiML
Google Gemini API
ConversationRelay
From here, you can extend this assistant with tool calling, external APIs, CRM integrations, or custom TTS providers like ElevenLabs.
Subscribe to TwilioDevs for more hands-on AI, voice, and real-time communication tutorials.
💻 Sign up for a Twilio account here: twil.io/trytwilio-yt
👉 Subscribe for more Twilio updates: twil.io/sub-twiliodevs
#Twilio #GoogleGemini #VoiceAI #FastAPI #WebSockets
Build a real-time AI voice assistant using Twilio Voice and Google Gemini. In this step-by-step tutorial, you’ll connect live phone calls to Gemini using Twilio ConversationRelay, FastAPI, and WebSockets — with incredibly low latency.
🚀 Full Source Code: github.com/rishabkumar7/twilio-cr-gemini-python
In this video, you’ll build a production-ready voice AI assistant that can answer real phone calls and maintain conversation context.
When someone calls your Twilio number:
• Twilio hits your /twiml endpoint
• TwiML opens a WebSocket connection
• Twilio streams speech transcripts in real time
• Your server sends prompts to Gemini
• Gemini generates a response
• Twilio converts it back to speech instantly
The result: a fully conversational AI assistant powered by Gemini 3 Flash.
What you’ll learn:
• How Twilio ConversationRelay enables low-latency AI calls
• How to generate and securely manage a Gemini API key
• How to configure a Twilio phone number for voice webhooks
• How to build a FastAPI WebSocket server
• How to maintain per-caller session memory using Call SID
• How to prevent memory leaks in high-volume call systems
Timestamps:
00:00 – Introduction & architecture overview
00:35 – End-to-end call flow explained
01:09 – Required tools (Twilio, Gemini, Enro/ngrok)
01:45 – Setting up FastAPI & dependencies
02:17 – Environment variables & system prompt
03:35 – Session management with Call SID
04:12 – TwiML & ConversationRelay breakdown
05:35 – WebSocket event handling
06:12 – Maintaining conversation context
06:56 – Cleanup & session teardown
07:36 – Exposing your server with Enro
08:56 – Configuring your Twilio number
10:11 – Live demo call with Gemini
11:17 – Context awareness & latency discussion
11:53 – Next steps & enhancements
Tech Stack:
Python
FastAPI
WebSockets
Twilio Voice
TwiML
Google Gemini API
ConversationRelay
From here, you can extend this assistant with tool calling, external APIs, CRM integrations, or custom TTS providers like ElevenLabs.
Subscribe to TwilioDevs for more hands-on AI, voice, and real-time communication tutorials.
💻 Sign up for a Twilio account here: twil.io/trytwilio-yt
👉 Subscribe for more Twilio updates: twil.io/sub-twiliodevs
#Twilio #GoogleGemini #VoiceAI #FastAPI #WebSockets
