How voice AI agents control your browser @googlecloudtech
How voice AI agents control your browser  @googlecloudtech
Uploaded October 2026 | Updated November 2026, 17 hours ago
Code for this episode → https://g.dev/cloud/repositories
Gemini Live API docs → https://g.dev/cloud/gemini-live
CDP Page commands → https://g.dev/cloud/devtools-protocol

How does an AI voice agent control a real browser while keeping the conversation going?

This episode explains a design pattern for live voice agents: run suitable browser actions in the background, then use the next screenshot to observe the result. Keep read operations synchronous when the model needs their returned data to decide what happens next.

We cover the screenshot, reason, act loop, Chrome DevTools Protocol, tool classification through a callback, and server-side confirmation for consequential actions.

Chapters:
00:00 - Voice-Controlled Browser Demo
00:12 - Giving a Live Voice Agent Browser Control
00:46 - The Browser Control Loop
01:27 - From Model Decisions to Clicks: Chrome DevTools Protocol
01:57 - Why This Agent Uses Screenshots
02:43 - Keeping Slow Tools from Stopping the Conversation
03:45 - Async Actions vs. Synchronous Reads
04:11 - Classifying Tools with a Callback
04:36 - Human Confirmation and Server-Side Guardrails
05:25 - Next Episode: Long-Term Agent Memory


More resources:
CDP Input commands → https://g.dev/cloud/page-devtools-protocol
CDP Page commands → https://g.dev/cloud/input-devtools-protocol
ADK tool callbacks → https://g.dev/cloud/types-of-callbacks

Follow Annie online:
LinkedIn → https://goo.gle/annie-linkedin
X → https://goo.gle/annie-x

Watch more The Omni App → https://g.dev/cloud/modern-ai-app
🔔 Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech

#AIAgents #BrowserAutomation #ToolCalling #MultimodalAI

Speaker: Annie Wang
Products Mentioned: Gemini Live API, Agent Development Kit, Chrome
How voice AI agents control your browserWhy vibe coding fixed her imposter syndrome | The Agent FactoryLaunch your portfolio website with AIMore AI agents = better results?Cloud Run vs.Compute Engine: When should you run a container on serverless vs. managing a full VM?Gemini 4 Argon demo: Remixing 3D apps & Firebase in Google AI StudioBuild a real-time voice AI agent with Google ADK and Gemini Live APIPrevent surprise cloud bills: enforce hard spending caps on Gemini API & Vertex AINetwork Security Integration with Check Point - DemoHow I built Battle Chess in one weekend… with no game dev experienceHow to build and scale multi-agent AI systems on GKE
Google Cloud Tech |

How voice AI agents control your browser

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER