Uploaded July 2026 | Updated September 2026, 3 weeks ago
Almost every agent today is built for one user, and Sai Krishna Rallabandi has spent about eight months on what breaks when you drop one into a group chat instead. Running a personal agent across a real group, and eventually onto glasses, forces two hard problems. The first is memory: conversations that last for days across several people, where the agent has to track who said what and surface only what is relevant without bloating its context.
The second is security, and it is where the talk digs in. He points to a finding that two skills, each safe on its own, can collide once they share infrastructure, so a reporting agent leaks information at low frequency. His defense is to have an agent read everything and design a guard, then fine tune a small model with a per user adapter that only releases information appropriate to the context and catches prompt injection that regex would miss. The through line is that group and wearable settings break the single user assumptions most agent harnesses are built on, so memory and security have to be redesigned around the group.
Speaker info:
- https://x.com/Saikallis9012
- linkedin.com/in/sai-krishna-rallabandi-8595418b
- saikrishnarallabandi.github.io
Timestamps:
0:00 - Introduction: agents for groups, not one user
1:28 - Why single user agents fall short
2:45 - Eight months in a real group chat
3:47 - What changes when the agent joins a group
5:17 - Two problems: guarding and memory
5:55 - Securing an agentic system
7:44 - When two safe skills collide
9:27 - Designing a guard agent
10:21 - A small model with per user adapters
11:49 - Catching prompt injection
12:29 - Designing memory for groups
15:29 - Context growth and token cost
Almost every agent today is built for one user, and Sai Krishna Rallabandi has spent about eight months on what breaks when you drop one into a group chat instead. Running a personal agent across a real group, and eventually onto glasses, forces two hard problems. The first is memory: conversations that last for days across several people, where the agent has to track who said what and surface only what is relevant without bloating its context.
The second is security, and it is where the talk digs in. He points to a finding that two skills, each safe on its own, can collide once they share infrastructure, so a reporting agent leaks information at low frequency. His defense is to have an agent read everything and design a guard, then fine tune a small model with a per user adapter that only releases information appropriate to the context and catches prompt injection that regex would miss. The through line is that group and wearable settings break the single user assumptions most agent harnesses are built on, so memory and security have to be redesigned around the group.
Speaker info:
- https://x.com/Saikallis9012
- linkedin.com/in/sai-krishna-rallabandi-8595418b
- saikrishnarallabandi.github.io
Timestamps:
0:00 - Introduction: agents for groups, not one user
1:28 - Why single user agents fall short
2:45 - Eight months in a real group chat
3:47 - What changes when the agent joins a group
5:17 - Two problems: guarding and memory
5:55 - Securing an agentic system
7:44 - When two safe skills collide
9:27 - Designing a guard agent
10:21 - A small model with per user adapters
11:49 - Catching prompt injection
12:29 - Designing memory for groups
15:29 - Context growth and token cost
