Uploaded May 2025 | Updated November 2026, 17 hours ago
[2025 - Day 3 - Lightning Talks] Tomas Kofman shares insights from building model routers that optimize LLM selection, exploring how to improve accuracy up to 25% while reducing costs up to 90% through intelligent routing methodologies. For engineers working with multi-model systems, this talk offers valuable perspectives on evaluation techniques and scalable implementation strategies to overcome common technical hurdles in deploying cost-effective, high-performance AI applications.
ABOUT THE SPEAKER:
Tomas Kofman, Co-Founder & CEO, Not Diamond -
๐๏ธ GET YOUR TICKET TO AI COUNCIL 2026 ๐๏ธ
Meet the world's top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
โ aicouncil.com/sf-2026
โก FIND US:
X: https://x.com/AICouncilConf
LinkedIn: linkedin.com/company/aicouncilconf
Website: aicouncil.com
[2025 - Day 3 - Lightning Talks] Tomas Kofman shares insights from building model routers that optimize LLM selection, exploring how to improve accuracy up to 25% while reducing costs up to 90% through intelligent routing methodologies. For engineers working with multi-model systems, this talk offers valuable perspectives on evaluation techniques and scalable implementation strategies to overcome common technical hurdles in deploying cost-effective, high-performance AI applications.
ABOUT THE SPEAKER:
Tomas Kofman, Co-Founder & CEO, Not Diamond -
๐๏ธ GET YOUR TICKET TO AI COUNCIL 2026 ๐๏ธ
Meet the world's top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
โ aicouncil.com/sf-2026
โก FIND US:
X: https://x.com/AICouncilConf
LinkedIn: linkedin.com/company/aicouncilconf
Website: aicouncil.com
![Failure is a Funnel
[2025 - Day 1 - Data Science & Algos] Bryan Bischof shares insights from systematic approaches to evaluating and improving LLM performance, exploring practical frameworks for transforming experimental models into production-ready systems. This talk offers valuable perspectives on identifying failure patterns, establishing reliable production environments, and moving from experimental slop to scalable production systems, essential for teams assessing LLM quality or building robust deployment pipelines.
ABOUT THE SPEAKER:
Bryan Bischof, Head of AI, Theory Ventures -
๐๏ธ GET YOUR TICKET TO AI COUNCIL 2026 ๐๏ธ
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
โ https://aicouncil.com/sf-2026
โก FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ Failure is a Funnel](https://i.ytimg.com/vi/k98gDjYbSaU/mqdefault.jpg)
![AI Launchpad 2025: Quesma
[2025 - Day 1 - AI Launchpad] Jacek Migdal and Piotr Migdal share insights from Quesmas database gateway for smarter queries, exploring how to tackle the legacy and underbuilt side of data ecosystems through interoperability layers. Drawing from decades of experience with enterprise data transformations and modernization projects that save millions annually, this session offers valuable perspectives on untangling complex data platforms and bringing order to chaos without greenfield assumptions.
ABOUT THE SPEAKERS:
Jacek Migdal, Co-Founder & CEO, Quesma
Piotr Migdal, Founding Engineer, Quesma -
๐๏ธ GET YOUR TICKET TO AI COUNCIL 2026 ๐๏ธ
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
โ https://aicouncil.com/sf-2026
โก FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ AI Launchpad 2025: Quesma](https://i.ytimg.com/vi/kK_YHYe5ReE/mqdefault.jpg)

![Inference for Async Agents in Production | Doubleword
[2026 - Day 1 - AGENT INFRASTRUCTURE] As models improve, we are starting to build long-running, asynchronous agents such as deep research agents and browser agents that can execute multi-step workflows autonomously. These systems unlock new use cases, but they use orders of magnitude more tokens and compute, creating scaling bottlenecks.
This talk discusses practical strategies builders can use to maximize async agent performance while keeping inference costs under control. Topics covered include context engineering, compaction, cache maintenance, model routing, and batch inference. This talk is aimed at use case developers, with secondary relevance to platform engineers.
SPEAKER:
Meryem Arik - Co-founder & CEO, Doubleword
๐ Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf Inference for Async Agents in Production | Doubleword](https://i.ytimg.com/vi/kfn0khygHYQ/mqdefault.jpg)
![Context Engineering at the Frontier | Thrive Capital
[2026 - DAY 1 - AGENT INFRASTRUCTURE] As models become more capable and reliable, itโs more important than ever to design tools and context thoughtfully. When models had short context windows and low agency, basic document retrieval into the modelโs context window made a dramatic difference; but as frontier models boast millions of tokens of context and run for minutes and hours through tool calls and cycles of compaction, there is an ever growing list of concerns all vying for our modelsโ scarce attention. Extending context windows is an expensive and incomplete workaround.
In this talk, we will share some of the principles and techniques we found useful in navigating these problems ourselves as we worked on the goal of improving our assistant, Puck, from a simple retrieval-based chatbot to a deeply knowledgeable general assistant. In particular, we will touch on two prevailing challenges. First, weโll share how weโve sought to raise the signal-to-noise ratio in Puckโs context by thinking of context engineering itself as a search problem, involving every step of the pipeline from indexing to subagents. Second, weโll share how Puck approaches blending faithful, up-to-date structured data queries with the richness, breadth, incompleteness, and frequent conflicts latent in unstructured data in production. Weโll walk through a few concrete tactics in detail within both of these pillars, demonstrating that creative and useful approaches often come when we stop thinking of databases, search indexes, tools, and subagents as separate components, but as different solutions to the same underlying, age-old problem: searching for signal in a confusing and noisy world.
SPEAKER:
Linus Lee - Head of AI, Thrive Capital
๐ Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf Context Engineering at the Frontier | Thrive Capital](https://i.ytimg.com/vi/knUetkjGbpw/mqdefault.jpg)
![The Agentic Database: A New Way to Interact With Your Data
[2025 - Day 3 - Databases] Etienne Dilocker shares insights from exploring the paradigm shift beyond traditional CRUD operations, examining how AI agents are transforming database interactions from structured queries to natural language conversations. This talk offers valuable perspectives on the future of database interaction where you can communicate with your data as if it were a co-worker, essential for teams working with vector databases or building agent-based systems.
ABOUT THE SPEAKER:
Etienne Dilocker, CTO, Weaviate -
๐๏ธ GET YOUR TICKET TO AI COUNCIL 2026 ๐๏ธ
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
โ https://aicouncil.com/sf-2026
โก FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ The Agentic Database: A New Way to Interact With Your Data](https://i.ytimg.com/vi/kxy0Dofm3DI/mqdefault.jpg)
![Enabling Highly Autonomous Trusted Agents
[2026 - DAY 3 - AI SECURITY & SAFETY] As AI agents become more autonomous, organizations face both unprecedented opportunities and emerging risks. Organizations need trusted agents not only to drive productivity but also to defend against emerging AI-powered threats.
So why arenโt more agents in production? This panel unpacks the key technical, organizational, and trust barriers, and explains why success requires buy-in beyond security, spanning CIOs, developers, and product teams.
Weโll close with what it takes to run agents safely at scale: governance, monitoring, and controls that build trust in real-world deployments. As AI agents become more autonomous, organizations face both unprecedented opportunities and emerging risks. Organizations need trusted agents not only to drive productivity but also to defend against emerging AI-powered threats.
PANELISTS:
Heather Ceylan - CISO, Box
Diana Kelley - CISO, Noma Security
Feross Aboukhadijeh - CEO, Socket
Allie Howe - Host, Insecure Agents podcast (moderator)
๐ Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf Enabling Highly Autonomous Trusted Agents](https://i.ytimg.com/vi/l6aHWUfzk7E/mqdefault.jpg)
![Building Reliable Agentic AI Systems
[2025 - Day 1 - AI Engineering] Eno Reyes shares insights from agentic system design, exploring how to build reliable systems from stochastic components by drawing inspiration from robotics, cybernetics, and biology. This talk offers valuable perspectives on developing intuitions for non-deterministic decision making systems, particularly relevant for engineers working in this rapidly evolving field with huge potential for transforming software across industries.
ABOUT THE SPEAKER:
Eno Reyes, CTO, Factory -
๐๏ธ GET YOUR TICKET TO AI COUNCIL 2026 ๐๏ธ
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
โ https://aicouncil.com/sf-2026
โก FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ Building Reliable Agentic AI Systems](https://i.ytimg.com/vi/loYitFrgxe8/mqdefault.jpg)
![Recursively Self-Improving Agents as Autonomous Software Engineering | Factory
[2026 - DAY 3 - LIGHTNING TALK] How people use agents changes every day. Capabilities expand, use cases broaden, and the surface of workflows, stacks, codebases, and models keeps growing with them. To keep pace and ship the highest-performing product, the agent has to continuously learn without human intervention. At Factory, we close the loop between production behavior and improvements, and our agent Droid now ships its own fixes back into our codebase every day.
We will trace one full cycle of Droid improving itself, from detecting user friction in production to a Droid-authored PR, validated against our regression suites and merged. We will use the cycle to address key design questions: how to privately extract and cluster signal from sessions in aggregate, how to ensure quality as the agent and evals coevolve, and how to reduce human review burden as the loop scales.
The signal-to-fix loop is a general architecture pattern for autonomous software engineering: telemetry and logs for input, tests and evals for validation, merging for deployment, and monitoring for feedback. Any production AI system that can describe its own behavior, validate its own changes, and ship its own code compounds itself.
SPEAKER:
Abhay Singhal - Member of Technical Staff, Factory
๐ Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf Recursively Self-Improving Agents as Autonomous Software Engineering | Factory](https://i.ytimg.com/vi/m0VyJ4v-jrA/mqdefault.jpg)
![Eval Agents: How to Solve Error Cascades in Agents
[2025 - Day 1 - AI Engineering] Dhruv Singh shares insights from building multi-turn AI systems, exploring how errors compound and cascade in agents and RAG chatbots through real-world failure examples. For developers building interactive AI systems, this talk offers valuable perspectives on using eval agents tuned on real-world interactions to evaluate and improve agent performance, introducing the new world of trajectory evaluation needed for accurate agent assessment.
ABOUT THE SPEAKER:
Dhruv Singh, Co-Founder & CTO, HoneyHive AI -
๐๏ธ GET YOUR TICKET TO AI COUNCIL 2026 ๐๏ธ
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
โ https://aicouncil.com/sf-2026
โก FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ Eval Agents: How to Solve Error Cascades in Agents](https://i.ytimg.com/vi/mixuVOQRYOc/mqdefault.jpg)
![5 Lessons from the Classroom for Evaluating Agents | Dev Interrupted
[2026 - DAY 2 - ANALYTICS & DATA SCI] Education researchers have spent a century figuring out how to build environments where complex, unpredictable systems self-correct through structured feedback. Software engineers building AI agents are solving the exact same problem from scratch and ignoring all of it. This talk bridges that gap. Im a former classroom teacher turned AI engineer, and Ill walk you through five pedagogical frameworks that map directly to eval design patterns for AI agents: backward design (define success criteria before you build), formative assessment (eval continuously, not just at the end), rubric design (multi-dimensional scoring instead of pass/fail), error analysis (categorize failure modes because same symptom doesnt mean same cause), and differentiated feedback (the agent, the user, and the knowledge base each need their own signal channel). What ties them together is back pressure: each framework is a way to capture signal from problems and route it to where it drives change. Thats what makes a system self-correcting instead of just self-reporting. Most AI observability is still about watching systems after the fact. This talk is about designing systems where the eval layer captures back pressure from failures and feeds it back upstream, so the system iterates on itself. Ive built production agents and won hackathons with this approach, and the core insight is simple: the best eval systems arent tests, theyre environments. And nobody knows more about designing those environments than teachers.
SPEAKER:
Andrew Zigler - GTM Engineer, LinearB & Podcast Host, Dev Interrupted
๐ Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf 5 Lessons from the Classroom for Evaluating Agents | Dev Interrupted](https://i.ytimg.com/vi/mkuwepBWKsU/mqdefault.jpg)