Uploaded June 2026 | Updated November 2026, 4 days ago
[2026 - DAY 2 - AI ENGINEERING] Agents have evolved. We've moved from orchestration frameworks — where agents operate through definite steps and turns you defined upfront — to harnesses, where the LLM uses skills and tools to chart its own trajectory. Modern agents run for hours or days, producing hundreds to thousands of steps in a single session. This calls for a fundamentally different methodology for monitoring and evaluating them in production.
This talk shares what we've learned building observability infrastructure for agent harnesses at HoneyHive. We'll start with why the harness — not the model — has become the hardest engineering problem in production AI, and why traditional APM breaks down when traces are 10,000 spans deep and failures happen four tool calls deep. We'll walk through a live trajectory view to see what long-running agent traces actually look like at scale, and the specific challenges they create: context rot, semantic failure modes, and the needle-in-a-haystack problem of finding the moment that mattered.
Then we'll dig into skills as the new unit of behavior and the dual role of clustering in agent development: unsupervised clustering for discovering emergent patterns and identifying where guardrails are needed, and supervised classifiers for production evaluation at scale. We'll close on what comes next — swarm observability for multi-agent systems.
SPEAKER:
Sunny Bakhda - Founding Engineer, HoneyHive
👉 Sign up for our "No BS" Newsletter to get the latest technical data & AI content: aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: aicouncil.com
LinkedIn: linkedin.com/company/aicouncilconf
X: https://x.com/aicouncilconf
[2026 - DAY 2 - AI ENGINEERING] Agents have evolved. We've moved from orchestration frameworks — where agents operate through definite steps and turns you defined upfront — to harnesses, where the LLM uses skills and tools to chart its own trajectory. Modern agents run for hours or days, producing hundreds to thousands of steps in a single session. This calls for a fundamentally different methodology for monitoring and evaluating them in production.
This talk shares what we've learned building observability infrastructure for agent harnesses at HoneyHive. We'll start with why the harness — not the model — has become the hardest engineering problem in production AI, and why traditional APM breaks down when traces are 10,000 spans deep and failures happen four tool calls deep. We'll walk through a live trajectory view to see what long-running agent traces actually look like at scale, and the specific challenges they create: context rot, semantic failure modes, and the needle-in-a-haystack problem of finding the moment that mattered.
Then we'll dig into skills as the new unit of behavior and the dual role of clustering in agent development: unsupervised clustering for discovering emergent patterns and identifying where guardrails are needed, and supervised classifiers for production evaluation at scale. We'll close on what comes next — swarm observability for multi-agent systems.
SPEAKER:
Sunny Bakhda - Founding Engineer, HoneyHive
👉 Sign up for our "No BS" Newsletter to get the latest technical data & AI content: aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: aicouncil.com
LinkedIn: linkedin.com/company/aicouncilconf
X: https://x.com/aicouncilconf
![Building a Data Foundation for Multimodal Foundation Models
[2025 - Day 2 - Foundation Models] Ethan Rosenthal shares insights from building a petabyte-scale multimodal feature lakehouse at Runway, exploring the unique requirements of multimodal data curation beyond conventional machine learning systems. This talk offers valuable perspectives on balancing data quality with quantity and building systems that support both analytical querying and high-performance feature serving for foundation model training, crucial for engineers managing heterogeneous data or supporting large-scale distributed training jobs.
ABOUT THE SPEAKER:
Ethan Rosenthal, Technical Staff, Runway -
🎟️ GET YOUR TICKET TO AI COUNCIL 2026 🎟️
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
→ https://aicouncil.com/sf-2026
⚡ FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ Building a Data Foundation for Multimodal Foundation Models](https://i.ytimg.com/vi/6kf58xD5s3Q/mqdefault.jpg)
![Let’s Teach Claude Code Semantic Code Search With turbopuffer
[2026 - DAY 2 - WORKSHOP] Many AI code editors use semantic search to improve the quality of code retrieval and the performance of agent coding tasks. Cursor reports up to 23.5% higher accuracy in answering questions about the codebase and increased code retention by combining grep with semantic search.
Notably, Claude Code does not use semantic search in code retrieval. Let’s change that!
In this workshop, we’ll index our codebases on turbopuffer and give Claude Code (or any agent) a suite of code search tools – including semantic search, full-text search, and regex search. You’ll build a practical tool while learning how to maximize performance and recall on turbopuffer’s object-storage-native search database.
What you’ll learn:
- How to index code and other documents on turbopuffer for vector search, full-text search, regex search, and attribute filtering – optimizing for cost and performance
- Tips for improving search performance and recall on turbopuffer
- How to build code search tools for agents that reduce token consumption and improve agent code quality
Bring your own codebase, or index the Linux kernel.
SPEAKER:
Kuba Rogut - Deployed Engineer, turbopuffer
👉 Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf Let’s Teach Claude Code Semantic Code Search With turbopuffer](https://i.ytimg.com/vi/6t29UHJQpvA/mqdefault.jpg)


![Building LLM Applications with Llama Stack
[2025 - Day 2 - Foundation Models] Hardik Shah and Ashwin Bharambe and share insights from building production-grade LLM applications, exploring the unique challenges of non-deterministic systems and the specialized building blocks required for the full software development lifecycle. This talk offers valuable guidance on using Llama Stack to build, test, deploy, monitor, and improve LLM applications at scale, essential for teams developing locally or deploying across cloud, on-prem, or mobile environments.
ABOUT THE SPEAKERS:
Hardik Shah, AI Engineer, Meta
Ashwin Bharambe, Software Engineer, Meta -
🎟️ GET YOUR TICKET TO AI COUNCIL 2026 🎟️
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
→ https://aicouncil.com/sf-2026
⚡ FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ Building LLM Applications with Llama Stack](https://i.ytimg.com/vi/7vMeQMaOCyw/mqdefault.jpg)
![Building InfluxDB 3 Core: Real-Time Columnar DB & Data Processor On Object Storage
[2025 - Day 3 - Databases] Paul Dix shares insights from reimagining time series databases with InfluxDB 3 Cores ground-up Rust rewrite using Apache Arrow, DataFusion, and Parquet. For engineers building monitoring systems or handling real-time data ingestion, this session offers valuable perspectives on innovative diskless architecture that leverages object storage and transforms databases into comprehensive data collection and transformation platforms through embedded Python capabilities.
ABOUT THE SPEAKER:
Paul Dix, Founder & CTO, Influx Data -
🎟️ GET YOUR TICKET TO AI COUNCIL 2026 🎟️
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
→ https://aicouncil.com/sf-2026
⚡ FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ Building InfluxDB 3 Core: Real-Time Columnar DB & Data Processor On Object Storage](https://i.ytimg.com/vi/8A4RO3ruKfU/mqdefault.jpg)
![Designing Memory Systems for AI Agents | MongoDB
[2026 - DAY 2 - WORKSHOP] AI agents need memory to maintain context across sessions, learn from experience, and handle long-running tasks. The challenge? Deciding what to remember, where to store it, and how to retrieve it when it matters. In this workshop, youll learn a practical framework for architecting memory systems that actually work in production. Well cover:
- Types of memory in agentic systems
- Storage patterns: Where to persist memories and how to structure them for retrieval
- Retrieval strategies: Combining vector search with metadata, recency, and other signals
- Memory lifecycle: When to create, update, or prune memories to keep your system performant
Youll apply this framework by building memory into an AI agent and seeing how different design choices impact behavior.
SPEAKER:
Apoorva Joshi - Staff Developer Advocate, AI/ML, MongoDB
👉 Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf Designing Memory Systems for AI Agents | MongoDB](https://i.ytimg.com/vi/8DAVx8urNn8/mqdefault.jpg)
![Putting Data To Work For Global Urban Development
[2025 - Day 3 - Lightning Talks] Tobias Lunt shares insights from harnessing modern data and AI tooling to improve lives in the developing world, exploring how real-time urban data can inform smart policy and planning decisions. For data practitioners interested in public good applications, this talk offers valuable perspectives on combining incremental improvements with tectonic shifts like AI-assisted development and LLM toolchains to deliver outsized impact for global urban challenges.
ABOUT THE SPEAKER:
Tobias Lunt, Co-Founder & Data Scientist, Development Data Lab -
🎟️ GET YOUR TICKET TO AI COUNCIL 2026 🎟️
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
→ https://aicouncil.com/sf-2026
⚡ FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ Putting Data To Work For Global Urban Development](https://i.ytimg.com/vi/93cKg-YyBDk/mqdefault.jpg)
![More Than a Vibe: AI-Driven SQL That Actually Works
[2025 - Day 1 - Workshops] Jacob Matson shares insights from AI-powered spatial data analysis, exploring how to vibe code with AI-generated SQL using MotherDuck and DuckDB for real-world decision-making scenarios. This hands-on workshop offers valuable perspectives on iterative, low-risk workflows that transform data handling into rapid, interactive processes, particularly relevant for practitioners looking to confidently incorporate AI into their coding and analysis workflows.
ABOUT THE SPEAKER:
Jacob Matson, Developer Advocate, MotherDuck -
🎟️ GET YOUR TICKET TO AI COUNCIL 2026 🎟️
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
→ https://aicouncil.com/sf-2026
⚡ FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ More Than a Vibe: AI-Driven SQL That Actually Works](https://i.ytimg.com/vi/9KC1CU-5mU8/mqdefault.jpg)

![Building an LLM powered Analytics Slack Bot @ Twitch
[2025 - Day 3 - GenAI Applications] Ethan Brown shares insights from building an LLM-powered data analytics bot at Amazon IVS/Twitch Video, exploring how to augment data operations through Slack integration and familiar chat interfaces. Whether youre automating data workflows or building internal AI tools, this practical walkthrough offers valuable perspectives on implementing SQL query generation, chat summarization, and account lookups using standard AWS services.
ABOUT THE SPEAKER:
Ethan Brown, Data & Applied Science Director, Twitch / AWS -
🎟️ GET YOUR TICKET TO AI COUNCIL 2026 🎟️
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
→ https://aicouncil.com/sf-2026
⚡ FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ Building an LLM powered Analytics Slack Bot @ Twitch](https://i.ytimg.com/vi/9fUcnuDrwYU/mqdefault.jpg)