Uploaded July 2025 | Updated September 2026, 2 weeks ago
Most A/B testing platforms center around significance testing, but there's a growing trend towards Bayesian frameworks.
Joseph Powers, Principal Data Scientist at Intuit, explains why they replaced p-values with Bayesian risk, reducing test durations by 60% and aligning experimentation more directly with decision-making under uncertainty.
-----
π Sign up for our "No BS" Newsletter to get the latest technical data & AI content: datacouncil.ai/newsletter
ABOUT DATA COUNCIL:
Data Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Twitter: twitter.com/datacouncilai
LinkedIn: linkedin.com/company/datacouncil-ai
Website: datacouncil.ai
Most A/B testing platforms center around significance testing, but there's a growing trend towards Bayesian frameworks.
Joseph Powers, Principal Data Scientist at Intuit, explains why they replaced p-values with Bayesian risk, reducing test durations by 60% and aligning experimentation more directly with decision-making under uncertainty.
-----
π Sign up for our "No BS" Newsletter to get the latest technical data & AI content: datacouncil.ai/newsletter
ABOUT DATA COUNCIL:
Data Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Twitter: twitter.com/datacouncilai
LinkedIn: linkedin.com/company/datacouncil-ai
Website: datacouncil.ai

![Powering AI Workflows with Tabular Graphs
[2025 - Day 1 - Workshops] Rui Lopes shares insights from DataLinks semantic layer for AI systems, exploring entity-linking technology through intuitive data integration and API flexibility demonstrations. For developers seeking automated entity resolution and graph-based insights, this workshop offers valuable perspectives on simplifying complex data visualization, enabling natural language search over data, and implementing AutoRAG capabilities in applications.
ABOUT THE SPEAKER:
Rui Lopes, Head of AI, Datalinks -
ποΈ GET YOUR TICKET TO AI COUNCIL 2026 ποΈ
Meet the worlds top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
β https://aicouncil.com/sf-2026
β‘ FIND US:
X: https://x.com/AICouncilConf
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
Website: https://aicouncil.com/ Powering AI Workflows with Tabular Graphs](https://i.ytimg.com/vi/rl75GBTbSg8/mqdefault.jpg)

![Building Agentic RAG Systems with ClickHouse
[2026 - DAY 2 - WORKSHOP] This training walks step by step through a hands-on project using the Agentic Data Stack, all running in a single Docker Compose. Youβll explore how ClickHouse, MCP, LibreChat, and Langfuse work together in a ready-to-run setup for building agentic AI systems. Weβll focus on the end-to-end workflow with data ingestion and retrieval in ClickHouse to agent interaction and observability, using a practical, production-inspired project you can run locally. You donβt need to be a ClickHouse expert to follow along. Familiarity with RAG or agent-based systems is helpful, but the stack is designed to be approachable and easy to get started with.
Prerequisites: Attendees will need a ClickHouse cloud account or the Open source version on their laptops, either works, API key from Anthropic, and Docker.
SPEAKER:
Dustin Healy - Software Engineer, ClickHouse
π Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf Building Agentic RAG Systems with ClickHouse](https://i.ytimg.com/vi/s-yB8C0wd78/mqdefault.jpg)
![Towards Reliable Financial Agents: How a 4B Model Outsmarted a 235B Giant | Snorkel AI
[2026 - DAY 1 - WORKSHOP] Large generalist models have excellent reasoning but this does not necessarily imply specialized knowledge and tool calling capabilities. They can still hallucinate column names, ignore constraints, and generate SQL that returns nonsensical results. The problem isnβt intelligenceβitβs reliability and specialization.
In this talk weβll show how a 4B model was fine-tuned to outperform a 235B model on real financial analysis tasks. The key was not adding more reasoning ability, but enforcing tool discipline. Using synthetic data generation and reinforcement learning with the open-source rLLM framework, the model learned to explore schemas, validate outputs, and retry failures instead of hallucinating confident nonsense.
One key result: tool-use fundamentals generalize. Training on simple tool interactions transferred to much harder, multi-step financial tasks. If youβre building LLM systems that interact with databases, APIs, or internal tools, this talk focuses on the behaviors that actually matter β and how to teach them without frontier-scale compute.
SPEAKER:
Charles Dickens - Senior Research Scientist, Snorkel AI
π Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf Towards Reliable Financial Agents: How a 4B Model Outsmarted a 235B Giant | Snorkel AI](https://i.ytimg.com/vi/s3P2d6DfHxA/mqdefault.jpg)
![From Spans to Trajectories: Observability for Long-Running Agents | HoneyHive
[2026 - DAY 2 - AI ENGINEERING] Agents have evolved. Weve moved from orchestration frameworks β where agents operate through definite steps and turns you defined upfront β to harnesses, where the LLM uses skills and tools to chart its own trajectory. Modern agents run for hours or days, producing hundreds to thousands of steps in a single session. This calls for a fundamentally different methodology for monitoring and evaluating them in production.
This talk shares what weve learned building observability infrastructure for agent harnesses at HoneyHive. Well start with why the harness β not the model β has become the hardest engineering problem in production AI, and why traditional APM breaks down when traces are 10,000 spans deep and failures happen four tool calls deep. Well walk through a live trajectory view to see what long-running agent traces actually look like at scale, and the specific challenges they create: context rot, semantic failure modes, and the needle-in-a-haystack problem of finding the moment that mattered.
Then well dig into skills as the new unit of behavior and the dual role of clustering in agent development: unsupervised clustering for discovering emergent patterns and identifying where guardrails are needed, and supervised classifiers for production evaluation at scale. Well close on what comes next β swarm observability for multi-agent systems.
SPEAKER:
Sunny Bakhda - Founding Engineer, HoneyHive
π Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf From Spans to Trajectories: Observability for Long-Running Agents | HoneyHive](https://i.ytimg.com/vi/sPCdZO8-vNc/mqdefault.jpg)
![RLVR in Practice: From Synthetic Data to GRPO | NVIDIA
[2026 - DAY 3 - MODEL SYSTEMS] Reinforcement Learning from Verifiable Rewards (RLVR) is increasingly common in post-training pipelines, but the practical details are often glossed over. How do you design reward functions that programmatically verify model outputs? What makes synthetic training data effective? How do you build a custom RL environment that doesnt silently break your training?
SPEAKER: Chris Alexiuk - Product Research Engineer, NVIDIA
π Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf RLVR in Practice: From Synthetic Data to GRPO | NVIDIA](https://i.ytimg.com/vi/sVyZVtnygD8/mqdefault.jpg)
![Optimizing Model Training End-to-End: A Tiny MoE Case Study Lambda
[2026 - DAY 3 - MODEL SYSTEMS] Cloud compute is expensive, and wasting runs on the guise of a just scale will fix any problems leaves you with less time to fix errors, and less compute to train the model you want. In this talk, I will discuss what are the easy optimizatiosn you might miss (minimizing communications, using the most effective algorithms, ensuring youre getting the most FLOPs possible) at the small scale, before ensuring that when you do scale up nothing is going to waste. In this particular talk, Ill be focusing on what worked at home, that then let me scale it further onto the cloud.
SPEAKER: Zach Mueller - Head of Developer Relations, Lambda
π Sign up for our No BS Newsletter to get the latest technical data & AI content: https://aicouncil.com/newsletter
ABOUT AI COUNCIL:
AI Council brings together the brightest minds in data to share industry knowledge, technical architectures and best practices in building cutting edge data & AI systems and tools.
FIND US:
Website: https://aicouncil.com/
LinkedIn: https://www.linkedin.com/company/aicouncilconf/
X: https://x.com/aicouncilconf Optimizing Model Training End-to-End: A Tiny MoE Case Study Lambda](https://i.ytimg.com/vi/s_hSPBYQ3BA/mqdefault.jpg)


