How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, Coinbase @aiDotEngineer
How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, Coinbase  @aiDotEngineer
Uploaded October 2026 | Updated November 2026, 3 days ago
Evan Kormos, a player-coach at Coinbase with 15 years in enterprise financial services, lays out a practical way to bring agents into policy-driven, regulated workflows. He models processes as DAGs and loops, explains why a loop is only as trustworthy as its check, and shows how evals and traces become a new confidence mechanism. Agents then move through three stages: fully supervised, then sampled, then humans handling only exceptions.

Chapters:
- 00:00 Policy is the original system prompt
- 02:53 Where agents fit: people, process, technology
- 03:49 Workflows as DAGs
- 05:31 Inside a node: the loop
- 07:16 Exception flows and bottlenecks
- 08:59 Quality inputs, quality outputs
- 10:43 The loop is only as good as the check
- 11:47 Evals as a confidence mechanism
- 12:39 Which steps should an agent own?
- 14:29 Supervised, then sampled, then exceptions only
- 15:31 Takeaways

Evan Kormos — coinbase.com

Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!
How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, CoinbaseYou Cant Tell What an Agent Skill Will Do by Reading It - Roey Ben Chaim, ZenityGive Your On-Call Its Own AI Teammate - Swaapnika Motapothula, Chinmay Gandhi & Vlad Tsvang, MetaWhy AI Writing Still Reads Like Slop — Louis-François Bouchard, Towards AIHow a Trading Firm Ships New AI Models the Same Day - Noyan Tokgozoglu, OptiverWhy Frontier Models Fail as Buy-Side Analysts - Janie Gu, Google DeepMind & Suyeol Yun, LinqAlphaA Bank Built Its Own Coding Agent. It Took Off — Felippe Felisola Caso & Francesco Galletta, MonzoAI Engineer NYC Livestream | Bridgewater, Two Sigma, Arize, Coatue, Nous (Hermes), Gen. IntuitionDoes Code Quality Still Matter? It Depends on This — Max Kanat-Alexander, Capital OneStop Shipping LLM Settings That Only Win on Paper - Rohan Pandey & Balaji Varadarajan, DigitalOceanHow to Test AI Agents When No Benchmark Exists — Nathanael Weill, Air CanadaStop Giving Your Agent Tools. Give It a Sandbox - Mikyo King, Arize AI
AI Engineer |

How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, Coinbase

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER