Uploaded August 2026 | Updated September 2026, 2 weeks ago
Join a Microsoft’s very own Data Engineering team member on the SQL Server team on how their team uses Microsoft Fabric with Copilot CLI, VSCode Devcontainers, dbt (Data Build Tool) and more to remove coding barriers, automating repetitive tasks, have robust dev/test automation and enable platform engineers to focus on optimizing data stacks and architectures to unlock faster insights and greater value from data.
P.S. Ralph Wiggum from the Simpson’s TV show may also make a surprise appearance
🔗 This event is part of a series: https://aka.ms/DataDayslive/spark
0:00 Introduction and Speaker Overview
2:29 Agenda and AI-Native Data Engineering Overview
5:25 Modern Data Architecture (Vendor Agnostic)
6:42 Bronze, Silver, and Gold Data Zones
9:55 Introduction to dbt (Data Build Tool)
16:20 Declarative vs. Imperative Orchestration
20:30 Pillar of dbt: Don't Repeat Yourself (DRY)
21:46 Leveraging LLMs with dbt
26:10 Minimizing Custom Code with Spark
28:00 Spark Streaming and Incremental Processing
32:54 Local Development and Dev Containers
36:20 OneLake and Local Development Environment
39:35 "Ralph Wiggum" Loop Engineering Concept
43:08 Demo: Real-world Use Case in VS Code
51:46 Walkthrough of the Spark Diagnostic Silver Loader
58:05 Summary and Final Wrap-up
[eventID:27472]
Join a Microsoft’s very own Data Engineering team member on the SQL Server team on how their team uses Microsoft Fabric with Copilot CLI, VSCode Devcontainers, dbt (Data Build Tool) and more to remove coding barriers, automating repetitive tasks, have robust dev/test automation and enable platform engineers to focus on optimizing data stacks and architectures to unlock faster insights and greater value from data.
P.S. Ralph Wiggum from the Simpson’s TV show may also make a surprise appearance
🔗 This event is part of a series: https://aka.ms/DataDayslive/spark
0:00 Introduction and Speaker Overview
2:29 Agenda and AI-Native Data Engineering Overview
5:25 Modern Data Architecture (Vendor Agnostic)
6:42 Bronze, Silver, and Gold Data Zones
9:55 Introduction to dbt (Data Build Tool)
16:20 Declarative vs. Imperative Orchestration
20:30 Pillar of dbt: Don't Repeat Yourself (DRY)
21:46 Leveraging LLMs with dbt
26:10 Minimizing Custom Code with Spark
28:00 Spark Streaming and Incremental Processing
32:54 Local Development and Dev Containers
36:20 OneLake and Local Development Environment
39:35 "Ralph Wiggum" Loop Engineering Concept
43:08 Demo: Real-world Use Case in VS Code
51:46 Walkthrough of the Spark Diagnostic Silver Loader
58:05 Summary and Final Wrap-up
[eventID:27472]
