Apache Spark Upgrade Agent on EMR | Lets Talk About Data @AWSEventsChannel
Apache Spark Upgrade Agent on EMR | Lets Talk About Data  @AWSEventsChannel
Uploaded June 2026 | Updated November 2026, 10 hours ago
"Upgrading Apache Spark applications takes 3-6 months per major version. Troubleshooting production failures consumes
20-40% of data engineering time. These two workflows are the biggest time sinks teams face today.

In this session, we'll live-demo two AI-powered agents that compress these workflows from weeks to minutes. The Spark
Troubleshooting Agent diagnoses real production failures — executor OOM, Iceberg schema drift, YARN container kills —
by automatically correlating signals across driver logs, container diagnostics, and Spark event history. The Spark
Upgrade Agent takes a PySpark application from Spark 3.5 to 4.0 on EMR Serverless — iteratively fixing breaking
changes through runtime validation until the job succeeds, with data quality comparison at the end.

Everything happens through natural language. No manual log searching. No reading migration guides. Just describe the
problem and let the agent work."
Apache Spark Upgrade Agent on EMR | Lets Talk About DataAWS FSI NYC 2026 - Rocket fuels growth with unified customer data on AWSSupercharging DynamoDB with Bulk Executor: Large-Scale Operations Made SimpleWhats new with VPC Lattice | The Routing LoopFSI Sydney 2026: Keynote – Rethink EverythingAWS re:Invent 2025 - Automated cloud security remediation with agentic workflows (SEC217)NYC Executive Forum 2026 - A Fireside Chat with Swami SivasubramanianAWS Summit Johannesburg 2026 | AWS EventsAWS Summit Bengaluru 2026: Innovators Edition Keynote | AWS EventsAccelerating data workflows with AI Agents in your IDE | Lets Talk About DataDTW 2026 - Latest AWS AI Innovations as of H1 2026 | AWS EventsMexico City Executive Forum 2026 - Cuando la IA Decide: Autonomía con Rumbo de Negocio
AWS Events |

Apache Spark Upgrade Agent on EMR | Let's Talk About Data

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER