Uploaded October 2024 | Updated September 2026, 8 minutes ago

---

Hey everyone! Thank you so much for watching the 107th episode of the Weaviate Podcast! This one dives into SWE-bench, SWE-agent, and most recently SWE-bench Multimodal with John Yang from Stanford University and Carlos E. Jimenez from Princeton University! One of the most impactful applications of AI we have seen so far is in programming and software engineering! John, Carlos, and team are at the cutting-edge of developing and benchmarking these systems! I learned so much from the conversation and I really hope you find it interesting and useful as well!

Chapters
0:00 Welcome John and Carlos!
1:03 How SWE-bench began
2:32 The Data Problem
7:50 SWE-agent Interface and Tools
14:40 What’s missing in Agents + IDEs?
17:05 Reviewing Pull Requests from AI Agents
22:40 Using external libraries
26:00 Code Execution as a Tool
30:10 GPU Computing in SWE Agents
33:30 Designing SWE Agents
43:40 SWE-bench Multimodal
55:05 Exciting directions for the future of AI

Links:

SWE-bench: swebench.com
SWE-agent: github.com/princeton-nlp/SWE-agent
SWE-bench Multimodal: swebench.com/multimodal.html