Uploaded November 2025 | Updated September 2026, 2 weeks ago

---

Vivek Yadav, an engineering manager from Stripe, shares his experience in building a testing system based on multi-year worth of data. He shares insights into why Apache Spark was the choice for creating such a system and how it fits in the "traditional" engineering practices.

Read a transcript of this interview: bit.ly/4o08NjD

Subscribe to the Software Architects’ Newsletter for your monthly guide to the essential news and experience from industry peers on emerging patterns and technologies:
infoq.com/software-architects-newsletter

Upcoming Events:

QCon AI New York 2025 (December 16-17, 2025)
ai.qconferences.com

QCon London 2026 (March 16-19, 2026)
QCon London equips senior engineers, architects, and technical leaders with trusted, practical insights to lead the change in software development. Get real-world solutions and leadership strategies from senior software practitioners defining current trends and solving today's toughest software challenges.
qconlondon.com

QCon AI Boston 2026 (June 1-2, 2026)
Learn how real teams are accelerating the entire software lifecycle with AI.
boston.qcon.ai


The InfoQ Podcasts:
Weekly inspiration to drive innovation and build great teams from senior software leaders. Listen to all our podcasts and read interview transcripts:

- The InfoQ Podcast infoq.com/podcasts
- Engineering Culture Podcast by InfoQ infoq.com/podcasts/#engineering_culture
- Generally AI: infoq.com/generally-ai-podcast

Follow InfoQ:
- Mastodon: https://techhub.social/@infoq
- X: https://x.com/InfoQ?from=@
- LinkedIn: linkedin.com/company/infoq
- Facebook: facebook.com/InfoQdotcom#
- Instagram: instagram.com/infoqdotcom/?hl=en
- Youtube: youtube.com/infoq
- Bluesky: https://bsky.app/profile/infoq.com


Write for InfoQ: Learn and share the changes and innovations in professional software development.
- Join a community of experts.
- Increase your visibility.
- Grow your career.
infoq.com/write-for-infoq