Strange Loop Conference | "In the Land of the Sizing, the One-Partition Kafka Topic is King" by Ricardo Ferreira @StrangeLoopConf | Uploaded 1 year ago | Updated 1 day ago
Every technology has that key concept that people struggle to understand. With databases, is which join to use for fetching data from multiple tables. With containers, is which storage type to use depending on the workload. With Apache Kafka, the absolute winner is how many partitions to set for a topic. Sizing the number of partitions correctly is a hot topic for Kafka practitioners since doing it wrong affects other aspects of the system, such as consistency, concurrency, and durability. Worse, it also affects how much load Kafka can handle. This talk will peel off the concept of partitions and, using a what-if presentation style, explain what happens with the other aspects of the system given a number. It will highlight the consequences of poor decisions and whether you will be able to recover from them.
Ricardo Ferreira
Senior Developer Advocate
@riferrei
Ricardo is Senior Developer Advocate at AWS, working in the developer relations team for North America. With +20 years of experience, he may have learned a thing or two about distributed systems, messaging, fast data analytics, databases, and observability. Before AWS, he worked for software vendors like Elastic, Confluent, and Oracle. Ricardo is well known for his remarkable ability to explain complex topics. He cunningly breaks them down into bite-sized pieces until anyone can understand. While not working, he loves barbecuing in his backyard with his family and friends, where he finally gets the chance to talk about anything unrelated to computers. He currently lives in North Carolina, USA, with his wife and son. Follow Ricardo on Twitter: @riferrei
------ Sponsored by: ------
Stream is the # 1 Chat API for custom messaging apps. Activate your free 30-day trial to explore Stream Chat. https://gstrm.io/tsl
Every technology has that key concept that people struggle to understand. With databases, is which join to use for fetching data from multiple tables. With containers, is which storage type to use depending on the workload. With Apache Kafka, the absolute winner is how many partitions to set for a topic. Sizing the number of partitions correctly is a hot topic for Kafka practitioners since doing it wrong affects other aspects of the system, such as consistency, concurrency, and durability. Worse, it also affects how much load Kafka can handle. This talk will peel off the concept of partitions and, using a what-if presentation style, explain what happens with the other aspects of the system given a number. It will highlight the consequences of poor decisions and whether you will be able to recover from them.
Ricardo Ferreira
Senior Developer Advocate
@riferrei
Ricardo is Senior Developer Advocate at AWS, working in the developer relations team for North America. With +20 years of experience, he may have learned a thing or two about distributed systems, messaging, fast data analytics, databases, and observability. Before AWS, he worked for software vendors like Elastic, Confluent, and Oracle. Ricardo is well known for his remarkable ability to explain complex topics. He cunningly breaks them down into bite-sized pieces until anyone can understand. While not working, he loves barbecuing in his backyard with his family and friends, where he finally gets the chance to talk about anything unrelated to computers. He currently lives in North Carolina, USA, with his wife and son. Follow Ricardo on Twitter: @riferrei
------ Sponsored by: ------
Stream is the # 1 Chat API for custom messaging apps. Activate your free 30-day trial to explore Stream Chat. https://gstrm.io/tsl
