Uploaded March 2021 | Updated September 2026, 5 hours ago
As we enter the era of Big Data, a large number of data-driven systems and applications have become an integral part of our daily lives, and this trend is accelerating dramatically. It is estimated that 1.7MB of data is created every second for every person on Earth, for a total of over 2.5 quintillion bytes of new data every day, reaching 163 zettabytes by 2025. In addition to the growing volume, velocity, and variety of continuously generated data, novel technological trends such as edge processing, IoT, 5G, and federated AI bring new requirements for faster processing and deeper, more computationally heavy data analysis. Therefore, we can no longer rely on the “old school” data processing mechanisms, and a new paradigm is needed.
In this talk, we will explore this new paradigm, which we call Big Data Stream Processing (BDSP). BDSP comprises a variety of methods for scalable and efficient data processing that do not rely on traditional databases for storing and processing the data. We will cover examples of real-life applications that can greatly benefit from incorporating BDSP capabilities.
In particular, we will cover:
– On-the-fly detection of complex patterns in streaming data
– Stream-oriented machine learning and data mining
Speaker: Ilya Kolchinsky, Senior Software Engineer at Red Hat
Conversation Leader: Oren Oichman, Senior Cloud Consultant at Red Hat
►About the event
Research Days is a series of conversations about research & innovations in open source between researchers and Red Hat experts.
►Are you interested in open source projects we are incubating at Red Hat Research? Visit research.redhat.com to learn more and get involved!
As we enter the era of Big Data, a large number of data-driven systems and applications have become an integral part of our daily lives, and this trend is accelerating dramatically. It is estimated that 1.7MB of data is created every second for every person on Earth, for a total of over 2.5 quintillion bytes of new data every day, reaching 163 zettabytes by 2025. In addition to the growing volume, velocity, and variety of continuously generated data, novel technological trends such as edge processing, IoT, 5G, and federated AI bring new requirements for faster processing and deeper, more computationally heavy data analysis. Therefore, we can no longer rely on the “old school” data processing mechanisms, and a new paradigm is needed.
In this talk, we will explore this new paradigm, which we call Big Data Stream Processing (BDSP). BDSP comprises a variety of methods for scalable and efficient data processing that do not rely on traditional databases for storing and processing the data. We will cover examples of real-life applications that can greatly benefit from incorporating BDSP capabilities.
In particular, we will cover:
– On-the-fly detection of complex patterns in streaming data
– Stream-oriented machine learning and data mining
Speaker: Ilya Kolchinsky, Senior Software Engineer at Red Hat
Conversation Leader: Oren Oichman, Senior Cloud Consultant at Red Hat
►About the event
Research Days is a series of conversations about research & innovations in open source between researchers and Red Hat experts.
►Are you interested in open source projects we are incubating at Red Hat Research? Visit research.redhat.com to learn more and get involved!
