AWS Developers | Building a Leaderboard with Amazon Managed Service for Apache Flink | 1/5 @awsdevelopers | Uploaded 6 months ago | Updated 3 hours ago
Looking to level up your analytics? In this video, we’re diving into the fundamentals of streaming architecture, and how you can use Amazon Kinesis Data Streams for data ingestion. We’ll also cover gaming leaderboards as a use case, before walking you through how to set up the architecture and complete the data ingestion challenge in your own AWS account.
Please note deploying the infrastructure shown in this series will incur costs in your AWS account, so remember to stop all resources like Managed Flink Studio Notebooks when you’re not using them.
In this series, Anand Shah (Data Analytics and Streaming Specialist at AWS) will help you build a modern data streaming architecture for a real-time gaming leaderboard. This architecture includes data ingestion, real-time enrichment with database change data capture (CDC), data processing, as well as computing, storing and visualizing the results. You will also learn advanced streaming analytics techniques, such as the control channel
method for A/B testing, updating features and parameters with zero downtime, and how to handle late arrival of data. Anand will also talk you through the process of data de-duplication, as well as how you can store historical data for replay on-demand. 🎉
🌟 Get started with Amazon Managed Service for Apache Flink: aws.amazon.com/managed-service-apache-flink today to build and run your fully managed Apache Flink applications on AWS!
🔗 Github repository: github.com/build-on-aws/real-time-gaming-leaderboard-apache-flink
Resources used in this video:
🔗 Overview of AWS CDK: docs.aws.amazon.com/cdk/v2/guide/home.html
🔗 Using a Studio notebook with Managed Service for Apache Flink: docs.aws.amazon.com/managed-flink/latest/java/how-notebook.html
🔗 Apache Flink SQL API: nightlies.apache.org/flink/flink-docs-release-1.18/docs/dev/table/sql/gettingstarted
🔗 Intro to Amazon Managed Service for Apache Flink:
docs.aws.amazon.com/managed-flink/latest/java/what-is.html
🔗 Intro to Amazon Kinesis Data Streams:
docs.aws.amazon.com/streams/latest/dev/introduction.html
Continue your learning with the references below!
🔗 Amazon Kinesis Data Streams: aws.amazon.com/kinesis/data-streams
🔗 Amazon Managed Flink Studio Notebooks: docs.aws.amazon.com/managed-flink/latest/java/how-notebook.html
🔗 Modern Streaming Data Architecture on AWS: docs.aws.amazon.com/whitepapers/latest/build-modern-data-streaming-analytics-architectures/what-is-a-modern-streaming-data-architecture.html
00:00 Intro
00:32 What will you learn?
01:26 Streaming analytics fundamentals
02:01 Most common streaming use cases
02:20 Streaming architecture overview
03:09 Gaming leaderboard use case
03:57 Gaming leaderboard streaming architecture
05:09 Streaming data ingestion
05:35 Demo: Github repository
07:43 Demo: CDK source code walkthrough and deploy
08:18 Demo: Infrastructure setup
10:30 Demo: Amazon Managed Apache Flink Studio Notebook setup
11:40 Demo: Challenge 1 - Select gaming event data
13:11 Conclusion
Follow AWS Developers:
👾 Twitch: twitch.tv/aws
🐦 Twitter: twitter.com/awsdevelopers
💻 LinkedIn: linkedin.com/showcase/aws
Follow Anand Shah:
🐦 Twitter: twitter.com/anandshah110
💻 LinkedIn: linkedin.com/in/anandshah110
#DataIngestion, #ApacheFlink, #ManagedServiceForApacheFlink
Looking to level up your analytics? In this video, we’re diving into the fundamentals of streaming architecture, and how you can use Amazon Kinesis Data Streams for data ingestion. We’ll also cover gaming leaderboards as a use case, before walking you through how to set up the architecture and complete the data ingestion challenge in your own AWS account.
Please note deploying the infrastructure shown in this series will incur costs in your AWS account, so remember to stop all resources like Managed Flink Studio Notebooks when you’re not using them.
In this series, Anand Shah (Data Analytics and Streaming Specialist at AWS) will help you build a modern data streaming architecture for a real-time gaming leaderboard. This architecture includes data ingestion, real-time enrichment with database change data capture (CDC), data processing, as well as computing, storing and visualizing the results. You will also learn advanced streaming analytics techniques, such as the control channel
method for A/B testing, updating features and parameters with zero downtime, and how to handle late arrival of data. Anand will also talk you through the process of data de-duplication, as well as how you can store historical data for replay on-demand. 🎉
🌟 Get started with Amazon Managed Service for Apache Flink: aws.amazon.com/managed-service-apache-flink today to build and run your fully managed Apache Flink applications on AWS!
🔗 Github repository: github.com/build-on-aws/real-time-gaming-leaderboard-apache-flink
Resources used in this video:
🔗 Overview of AWS CDK: docs.aws.amazon.com/cdk/v2/guide/home.html
🔗 Using a Studio notebook with Managed Service for Apache Flink: docs.aws.amazon.com/managed-flink/latest/java/how-notebook.html
🔗 Apache Flink SQL API: nightlies.apache.org/flink/flink-docs-release-1.18/docs/dev/table/sql/gettingstarted
🔗 Intro to Amazon Managed Service for Apache Flink:
docs.aws.amazon.com/managed-flink/latest/java/what-is.html
🔗 Intro to Amazon Kinesis Data Streams:
docs.aws.amazon.com/streams/latest/dev/introduction.html
Continue your learning with the references below!
🔗 Amazon Kinesis Data Streams: aws.amazon.com/kinesis/data-streams
🔗 Amazon Managed Flink Studio Notebooks: docs.aws.amazon.com/managed-flink/latest/java/how-notebook.html
🔗 Modern Streaming Data Architecture on AWS: docs.aws.amazon.com/whitepapers/latest/build-modern-data-streaming-analytics-architectures/what-is-a-modern-streaming-data-architecture.html
00:00 Intro
00:32 What will you learn?
01:26 Streaming analytics fundamentals
02:01 Most common streaming use cases
02:20 Streaming architecture overview
03:09 Gaming leaderboard use case
03:57 Gaming leaderboard streaming architecture
05:09 Streaming data ingestion
05:35 Demo: Github repository
07:43 Demo: CDK source code walkthrough and deploy
08:18 Demo: Infrastructure setup
10:30 Demo: Amazon Managed Apache Flink Studio Notebook setup
11:40 Demo: Challenge 1 - Select gaming event data
13:11 Conclusion
Follow AWS Developers:
👾 Twitch: twitch.tv/aws
🐦 Twitter: twitter.com/awsdevelopers
💻 LinkedIn: linkedin.com/showcase/aws
Follow Anand Shah:
🐦 Twitter: twitter.com/anandshah110
💻 LinkedIn: linkedin.com/in/anandshah110
#DataIngestion, #ApacheFlink, #ManagedServiceForApacheFlink
