AWS Developers | Compute, Store, and Visualize Results with Amazon MemoryDB for Redis | 3/5 @awsdevelopers | Uploaded 5 months ago | Updated 3 hours ago
In this video, we’ll show you how to process event data and compute your gaming leaderboard using Apache Flink, before storing the results to Amazon MemoryDB for Redis, and visualizing them with Grafana. You’ll see a deep dive into Redis data structures that combine with Apache Flink patterns, to power a gaming leaderboard. We’ll also discuss how to use a Lambda function that continuously sends queries from Apache Flink to Redis, and visualizes these results with Grafana dashboards.
In this series, Anand Shah (Data Analytics and Streaming Specialist at AWS) will help you build a modern data streaming architecture for a real-time gaming leaderboard. This architecture includes data ingestion, real-time enrichment with database change data capture (CDC), data processing, as well as computing, storing and visualizing the results. You will also learn advanced streaming analytics techniques, such as the control channel method for A/B testing, updating features and parameters with zero downtime, and how to handle late arrival of data. Anand will also talk you through the process of data de-duplication, as well as how you can store historical data for replay on-demand. 🎉
🌟 Get started with Amazon Managed Service for Apache Flink today, to build and run your fully managed Apache Flink applications on AWS! 👉 aws.amazon.com/managed-service-apache-flink
🔗 Github repository: github.com/build-on-aws/real-time-gaming-leaderboard-apache-flink
Resources used in this video:
🔗 MemoryDB for Redis: docs.aws.amazon.com/memorydb/latest/devguide/what-is-memorydb-for-redis.html
🔗 Guide to Amazon Managed Grafana: docs.aws.amazon.com/grafana/latest/userguide/what-is-Amazon-Managed-Service-Grafana.html
🔗 Redis data structure sorted set: redis.io/docs/data-types/sorted-sets
🔗 Redis ZINCRBY operation: redis.io/commands/zincrby
🔗 Redis ZADD operation: redis.io/commands/zadd
🔗 AWS CDK Overview: docs.aws.amazon.com/cdk/v2/guide/home.html
🔗 Modern Streaming Data Architecture on AWS: docs.aws.amazon.com/whitepapers/latest/build-modern-data-streaming-analytics-architectures/what-is-a-modern-streaming-data-architecture.html
Follow AWS Developers:
👾 Twitch: twitch.tv/aws
🐦 Twitter: twitter.com/awsdevelopers
💻 LinkedIn: linkedin.com/showcase/aws
Follow Anand Shah:
🐦 Twitter: twitter.com/anandshah110
💻 LinkedIn: linkedin.com/in/anandshah110
00:00 Intro
00:24 What will you learn?
01:11 What is Redis?
01:47 Redis operation ZINCRBY
02:31 Redis operation ZADD
03:33 Redis on AWS
04:00 What are we going to build?
04:22 Demo: CDK source code walkthrough and deployment
06:12 Demo: Importing notebook to Amazon Managed Flink Studio
07:21 Demo: Challenges 1 and 2 answers and explanations
09:59 Demo: Challenge 3 - Compute leaderboard and save to MemoryDB
11:18 Demo: Gaming leaderboard architecture evaluation
11:44 Demo: MemoryDB writer Lambda function walkthrough
12:31 Demo: Flink UI - Confirming data movement
12:48 Demo: Grafana - Connecting Redis data source
13:40 Demo: Grafana - Building dashboards
15:20 Conclusion
#MemoryDBforRedis, #ManagedServiceForApacheFlink, #ManagedGrafana
In this video, we’ll show you how to process event data and compute your gaming leaderboard using Apache Flink, before storing the results to Amazon MemoryDB for Redis, and visualizing them with Grafana. You’ll see a deep dive into Redis data structures that combine with Apache Flink patterns, to power a gaming leaderboard. We’ll also discuss how to use a Lambda function that continuously sends queries from Apache Flink to Redis, and visualizes these results with Grafana dashboards.
In this series, Anand Shah (Data Analytics and Streaming Specialist at AWS) will help you build a modern data streaming architecture for a real-time gaming leaderboard. This architecture includes data ingestion, real-time enrichment with database change data capture (CDC), data processing, as well as computing, storing and visualizing the results. You will also learn advanced streaming analytics techniques, such as the control channel method for A/B testing, updating features and parameters with zero downtime, and how to handle late arrival of data. Anand will also talk you through the process of data de-duplication, as well as how you can store historical data for replay on-demand. 🎉
🌟 Get started with Amazon Managed Service for Apache Flink today, to build and run your fully managed Apache Flink applications on AWS! 👉 aws.amazon.com/managed-service-apache-flink
🔗 Github repository: github.com/build-on-aws/real-time-gaming-leaderboard-apache-flink
Resources used in this video:
🔗 MemoryDB for Redis: docs.aws.amazon.com/memorydb/latest/devguide/what-is-memorydb-for-redis.html
🔗 Guide to Amazon Managed Grafana: docs.aws.amazon.com/grafana/latest/userguide/what-is-Amazon-Managed-Service-Grafana.html
🔗 Redis data structure sorted set: redis.io/docs/data-types/sorted-sets
🔗 Redis ZINCRBY operation: redis.io/commands/zincrby
🔗 Redis ZADD operation: redis.io/commands/zadd
🔗 AWS CDK Overview: docs.aws.amazon.com/cdk/v2/guide/home.html
🔗 Modern Streaming Data Architecture on AWS: docs.aws.amazon.com/whitepapers/latest/build-modern-data-streaming-analytics-architectures/what-is-a-modern-streaming-data-architecture.html
Follow AWS Developers:
👾 Twitch: twitch.tv/aws
🐦 Twitter: twitter.com/awsdevelopers
💻 LinkedIn: linkedin.com/showcase/aws
Follow Anand Shah:
🐦 Twitter: twitter.com/anandshah110
💻 LinkedIn: linkedin.com/in/anandshah110
00:00 Intro
00:24 What will you learn?
01:11 What is Redis?
01:47 Redis operation ZINCRBY
02:31 Redis operation ZADD
03:33 Redis on AWS
04:00 What are we going to build?
04:22 Demo: CDK source code walkthrough and deployment
06:12 Demo: Importing notebook to Amazon Managed Flink Studio
07:21 Demo: Challenges 1 and 2 answers and explanations
09:59 Demo: Challenge 3 - Compute leaderboard and save to MemoryDB
11:18 Demo: Gaming leaderboard architecture evaluation
11:44 Demo: MemoryDB writer Lambda function walkthrough
12:31 Demo: Flink UI - Confirming data movement
12:48 Demo: Grafana - Connecting Redis data source
13:40 Demo: Grafana - Building dashboards
15:20 Conclusion
#MemoryDBforRedis, #ManagedServiceForApacheFlink, #ManagedGrafana
