Gaurav SenGarbage collection is a way of automatic memory management provided by modern programming languages like Java, Go, Python and JavaScript.
We talk about how garbage is identified, the tricolor algorithm, the generational hypothesis and it's effect on latency and throughput.
Scenarios where the generational hypothesis doesn't work are also discussed. This includes caching and nepotism in queues (old generation objects promote young ones without due cause).
0:00 - Agenda 0:38 - Garbage Identification 0:50 - The Tricolor Algorithm 2:53 - Making the GC run faster 3:12 - Concurrent Collectors 3:49 - Generational Hypothesis 4:50 - Memory diagram 7:43 - Code injection JIT 9:09 - General hypothesis exceptions 9:43 - Nepotism
References: Tricolor Algorithm: Knuth, The art of computer programming Generational Hypothesis: https://people.cs.umass.edu/~emery/classes/cmpsci691s-fall2004/papers/p157-ungar.pdf
Garbage Collection Algorithms: Mark Sweep, Generation Hypothesis and JIT code injectionGaurav Sen2020-12-15 | Garbage collection is a way of automatic memory management provided by modern programming languages like Java, Go, Python and JavaScript.
We talk about how garbage is identified, the tricolor algorithm, the generational hypothesis and it's effect on latency and throughput.
Scenarios where the generational hypothesis doesn't work are also discussed. This includes caching and nepotism in queues (old generation objects promote young ones without due cause).
0:00 - Agenda 0:38 - Garbage Identification 0:50 - The Tricolor Algorithm 2:53 - Making the GC run faster 3:12 - Concurrent Collectors 3:49 - Generational Hypothesis 4:50 - Memory diagram 7:43 - Code injection JIT 9:09 - General hypothesis exceptions 9:43 - Nepotism
References: Tricolor Algorithm: Knuth, The art of computer programming Generational Hypothesis: https://people.cs.umass.edu/~emery/classes/cmpsci691s-fall2004/papers/p157-ungar.pdf
#GarbageCollection #JavaGC #JavaGarbageCollectionScaling Memcache at Facebook: Research Paper WalkthroughGaurav Sen2024-10-19 | In this video, we walk through the 2010 research paper "Scaling Memcache at Facebook." This paper has been very influential in companies building distributed caching systems.
For example, Uber and Twitter have caching systems inspired by Memcache.
The requirements of Memcache were: 1. Easy adoption, minimal APIs 2. Low Latency 3. Very High Availability (without sacrificing too much on consistency) 4. Large globally distributed system
The design choices of Memcache are influenced by the above, as we will see in this video.
#SystemDesign #InterviewReady #Coding5 questions in 4 minutes: Gaurav Sen interviews Hemant PandeyGaurav Sen2024-10-14 | I asked a series of (hopefully relevant) questions to Senior Software Engineer Hemant Pandey, who currently works at Meta. Watch this short video to understand what a day in their life is like.
00:00 1. What does a senior engineer do? 00:48 2. How do you motivate the team? 01:27 3. Individual Contributor or Engineering Manager? 02:06 4. How do you stand out in FAANG? 02:35 5. Is ChatGPT affecting the job market? 03:53 Thank you!
#RapidFireQuestions #SeniorEngineerAlgorithm Deep Dive: Realtime Audio Matching In ShazamGaurav Sen2024-10-11 | Have you ever been at a restaurant, and noticed a song playing in the background? You may want to know the original song to listen to later!
This is the use case of Shazam, an audio search app that takes recorded clips from users in their natural environment. It then finds the original song of the recorded clip.
I have personally used it often, and the algorithm beneath fascinates me :D
Algorithm: - Graph the frequency and amplitude of each point in time in the song. - Find the points with high amplitude and frequency variation in this graph. - Take all k consecutive points to get a set of chunks. - Take the combinatorial hash of each chunk. - Store these hashes in the DB.
#Algorithm #Audio #HashingSystem Design of a Ticket Booking System: BookMyShowGaurav Sen2024-09-24 | Let's try to design the ticket booking system of BookMyShow.
#systemdesign #interviews #examplesSoftware Systems: What is a load balancer?Gaurav Sen2024-07-15 | Load balancing is distributing work evenly onto a set of resources.
In this video, we explain the main reason for load balancing, along with some common algorithms: 1. Round robin 2. Geo-distributed 3. Least Connection
00:00 What is load balancing? 00:18 Example 02:14 Resource Optimization 03:40 Round robin 04:39 Geo-distributed load balancing 05:31 Least connections 06:23 Practical example 07:08 Conclusion
Looking to ace your next interview? Try this System Design video course! π₯ interviewready.io
#LoadBalancer #Algorithms #SystemDesignInterview Question: Tell me about yourselfGaurav Sen2024-07-14 | She is the CEO of Edelweiss, and she isn't alone.
Many people believe this is a good question to ask. Because it's open-ended and can be answered by anyone (no barrier to entry).
Here is how you can answer the question: Highlight a story from your recent past, that explains why you are good for the applied job.
If you talk about irrelevant things, the interviewer will have a great conversation for 45 minutes, but they will not move forward. Because they won't know if you are right for the role or not.
-- From the interviewer's perspective.
Unstructured questions have very biased evaluations. Someone who plays chess impresses me, someone who plays the guitar impresses you. We will not be able to come to a consensus on who is right for the role.
You have limited time. You have a small set of questions to ask. The direction you set in the interview will matter for the next 2 years.
Ask yourself: Will the answer to this question help me understand whether this candidate is to be accepted or rejected?
A heuristic is okay. A substitute is okay. But random, unstructured interview questions are like shooting darts blindfolded.
So if you are the interviewee, answer the question with a smile while continuing to look for other roles. If you are the interviewer, replace this question with a more specific job-relevant one.
00:00 The question 00:16 How to answer 01:14 Is this a good question?
#Interview #behavioralinterviewquestionsCaching in distributed systems: A friendly introductionGaurav Sen2024-07-13 | Caching is an amazingly effective technique to reduce latency. It helps build scalable, distributed systems.
We first discuss what is a cache and why we use it. We then talk about the key features of a cache in a distributed system.
Cache management is important because of its relation to cache hit ratios and performance. We talk about various scenarios in a distributed environment.
#Caching #DistributedSystems #SystemDesignSystem Design of Zamzar File Convertor #SystemDesign #OnlineJudgeGaurav Sen2024-07-11 | Me playing a system design game at InterviewReady for an online file convertor like Zamzar.
#SystemDesign #InterviewReady #CodingWhat is a CDN (Content Delivery Network)?Gaurav Sen2024-07-09 | Content Delivery Networks are a bunch of servers spread across the globe to serve information. These networks are available on rent to deliver static content quickly to nearby users.
Some examples of CDNs are Amazon CloudFront and the Akamai CDN. They are (relatively) cheap to rent and have high availability. They also provide pluggable algorithms to invalidate and fetch data.
We discuss why content delivery networks are useful, and how they serve data with examples.
00:00 What is covered here? 00:23 Use case 00:57 Basic Idea 01:20 Challenges with central caching 02:32 Localized Geodistributed cache 02:51 Benefits 04:01 Content Delivery Networks 04:56 CDN example - AWS CloudFront 05:28 TLDR
#SystemDesign #CDN #SoftwareEngineeringHow to get promoted to Senior Software Engineer rolesGaurav Sen2024-06-30 | What gets you promoted to senior engineer?
-- As a senior, find simple solutions to problems. Simple is fast. Simple is extendible. Simple is easy to explain.
-- As a senior, do things (including boring things) that have an impact. Business considerations come before our preferences, even before our learning opportunities.
-- Senior engineers have a reputation for reliability. Maybe not 100% tested code, but they check for basic edge cases and monitor prod for problems.
If you are trying to be promoted to senior, go for the most impactful projects, and do them without complaint. Yes, the code sucks. Yes, the vendor APIs suck. Damn, it's just a CRUD app with zero innovation.
That's where the money is though. And once you have your manager's trust, you will see the most interesting projects come to you. Even better, you will be promoted quicker, having a bigger and bigger impact on your organization.
Ironically, with the above attitude, you are likely to get the most exciting projects when they come up.
#SoftwareEngineering #PromotionsThis is why Senior Software Engineers arent clearing interviewsGaurav Sen2024-06-08 | Senior engineers are finding it harder and harder to clear interviews.
It's my dream to build an online system design judge for this community. A place to practice, learn and prepare for system design interviews. This judge has big aims:
1. To democratize system design preparation -- provide an alternative to expensive mock interviews -- go beyond languages and time zones, with an automated judge -- reduce bias by bringing objectivity into system design tests 2. Cater to all engineers, especially those who are new to system design
#SystemDesign #InterviewReady #OnlineJudgeFind the distance between friends/connections - LinkedIn System Design with @KeertiPurswaniGaurav Sen2024-04-08 | Have you noticed a small icon on the right of LinkedIn profiles?
It tells you how closely you are related to a user (1st, 2nd or 3rd-degree connection).
These distances are calculated using graph algorithms. Specifically, LinkedIn uses Bi-directional BFS.
This is a two-way algorithm where a search is started from both source and destination, and terminated midway. The total distance traveled from source+destination is the degree of the connection.
But at scale, this solution brings challenges. Let's limit ourselves to finding 3rd-degree connections:
1. Every query needs a graph traversal on our social network. 2. A naive BFS would take O(n^3) for search. A bi-directional BFS needs O(n^2) for searching users + O(n^2) for merging results. 3. To avoid performing these queries repeatedly, we cache the second-degree connections of users. This saves up on the first O(n^2) factor for subsequent queries.
Great, but how do we store all this data in a single cache?
Well, we can't. LinkedIn is forced to scale out, with multiple cache nodes storing second-degree connections. User-Connection data is sharded into caches according to user_id.
But what if a machine crashes? All queries to the shards hosted by that machine will fail.
To avoid this, we replicate shards into different machines. Even if one of the machines fails, the shard can be found on another machine.
And here we meet our main villain: High Latency.
We don't want to hit all the machines with our shards. We want to hit the SMALLEST possible number of machines, that contain all our shards.
For the mathematically inclined, this is a set-cover problem. LinkedIn uses a modified greedy version of the problem to reduce its compute costs in half!
The basic idea is to find machines that host most of the users who are your 2nd-degree connections, and avoid those that have few 2nd-degree connections.
#SystemDesign #InterviewReady #SocialNetworksSystem Design of a Delivery System like Zomato with @KeertiPurswaniGaurav Sen2024-03-30 | In this video, we look at what powers the delivery and tracking systems of e-commerce apps like Zomato, Swiggy, and Amazon.
Along the way, we go through relevant spatial algorithms, network protocols, and design components used in location-based systems.
#SystemDesign #InterviewReady #CodingSystem Design of GitHub Code Search - SDC Episode 1 with @KeertiPurswaniGaurav Sen2024-03-16 | Github code search allows developers to view and edit code online. This is particularly useful when debugging code in remote locations (like on vacation)!
GitHub manages permissions, storage, and retrieval through a set of services. In this video, we look at systems that power its search APIs.
If you have any doubts or suggestions, please share them in the comments below.
This is the first episode of the System Design Charcha series. Subscribe for notifications and updates!
00:00 Problem Statement 01:55 Capacity Estimations 02:52 Brute Force Approach 04:00 High Level Architecture 06:30 API calls 11:04 Form of Response Object? 16:10 API flow 17:40 Search Engine 31:10 Summary 33:10 Peek under the hood 36:14 Final thoughts 37:00 Thank you!
References: Numbers every programmer should know: gist.github.com/jboner/2841832 Github statistics: https://github.blog/2023-01-25-100-million-developers-and-counting/
We are going to solve this new question at InterviewReady related to Valentine's Day!
#SystemDesign #InterviewReady #valentinesdaySoftware Engineering Whitepapers that are worth reading. Live Readings at InterviewReady.Gaurav Sen2024-01-18 | ...Guy just discovered NoSQLGaurav Sen2024-01-11 | ...System Design Recommendation #ytshorts @sudocodeGaurav Sen2023-12-19 | A recommendation for system design study.Latency Numbers Every Programmer Should Know - 1000x slow-downGaurav Sen2023-11-29 | These are some latency numbers every programmer should know.
00:00 Memory and SSD 00:13 Sequential Disk Read 00:33 Roundtrip CA to Netherlands 02:48 Roundtrip CA to India 06:08 Ends
#SystemDesign #Software #CodingResource Lists for Senior Engineers: WhitePapers, Blogs and VideosGaurav Sen2023-11-27 | We share some fantastic technical papers to level up your skills!
From must-read whitepapers to game-changing blogs, we've got it all covered. π
Grab your favorite drink, kick back, and explore the latest tech trends together!
00:00 WhitePapers 01:16 Company cultures reflected in papers 04:28 Example - Scalability at what COST 06:06 Sample Walkthrough 08:26 Engineering Blogs 10:00 Difference between papers and blogs 10:56 Example - Uber Engineering Blog 11:30 Resource links!
#SystemDesign #InterviewReady #CodingThe Most Advanced Multiplication Algorithms: Why Karatsuba and FFT beat high-school mathematicsGaurav Sen2023-11-08 | For folks interested in multiplication algorithms (turns out this is a big deal in computer science and math).
1. High school method - O(n^2) 2. Karatsuba - O(n^1.58) 3. Toom-Cook - O(n^1.46) 4. FFT - O(n*log(n)*log(log(n)))
The last algorithm is pretty complex. We explain the intuition behind it in this video.
00:00 Who should watch this? 00:18 Example Problem 01:23 Karatsuba Algorithm 02:38 Time complexity analysis 03:57 Toom-Cook Algorithm 04:48 Problem - Polynomial Multiplication 05:27 Plotting points on a graph 05:57 Plotting nth degree curve 07:00 Multiplication Strategy 07:28 Optimisation Strategy 08:50 Choosing the right points 09:25 Complex Numbers! 10:26 The nth roots of unity 12:56 The impact of complex numbers 14:57 FFT Conclusion 15:52 Thank you!
#SystemDesign #InterviewReady #CodingThe painfully outdated practice of software interviewsGaurav Sen2023-11-01 | Software Interviews need a revamp. Coding rounds are IRRELEVANT to our jobs.
On average, software developers spend three months preparing for an interview.
A good engineer has to choose between:
1. Focusing on the job. 2. Practice useless DSA questions.
It's time to ask job-relevant questions and test candidates' WORKPLACE capabilities.
At InterviewReady, we are building tools to test engineering capabilities like:
1. Fault Tolerance 2. Async Programming 3. API Design
These are fundamental concepts that all engineers should know.
00:00 Agenda 00:16 Coding Interviews are broken 01:00 Company Perspective 01:19 Are Coding Questions Relevant? 02:24 Hiring in the 2000s 02:56 Hiring in the 2010s 03:44 Hiring in 2020s 04:10 Proposed Solution Features 05:00 Idea - Software Judge 05:31 Idea - System Design Judge 06:03 Launch Date 06:16 Why are we doing this? 06:51 Where we are now 07:17 Thank you!
We want to enable senior engineers to focus on their jobs rather than spending three months revising binary trees.
We want to help young engineers improve through self-practice using our automated tools.
We want to make hiring tests relevant.
Save the date: November 21st, 2023. Register at InterviewReady for updates.
Send your suggestions to: contact[at]interviewready[dot]io
#softwareengineering #softwareinterviews #codinginterviewHow ANOMALY DETECTION works in time series using the Holt-Winters AlgorithmGaurav Sen2023-10-26 | This video explains how anomalies are detected in a time-series graph.
The algorithm's name is Holt-Winters. The idea is simple, and the results are often useful.
Cheers!
00:00 What is a time series? 00:30 How can we find anomalies? 01:00 Anomaly Detection Algorithm 02:25 Repeated Differentiation 03:36 Example 04:14 Thank you!
#SystemDesign #InterviewReady #Coding20 AWS services you should know [as a Software Engineer]Gaurav Sen2023-10-18 | Here are some AWS services you should know as a software developer or freelancer. These services can help you quickly build and manage the technical muscle of your organization!
#SystemDesign #InterviewReady #CodingSystem Design of a Startup: How to host a website with AWSGaurav Sen2023-10-11 | Here is practical advice for software engineers working in small to medium organizations. We take a deep dive into the architecture of InterviewReady to understand how tech works in startups.
#Startups #architecture #InterviewReady20 Whitepapers that changed the world [For Senior Software Engineers]Gaurav Sen2023-10-04 | Here are some whitepapers that you must read as a software engineer. These technologies have changed the world and inspired hundreds of systems through their ideas to tackle scale, reliability, and performance.
Whitepapers have the benefit of discussing practical implementation details and the related tradeoffs to engineering products. The best engineers worldwide have written them after years of work, so don't miss any of them!
00:00 Who should watch this? 00:29 Memcached Design Tradeoff 01:40 20. TikTok Monolith 02:55 19. Meta FlexiRaft 03:54 18. Google Spanner 04:30 17. Meta Minesweeper 05:33 16. Apache Cassandra 06:07 15. Apple FoundationDB 06:36 14. Amazon AuroraDB 07:11 13. Google Pregel 07:45 12. Google Dapper 08:37 11. Google Chubby 09:12 10. Google Megastore 10:04 9. Google Bigtable 10:44 8. Google Map-Reduce 11:38 7. Google File System 12:25 6. Meta TAO 13:05 5. Meta Memcached 13:40 4. Google Monarch 14:23 3. Meta GorillaDB 15:22 2. Amazon DynamoDB 15:52 1. Google Zanzibar 17:00 Links in Description
#SystemDesign #WhitePapers #SeniorEngineerDesign Patterns for High Availability: What gets you 99.999% uptime?Gaurav Sen2023-09-27 | In this video, we discuss the topic of availability in distributed systems.
We categorize organizations based on their acceptable levels of availability, ranging from startups to mature companies aiming for five to six nines of availability.
We share a real-world example of a startup facing availability challenges with its database hosted in the wrong region. The solution involves migrating the database to a more suitable location and implementing a step-by-step process to minimize downtime.
Here are five principles for building highly available systems:
1. Simplicity over Perfection 2. Downtime Over Loss 3. Lesser Moving Parts 4. Chaos Engineering 5. Incident Reports and Root Cause Analysis
We also touch upon fault tolerance strategies such as redundancy, load balancing, and database replication to ensure high availability in distributed system components.
Engineers should either leverage existing highly available systems or adopt a principled approach to building and maintaining availability in their systems.
00:00 Who is this video for? 00:20 The 9s of availability 02:43 War Story at InterviewReady 06:14 Principles for High Availability 09:05 Design Patterns for Availability 12:31 Conclusion 12:49 Thank you!
#HighAvailability #SystemDesign #SoftwareEngineeringSystem Design PRACTICE Platform - Online Judge for High Level DesignGaurav Sen2023-09-23 | Learn system design by practicing it.
00:00 Ignore - waiting 05:57 Announcement - Online Design Judge 07:46 GAME - Live Streaming 16:08 Feedback 18:07 GAME - Recommendation Engine 26:30 Who should play this game? 27:35 Break 28:20 GAME - Chess Website 38:00 Ignore - Screen Share Fail 41:20 Anomaly Detection Algorithm 43:50 How to play - Instructions 47:00 GAME - Email Design 55:06 Live Feedback
#SystemDesign #OnlineJudge #PracticeTech that DOESNT WORK for Start-ups and Mid-sized companiesGaurav Sen2023-09-13 | Tech advice is like refactoring. It's easy to talk about, but hard to implement.
In small companies, picking up low-impact projects can derail the entire organization. In this video, we look at what are the worst pieces of tech advice for startups, and how to avoid them.
At the end of this video, you should be able to detect superficial hype-fueled advice from a distance. The overall idea of the video can be summarised in two acronyms: KISS and YAGNI.
The video explains the above with examples :D
00:00 Who is this for? 00:51 5. Containers 02:20 4. NoSQL 04:35 3. Microservices 05:59 2. Quality Obsession 08:00 1. Automation 11:13 TLDR 11:38 Thank you!
#Startups #NoSQL #ContainersAuthorization across Distributed Systems: The OAuth ProtocolGaurav Sen2023-09-06 | Websites are like castles, with large moats around them. You need a password at the gate to get in.
And the average person has to remember 27 passwords!
Remembering passwords is a pain. About 30% of all customer queries are "How do I reset my password?"
So websites found a unique solution: outsource the authentication problem to the πππ castles.
---------------------------------------------
Instead of asking users to enter an email and password, websites now ask users to "connect" them with Google for registration.
The user is now authenticated, and a session token can be sent for further auth requests. This process of outsourcing user authentication (technically authorization, since the user authorized you to view their name) is called OAuth.
---------------------------------------------
Third-party sign-in reduces login hesitance, ease of mobile registration, and password reset issues.
It also consolidates data power into a few companies, which know exactly which websites you visited to tailor your ads (Did you register on FirstCry? Let me show you a diaper ad).
You can learn more about OAuth, SSO, and Access Control Lists at InterviewReady.
Cheers!
00:00 What will we learn? 00:20 The Problem with Passwords 01:25 OAuth Flow 04:22 War story: OAuth Doubles Signups 06:43 Advantages of OAuth 08:55 Drawbacks of OAuth 11:31 Conclusion 12:13 Distributed Security Terms 15:30 Thank you!
Use the special DISCOUNT coupon of "HELLOWORLD" to avail an exclusive YouTuber channel offer!
#OAuth #Security #DistributedSystemsHow Stored Procedures make databases FASTGaurav Sen2023-08-31 | Stored Procedures are functions which can run on a database. Here, we discuss how stored procedures work, when and why they should be used, and the potential drawbacks of this approach.
We illustrate these points with a real-world example: a war story!
00:00 What are stored procedures? 00:40 Where and how do they run? 01:25 Real-world example 02:30 Request-Response Flow 03:20 First Suggestion - BiDirectional BFS 04:14 Cache results 05:30 Debugging 06:02 The Problem 07:08 Solution - Stored Procedures 08:03 Benefits of Stored Procedures 09:59 Drawbacks of Stored Procedures 12:00 Thank you!
#Databases #StoredProcedures #QueryOptimizationGarbage Collection Algorithms in Java: Concurrent Updates with Optimistic Locking - Part IIIGaurav Sen2023-08-24 | We discuss how concurrent compaction using Compare and Swap (CAS) operations helps speed up garbage collection algorithms and reduce response latencies.
This is the last piece of the puzzle in the garbage collection series. Cheers!
Garbage collection is a way of automatic memory management provided by modern programming languages like Java, Go, Python and JavaScript.
00:00 Who should watch this 00:43 Recap - Garbage Identification 02:00 The hunt for low latency 02:33 Generational Hypothesis 03:27 Concurrent Mark and Sweep 04:05 Partitioning into regions 04:50 Part III - Concurrent Compaction 05:17 Problem: Lost Writes 06:28 Solution: Forwarding Pointers 08:21 Garbage Collectors in Production 08:50 Conclusion 09:50 Thank you!
#GarbageCollection #Java #ConcurrentCompactionHow 100 milliseconds cost Amazon 3 BILLION DOLLARS: Latency, Concurrency and ParallelismGaurav Sen2023-08-19 | Can a 500-millisecond increase in page load times cost a website 20% of its incoming search traffic?
Yes. The impact of latency on user experience is disproportionately large, and both front-end and back-end software engineers are expected to know how to write async programs.
This helps businesses respond to user demands as soon as possible. There are two types of async programming techniques: 1. Concurrency 2. Parallelism
In this video, we explain the benefits and drawbacks of each technique using manufactured and real-world examples.
00:00 Who should watch this 00:20 What are Async Processes? 00:50 The Impact of Reduced Latency 01:54 Concurrency Example 02:28 Parallelism Example 03:02 Putting them together 03:40 Drawbacks of Async Processes 04:26 The Impact of Increased Complexity 05:36 Real-world examples
#SystemDesign #InterviewReady #CodingImpeccable API Design: What you MUST CONSIDER before deploying APIs to productionGaurav Sen2023-08-16 | In this video, we talk about what APIs are, how they help software engineers expose data for query and updates, and what are some best practices when exposing APIs on the server side.
We touch upon various topics including: 00:00 What is an API? 00:27 Indian Government Aadhaar APIs 00:40 How do APIs work? 01:16 REST vs. GraphQL 01:31 Webhooks 02:04 Writing Good APIs 02:10 What is Atomicity? 02:53 What is Idempotency? 03:35 Error Codes for HTTP APIs 05:28 OpenAPI Specs with Swagger 05:54 War Story - Aadhaar API Integration
We end with a war story of API design gone wrong. Let's make our software clients live in a better place :p
Are you looking to master system design? InterviewReady: interviewready.io
#SystemDesign #InterviewReady #CodingWhat is an API Gateway?Gaurav Sen2023-08-10 | What's an API gateway? It is a server that exposes internal service APIs to external client devices, using a protocol like HTTP.
The gateway also uses design patterns like REST and GraphQL to expose APIs, which allow clients to get and update data through servers with databases.
In this video, we will talk about API gateways, their use cases, and how they work with CDNs and DNSes to ensure fast and reliable user request handling.
00:00 What is an API Gateway? 01:02 Authentication on the Gateway 01:30 Request Transformation 02:07 Request Validation 02:43 Rate Limiting 03:09 Request Routing 04:20 Load Balancing 05:32 Architecture Overview 06:00 DNS 07:50 CDNs 09:30 Architecture Overview with AWS 10:31 War Story - Restart EVERY Week!
#SystemDesign #InterviewReady #CodingCareer Choices: Four differences between Managers and ICsGaurav Sen2023-06-11 | This video compares individual contributors and managerial roles. The critical points covered include the following:
Evaluating individual contributors and managers based on specific metrics and performance. Individual contributors' roles include technical knowledge, implementing solutions, and gathering product requirements.
The role of managers focuses on influence, integrity, transparency, fairness, and helping teammates succeed.
Handling uncertainty as a manager, dealing with volatility, uncertainty, complexity, and ambiguity (VUCA).
People management skills for managers, including hiring, onboarding, and creating a conducive work environment.
The impact of individual contributors across teams (horizontal impact) and managers within their vertical impact.
Managers' Competencies include task management, presence, ownership, and monitoring team health.
Mitigating risks as a manager, including avoiding single points of failure and minimizing risks to team success.
00:00 Introduction 00:38 Evaluation Metrics 02:05 Day in the life - IC 02:27 Influence as an Engineering Manager 03:38 What is Transparency? 04:13 What is Fairness? 04:49 Willingness to help 05:45 Managing Uncertainty - VUCA 07:18 People - Hiring, Appraisals and leaves 07:50 Impact on teams 08:27 Competencies 08:40 Task Management 09:09 Presence 09:35 What is Ownership? 10:08 Team Health - Motivation 10:32 Team Strength 11:05 Managing Risks 12:00 Share your feedback!
#IndividualContributor #EngineeringManagement #EMHow custom banners are shown INSTANTANEOUSLY: Contextual Advertising at ScaleGaurav Sen2023-06-05 | This is a lesson from the free chapter of "live-streaming app system design" at InterviewReady.
Context-based advertising during sports events is a big challenge, especially when serving millions of users with diverse interests.
In this video, we dive into what works behind the scenes, like recording calibrations, object recognition, ad ranking, user segmentation and video rendering.
The question answered here is: How do you serve dynamic ads to different user cohorts in real-time?
00:00 Intro 00:07 What are contextual ads? 00:44 Recommendation System 01:44 Rendering Options 02:22 Keying out with green screens 03:17 Fixed Camera Stands 04:29 Keeping the players in video 06:00 Object Recognition using AI 07:08 Thank you!
#SystemDesign #Contextual #AdvertisementsRunning a bootstrapped startup for 3 Years: The story of InterviewReady!Gaurav Sen2023-06-02 | It's been three years of running the awesome bootstrapped Edtech startup that we all know as InterviewReady!
Thank you for your support and love! We are here for the long run :D
00:00 Introduction 00:35 What is this video about? 01:18 What does bootstrapping mean? 04:05 Were there any tough times? 07:15 What makes you different from other EdTechs? 10:13 What was the vision when you started? 13:11 What's your daily routine? 16:11 Advice for young entrepreneurs 18:02 Advice for successful entrepreneurs 18:55 How did you inform your family? 21:10 How did you get such a high product rating? 24:50 Any setbacks? 26:48 Rapid Fire! 27:00 Texting vs. Talking 27:07 Favourite day of the week 27:20 Recent whitepaper you read 27:35 Recent song you are listening to 28:04 Who inspires you? 28:27 What's for dinner tonight?!
#SystemDesign #InterviewReady #CodingDark Mode: Pilot Launch at InterviewReadyGaurav Sen2023-05-25 | Hey guys, I am proud that we have this exclusive pilot feature to share with you:
Dark mode for videos.
At InterviewReady, we care about your learning and comprehension. Some of us study at night, and bright contrasting lights can be challenging to adjust to.
So, this is what we have (please play the attached video).
Cheers!
#learning #experience #darkmode
Link: interviewready.io/learn/system-design-course/consistency-in-distributed-systems/what-is-data-consistencyWhy you should do System Design.Gaurav Sen2023-05-18 | ...Zerodha Stock Broker System Design with @KeertiPurswaniGaurav Sen2023-05-17 | In this video, we discuss the high-level design of Zeroda, a stock broker app. We start by talking about the app's capabilities, including features like watchlists, market depth, stock performance analysis, buying and selling stocks, and portfolio management. We also mention the order book and trade book, which track orders and transactions.
We explain that Zeroda acts as a broker between clients and stock exchanges like the National Stock Exchange (NSE) and Bombay Stock Exchange (BSE). Zeroda takes orders from clients and places them with the exchanges. The market depth feature allows users to see the number of orders and their prices in the market. This information helps users make informed decisions about buying or selling stocks.
We discuss different types of orders, such as market orders and limit orders. Market orders involve buying or selling at the current market price, while limit orders allow users to set a specific price at which we want to buy or sell stocks. We also mention the importance of real-time stock price updates and the need for a fast and reliable system architecture.
Towards the end of the video, we briefly touch on the challenges of connecting to stock exchanges, including the cost of acquiring permissions and the need for reliable communication lines. We mention that exchanges strive for fairness by ensuring equal wire lengths for all brokers to minimize the latency of updates.
The video sets the stage for a discussion on designing the high-level system architecture of Zeroda, focusing on real-time data updates and communication with stock exchanges.
00:00 Intro 00:48 Stockbroker Requirements 12:11 Joke - Message queues aren't free? 12:44 Some trivia about stock machines 15:00 WebSockets for stock updates? 20:00 Capacity estimation of updates 23:30 Order Management 26:09 Some more trivia - War Story 28:30 Deciding on service boundaries 36:36 Message Queue - needed? 40:55 Cache - Global caching? 44:39 Executing orders 50:53 Real-life connection purchasing 52:52 Deep dive into watch lists 59:14 Eventual consistency for updates? 1:02:40 TCP vs. UDP for watch lists? 1:03:48 Validating the architecture 1:05:44 InterviewReady Discount!
System Design Course at InterviewReady: https://interviewready.io?_aff=KEERTI Designing Data-Intensive Applications Book: amzn.to/3SyNAOy
#SystemDesign #InterviewReady #CodingThe right attitude to interview rejectionGaurav Sen2023-04-11 | ...Will Developers be replaced by ChatGPT and other AI tools (like Github Copilot)?Gaurav Sen2023-04-05 | Many are wondering if their jobs will be taken over by machines. In the software development industry, this concern with AI is growing as ChatGPT gets popular.
ChatGPT is capable of generating code to a variety of input problems (it can even solve leetcode). Github Copilot is an AI-powered coding assistant that suggests code based on comments!
While this is impressive, the question remains: will they replace human developers? The answer is not straightforward.
#SystemDesign #InterviewReady #CodingTask Management - The one thing you must know as a software engineerGaurav Sen2023-03-29 | In this video, Gaurav Sen explains that software engineers are usually assigned to a project or task from a business, whether in a large company or a startup.
He emphasizes the importance of documenting and tracking tasks, even in large organizations, and introduces the use of task management systems like JIRA to do so.
These tools allow teams to discuss and track tasks, and provide a stage management lifecycle system for tasks to go through different stages, such as testing and code review, before being marked as complete.
00:00 Software Engineering Series 00:19 First task scenario 01:30 Tracking Tasks 02:00 Task Management Systems 02:21 Use cases 03:06 Ticketing management