Data Engineering Road Map - How To Learn Data Engineering Quickly( By A FAANG Data Engineer) Seattle Data Guy 2026-09-19 | Data Engineering Road Map - How To Learn Data Engineering Quickly( By A FAANG Data Engineer)
Data Engineers Vs Data Analysts - How Data Engineers Write SQL Vs Analysts Seattle Data Guy 2025-06-12 | If you enjoyed this video, check out some of my other top videos.Top Courses To Become A Data Engineer In 2022youtube.com/watch?v=kW8_l57w74gWhat Is The Modern Data Stack - Intro To Data Infrastructure Part 1youtube.com/watch?v=-ClWgwC0SbwIf you would like to learn more about data engineering, then check out Googles GCP certificatebit.ly/3NQVn7VIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
What Is BigQuery - Breaking Down What BigQuery Is And Diving Into A Hands On Walkthrough Seattle Data Guy 2025-05-14 | Despite all the talk about Snowflake and Databricks.I still run into plenty of data teams using BigQuery. If you work in data, then you’ve likely used BigQuery and you’ve likely used it without really thinking about how it operates under the hood. On the surface BigQuery is Google Cloud’s fully-managed, serverless data warehouse.It’s the Redshift of GCP except we like it a little more.The question becomes, how does it work? There is a lot going on under the hood. In this article we’ll discuss how BigQuery works, how you can load data into it, and more.If you're looking for help loading data into BigQuery, you can try out Estuary - bit.ly/4eQC3oQSome Other Videos You Should Check OutWhat Is A Data Platform And Why You Should Build Oneyoutu.be/_BoM2ahSJV05 Key Predictions for the Data Industry in 2025youtu.be/n2eNrDi6wI4If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Leading Data Teams in The AI Era - With Lindsay Murphy Seattle Data Guy 2025-05-01 | The data world can feel like it's moving fast.It's even faster when you're leading data teams, trying to balance requests from the business and long-term projects.Especially when the drum beat of AI hums in the background.So, I'll be talking to Lindsay Murphy about her experiences leading data teams and trying to ensure she drives the correct impact.
Biggest Productivity Killers for Data Engineers - Battling Context Switching and Distraction Seattle Data Guy 2025-04-23 | Who hasn’t started a new job or project and told themselves, “This time will be different”?You’ve promised to stay organized, plan better, and avoid distractions.But like that famous quote from Jurassic Park, “Life finds a way.”In this case, life finds a way to pull us back into habits that prevent us from reaching our full potential. If you’re leading a data team, this often means juggling multiple projects, managing ad-hoc data requests, and attempting to keep everything in balance.Amid the chaos, it often feels like nothing is truly getting done. You might even feel like you’re paddling in a vast ocean but never really making progress.So, how do you actually get things done while leading a data team? Let's dive into some strategies that can help you overcome these challenges.If you enjoyed this video, check out some of my other top videos.Types Of Data Projects And Running Them Successfullyyoutu.be/ZNXuyWeFcUIIs It Time to Say Goodbye to Data Engineers? - The Data Engineering Dilemmayoutu.be/4FcG_ocJ41YIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
The Realities Of Working As A Data Engineer - Self-Service, Single Truths, And All The Hype Seattle Data Guy 2025-03-28 | For decades, data teams have been chasing multiple ever-elusive Holy Grails. These end goals get talked about heavily, but it feels as if very few data teams have indeed seen them.And if they have, it was always at some other job.The data world has many of them. Some are real, some aren't, and many are used by marketing and sales teams to sell the dream. In fact, when I first came into the data world, Tableau was selling self-service analytics hard. Of course, being new to the world, it makes sense; if end-users can access their data, they'll ask you fewer questions, right?Well…sort of.But let's not get too deep into the actual meat of this article before barely even starting. In this article, I outline some of the key holy grails we are constantly chasing in the data world, what's often held them back, how some companies have succeeded, and where AI will likely play a role in some of these as well.You can read the full article hereseattledataguy.substack.com/p/holy-grails-of-data-self-serviceIf you enjoyed this video, check out some of my other top videos.What Is Snowflake - Breaking Down What Snowflake Is, How Snowflake Credits Work And Moreyoutu.be/GuM6dQGRFyQBuilding A Real Life Data Engineering Project With Healthcare Data - The Million Dollar Data Productyoutu.be/J7USfXwjOU0If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Data Engineering Q&A - Seattle Data Guy Seattle Data Guy 2025-03-27 | It's been a long time since I've done a proper QA.Sure I've been doing back to back lives and webinars over the past few weeks but I really haven't done much in terms of giving space for questions.So if you want to ask some questions on data engineering, infrastructure, etc.Feel free to join in on this event!
Daily data engineering rituals #dataengineering Seattle Data Guy 2025-03-22 | Data engineering daily rituals
What Is Snowflake - Breaking Down What Snowflake Is, How Snowflake Credits Work And More Seattle Data Guy 2025-03-19 | Snowflake is a popular choice when it comes to building your data platform.I actually recall the first time I was introduced to Snowflake it was at a tech event that I assumed was a data meet-up and low and behold it was a sales pitch(essentially) but hey, there was free food.Jokes aside, in this video we'll talk about Snowflake, and answer questions like how does Snowflake credits work, different snowflake features and more!If you're looking for help loading data into Snowflake, you can try out Estuary - bit.ly/4eQC3oQSome Other Videos You Should Check OutWhat Is A Data Platform And Why You Should Build Oneyoutu.be/_BoM2ahSJV05 Key Predictions for the Data Industry in 2025youtu.be/n2eNrDi6wI4If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Data Engineering Q&A - Seattle Data Guy Seattle Data Guy 2025-03-14 | It's been a long time since I've done a proper QA.Sure I've been doing back to back lives and webinars over the past few weeks but I really haven't done much in terms of giving space for questions.So if you want to ask some questions on data engineering, infrastructure, etc.Feel free to join in on this event!Also, if you're looking to read more about data infra and strategy you can read my newsletter here - seattledataguy.substack.com
Back To The Basics Of Data Engineering - Why SFTP Remains Essential for Companies Seattle Data Guy 2025-03-05 | Every day, companies share data sets of users, patient claims, financial transactions, and more with each other.Most people might assume this would be via API. However, companies have been sharing data for decades using CSVs, TSVs, positional files, and other formats you might not be familiar with. Not via API, but SFTP.I know I wouldn’t have guessed that’s how companies send data back and forth when I was in school.If you’ve been in the industry for a while, you’ve probably come across automated SFTP jobs that do just that. You’ve also likely had to encrypt or decrypt a CSV and had to interpret a schema file with parsing instructions that—somehow—are always a bit off the first time.Now, sure, we all want to build real-time systems that use LLMs and other flashy new tools and solutions. Sometimes, that’s not what is called for.After all, companies of all sizes—even tech giants like Facebook and Airbnb—still use SFTP to share critical information for analytical purposes(as well as operational). So, let’s dig into what SFTP is and how you will likely work with it.If you want to read the full article, you can find it here-seattledataguy.substack.com/p/data-sharing-in-the-real-world-whyOr check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Is It Time to Say Goodbye to Data Engineers? - The Data Engineering Dilemma Seattle Data Guy 2025-02-19 | Ever since tools like SSIS came onto the scene, vendors and business leaders have been on a mission to remove what they see as the biggest roadblock to data-driven decision-making: data engineers.Or their counterparts—DBAs, ETL Developers, and Data Architects.Sure, not everyone says it so explicitly, but you can see it in vendor marketing and in the decisions made by the business.I remember talking to a veteran data expert who’s been in the field for three decades. They told me that when SSIS first launched, people were genuinely afraid for their jobs. The idea that you could just drag-and-drop tasks that once required code was nerve-racking. But if you’ve used SSIS, well, you know the truth.To some extent, I get why the idea is appealing. When a leader requests a report, a software engineer wants to modify an application table, or a data scientist wants to explore a new dataset, who’s the one slowing down the project?The data engineers.With their rules. Their governance. Their insistence on building robust, scalable data pipelines instead of quick fixes.They really slow down the workflow!If you enjoyed this video, check out some of my other top videos.Data Modeling - Walking Through How To Data Model As A Data Engineer - Dimensional Modeling 101youtu.be/gG7upg6QaBIDatabases Vs Data Warehouses Vs Data Lakes - What Is The Difference And Why Should You Care?youtu.be/FxpRL0m9BcAIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/If you're looking for a tool to make ingesting data into Snowflake, BigQuery, Iceberg or Databricks easier, then check out Estuary(Full disclosure I am an advisor for them)!bit.ly/4eQC3oQAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Why Your Data Infrastructure Migration Project Will Fail (And How to Succeed) Seattle Data Guy 2025-02-14 | If you work in tech long enough, you’ll inevitably find yourself in the middle of a migration project—probably more often than you’d like.“We should all use Teradata!” Migrate.“Actually, Snowflake is the future!” Migrate.“Wait, let’s go all-in on Iceberg and Databricks!” Migrate.It’s a never-ending cycle. In fact, migration is such a core part of the industry that Databricks recently acquired BladeBridge just to help them make migrations easier.The problem? Migrations often don’t go as planned. Some get delayed, others stall out halfway, and many become massive time and money sinks. There are countless ways for a migration to go sideways, but with the right approach, you can avoid the most common pitfalls.Let’s break down the biggest reasons migration projects fail—and how you can keep yours on track.If you want to read the article version, you can find it here - seattledataguy.substack.com/p/why-your-data-infrastructure-migrationIf your team is looking for help planning out and executing your migration project, then feel free to set up some time to chat here - calendly.com/ben-rogojan/consultation?month=2025-01Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Data Architecture And Leadership - Whats Changed In The Last Decade Seattle Data Guy 2025-02-07 | In 2015 we were in peak Hadoop times, at least according to Google trends. Now a decade later, the only reason you're still on Hadoop is because it's hard to migrate. That's all to say that in the last decade a lot has changed. But what does that mean for the way we approach data architecture and leadership. That's what I wanted to ask Dan D'Orazio about. Dan has been leading data teams and planning out data architecture for over a decade. What should I ask him?
The Past, Present And Future Of SQL Seattle Data Guy 2025-01-31 | Never bet against SQL. The number of "SQL is Dead" articles I read back around 2016 felt endless. Yet here we are, and it feels like SQL is more popular than ever. If you're an analyst, data scientist, data engineer and honestly plenty of other job titles, you'll likely have to know SQL. But where is it all going? That's the conversation I am going to have with Rui and Richard this coming week. Rui Machado is the author of Analytics Engineering with SQL and dbtlinkedin.com/in/rpmachado And Richard Meng is an ex-Snowflake Software engineer that led the Gen AI project for Snowflake's SQL Copilot and now is the co-founder or Roe AI a company focused on making unstructured data easier to access via SQL.linkedin.com/in/berkeleymenggetroe.aiWhat questions should I ask?
Growing To Senior Data Engineer - How To Get Unstuck In Your Career Seattle Data Guy 2025-01-27 | How do you grow from a mid-level data engineer to senior?There are multiple axes that companies might use, for example:- Engineering Excellence- Independence- Scope- Ability to handle Ambiguity- Influence- ImpactTo name a few. But let's dive into this video to talk about it.If you enjoyed this video and want to learn more about growing as an data engineer I did a longer 50 minute webinar on my newsletter for paid members. If you'd like to check it out you can find it. hereseattledataguy.substack.com/p/upcoming-data-eventsIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
5 Key Predictions for the Data Industry in 2025 Seattle Data Guy 2025-01-20 | It’s 2025, and the data industry is already off to a whirlwind start. At the end of last year, Databricks secured one of the largest funding rounds in history, Rivery was acquired by Boomi, and the massive buyout of Tabular (Edit: I originally wrote Iceberg was purchased by Databricks which was incorrect) has everyone talking.And that’s without even mentioning AI.With so much happening, it’s natural to wonder: where is the data world headed in 2025? Over the past few years, we’ve witnessed an explosion of technologies designed to help organizations collect, process, and analyze data. But while these tools are undeniably powerful, they often cater to the technologist more than the business leader.The real question for 2025 isn’t just which tools will dominate the market, but what outcomes they’ll deliver.As organizations face increasing pressure to maximize the return on their data investments, they’ll need to make strategic technology choices and embrace tailored solutions that align with their unique needs. For some, this will mean pushing the boundaries of innovation with cutting-edge technologies. For others, simplicity will take center stage.Here’s what you can expect to see shaping the data world this year.If you'd like to read the article version of this video, you can check it out here -seattledataguy.substack.com/p/5-key-predictions-for-the-data-industry?utm_source=activity_itemOr check out my blogtheseattledataguy.comI also referenced RoeAI, if you're looking to analyze unstructured data via SQL, check them out here - bit.ly/42mSJBoI don't have a Patreon, but if you'd like to support the channel and the newsletter you can become a paying member of the newsletter here - seattledataguy.substack.com/subscribeOr check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
What Is A Data Platform And Why You Should Build One Seattle Data Guy 2025-01-07 | Over the past near decade I have worked with dozens of companies and seen every which way you can set up your data platform. From the early days where someone had a SQL Server instance running underneath their desk to the modern cloud data platforms.But what is a data platform anyway?Also, special thanks to Vast Data for sponsoring this video, if you're looking for a data platform that unifies storage, database, and containerized compute into a single, scalable software platform you can check them out belowbit.ly/40ozVQOIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
How To Build a Career in Data Engineering and Analytics You Actually Enjoy In 2025 Seattle Data Guy 2025-01-02 | Have you ever wondered how people go from a steady 9-to-5 job to something bigger—or at least different? For me, it all started with a single consulting project. Funny enough, I almost turned it down. However, that one yes ended up reshaping my entire career.At 23, I was clocking 40 hours a week at my day job and suddenly adding another 15-20 hours on a side hustle. It wasn’t part of some big master plan. My friends just needed help with projects, and I thought, why not? Back then, I had zero intention of leaving my full-time job.Yet, over the next six years, that little side gig not only pushed me past six figures for the first time but also gave me the confidence to leave my job at Facebook.No one can script their career perfectly. Technology evolves fast, and life is often unpredictable. But even in the face of uncertainty, it’s possible to build a career that helps you grow, challenges you in meaningful ways, and sets you up for success.If you enjoyed this video, check out some of my other top videos.Don't Lead A Data Team Before Watching This - 5 Lessons You Need To Know As A Head of Datayoutu.be/n74ke0Wjmi4Building A Data Engineering Project - Starting A Data Analytics Project With Healthcare Datayoutu.be/Qp4BfEOq7zAIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
How To Start A Consulting Company In 2025 Seattle Data Guy 2025-01-02 | Three years ago I bet on myself at quit my job at Facebook to go all in on my own consulting company. If you're considering going down the same path, I know it can feel scary. Not to mention, once you're on it, it's more than a little lonely. That's why I wanted to kick off 2025 with first, a YT live that is open to everyone! Where will talk about starting a consulting company, the challenges you'll face and some of the trends I am seeing. From there, if you'd like you can join a free 3-day accelerator where we will dive into key subjects like marketing, pricing, and so on! The days are outlined below. You can sign up for it here - bit.ly/3V0Bgdl 3 - Day Consulting Accelerator Schedule Consulting The Good The Bad And The Ugly - Day 1 - The goal of this first day will be to outline not just what consultants do, but also challenge you to ask yourself whether consulting is for you. We’ll go over the realities and lessons I have learned along the way. We’ll also cover: - A High-Level Of What You Should Be Doing If You Just Started Consulting - Consulting Vs Contracting - Side-Hustling Vs Full Time - And More Attracting Clients - Day 2 - The goal of day two will be to talk through some key methods to landing clients. Now I already have posted several videos on this theme already, so we’ll do a deeper dive into it and really aim at providing examples so you can start taking action! How To Price - Day 3 - It goes without saying that pricing is likely one of the main sticking points for many consultants. So on this day we’ll go over different pricing approaches and some examples of past projects. Depending on interest, I might add a few more days to the free section. But let me know if you'd like to be part of this!
Building A Career In Data In 2025 - What To Expect And QA with @AlexTheAnalyst Seattle Data Guy 2024-12-18 | Fact! The year is almost over, and I am sure that means there are plenty of people out there who have questions what is going on in the data world. So let's talk about it! Alex Freberg will be jumping on a live tomorrow and we'll talk about data careers, trends he is seeing in tooling and more. If you have any questions you'd like to ask, please post them below!
Lessons From 5 Years Of Consulting 1 Year Later - Jeff Skoldberg Seattle Data Guy 2024-12-10 | Nearly exactly a year ago I talked with Jeff about his experience as a data consultant. Now that we are at the end of the year for 2024 I wanted to get a recap. - What has consulting in 2024 been like? - What lessons has he learned? - What themes has he seen? So don't miss out if you're interested in hearing from Jeff's now 6 years of experience!
Building A Real Life Data Engineering Project With Healthcare Data - The Million Dollar Data Product Seattle Data Guy 2024-12-06 | Are you looking for an example of how you can build a data engineering project with real life data?Data that represents actual projects you might deal with in the real world. Well in this video I'll be diving into just that.We will be using Snowflake, Apache Airflow and more as the project goes on.By the end of this my goal is to build several data products all focused around this one data set.If you're interested in helping or looking at this project you can find it here. I'll be working on it when I have time. So...github.com/sdg-1/healthcare-claims-analytics-project0:00 Intro2:51 Building A Healthcare Data Pipeline With Airflow5:58 High Level Design Overview12:40 Inserting Data Scripts24:00 Airflow UI Overview 27:00 Github project overviewIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
How Are Companies Using Generative AI - With John Hwang Ex-AWS Generative AI Architect Seattle Data Guy 2024-11-26 | How are companies actually implementing their generative AI architecture and for what use cases. I am sure many of you are wondering, as am I. That's why I’m excited to share that I’ll be interviewing John Hwang a leader whose career journey offers a rare blend of insights from multiple industries—ranging from financial trading to AI architecture and now entrepreneurship. What questions should ask John?
10 Years Of Building Data Pipelines - What Has Changed And What Has Stayed The Same Seattle Data Guy 2024-11-22 | When I first started in the data world, the first tool I used to build a data pipeline was SSIS. Since then it feels like I have come across every possible tool and custom data pipeline set-up possible(of course thats far from the truth). There seem to be hundreds of tools and methods for data teams use to get data from point A to point B. So I wanted to share some of those experiences as well as hear from Daniel Palma's experiences building data pipelines. What has changed? What has stayed the same? What challenges do data engineers still face today? Feel free to share some of your questions below!Also, if you're looking to build a future data pipeline, you should read this article - bit.ly/40Vi8kA
Building A Data Engineering Project - Starting A Data Analytics Project With Real Healthcare Data Seattle Data Guy 2024-11-21 | Finding data sets that mimic real life data sets for a data engineering project is hard. But there are several synthetic healthcare data sets that look almost exactly(or at least have all the right data) as the real deal.That includes the synthetic claims data created by CMS.In this project we'll be taking a look at this data and just starting to extract it. But there really are so many ways we can take this project.Docker Gistgist.github.com/sdg-1/bc4ae0cfb2e2263c92c3efb2f0b1f624Airflow Gistgist.github.com/sdg-1/a409fbe20b2848ff92cf2ceaa0fa392aI should probably start putting this all in one projectIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Data Engineering Consulting - The Different Ways You Can Make Money Consulting Seattle Data Guy 2024-11-13 | Many people continue to ask me about consulting. Everything from the basics such as marketing and pricing as well as plenty of other questions. One of the topics that has come up recently is what are the different ways you can make money consulting and there are a few! So let's talk about them. Also, if you'd like to learn more about consulting then you can join my free consulting community! https://the-technical-freelancer-academy.circle.so/c/resources/
Building Your Own Data Pipeline Tool From Scratch - Should You Do It? Seattle Data Guy 2024-11-12 | Alright, let's start out with the fact that there are some distinctions between an orchestrator and a data pipeline tool. But for many data teams, they use Airflow as a tool to either act as their data pipeline tool, or the tool that orchestrates all the other tools that make up their data pipeline.As you start building your first data pipelines, you’ll slowly realize you need to address a growing number of recurring issues. Maybe you implement a component or process that tracks what jobs are running, a scheduler, a set of generic scripts to run transforms and data ingestion, or even some form of UI.Before you know it, you’ve pieced together something that looks like Airflow. Something that goes beyond just being a set of data pipelines but starts looking like an orchestrator.Surprisingly (or maybe not), I’ve seen countless homegrown orchestration/data pipeline systems. Often, it feels like, given enough time, the team might build its own Airflow-esque solution.So should you build it?If you prefer reading, here is a written version of this - seattledataguy.substack.com/p/should-you-build-a-custom-data-orchestrationAlso, if you're looking for an orchestrator, consider checking out Mage!mage.aiIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
What Types Of Projects Should Data Teams Work On? Seattle Data Guy 2024-10-18 | Joe Reis recently wrote a piece titled, Playing Not To Lose. The piece referenced the fact that many data teams are simply going through the motions and doing "data stuff" as he put it. Gordon Wong later tagged in and referenced the point of being defensive vs offensive in terms of what work you take on. This made me want to dig even deeper into what Joe Reis meant and how he feels data teams can go on the offensive effectively to take on projects that actually are worth investing in. Or perhaps are there certain companies that don't benefit as much from data teams? So don't miss out on this discussion!
Reviewing Data Engineering Memes And QA Seattle Data Guy 2024-10-17 | Happy Wednesday! Breaking it up today and going back through some memes I have put out in the past. But also feel free to send some content my way if you'd like me to read something or review it. Also, if you'd like to dive deeper into data strategy and infrastructure and you'd like to support me, you can consider becoming a paid member of my Substack. I have over 100 articles that cover everything from data engineering 101 to leading data teams. Sign up with the link below and get 30% off. - seattledataguy.substack.com/148e9023 Or check out my blog theseattledataguy.com And if you want to support the channel, then you can become a paid member of my newsletter seattledataguy.substack.com/subscribe Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio _____________________________________________________________ Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1 _____________________________________________________________ About me: I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data. *I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Get Me Out Of Data Hell Seattle Data Guy 2024-10-16 | Here is the article I am reading! https://ludic.mataroa.blog/blog/get-me-out-of-data-hell/#fn:1I actually am yet to read it myself but thus far I have always enjoyed the authors style and perspective on data.So let's see where this goes! Also, if you'd like to dive deeper into data strategy and infrastructure and you'd like to support me, you can consider becoming a paid member of my Substack. I have over 100 articles that cover everything from data engineering 101 to leading data teams. Sign up with the link below and get 30% off. - seattledataguy.substack.com/148e9023 Or check out my blog theseattledataguy.com And if you want to support the channel, then you can become a paid member of my newsletter seattledataguy.substack.com/subscribe Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio _____________________________________________________________ Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1 _____________________________________________________________ About me: I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data. *I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Growing As A Data Engineering - And Open Q&A Seattle Data Guy 2024-10-15 | Let's talk data engineering and growing in your career. I've got a few points I'd love to cover. But I'll also open up the floor for questions!If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Data Engineering Ask Me Anything With Seattle Data Guy - 2024 Seattle Data Guy 2024-10-12 | What questions do you have about data engineering? Are you curious about different tools, skills, how can I help answer them? I haven't done a live where it's just me in a while and I wanted to open it up to y'all!1:18 - Hey how are you?2:29 - What's the best 'stack' for data engineering?3:32 - What made you start this meeting at the 25th minute instead of the 30th?3:50 - Which cloud provider should I choose, and also is easy to switching between them if we know concepts?5:10 - Any tips for my first interview in big tech in UK?6:30 - What niche within data engineering is most rewarding at the moment? It's better to be a generalista or specialist?8:20 - What do we need to study to slowly learn and create architecture for project?9:34 - I have 2.5 years as Qa Tester, can I show this experience in my data engineer resume?10:20 - Which is best snowflake or Databricks?12:15 - I have pivoted Towards data platform engineer from Analytics engineer. I feel like my current job is more replaceable by AI, thoughts?13:00 - How much DSA is enough DSA for data engineers?14:05 - I Spent 20 years in the ERP and HCM applications, and moved to the enterprise data team last month. What tips do you have for me?15:45 - Whats easier pivoting rom backend dev to DE or the Other way around? What do you think is better to start with?16:59 - It appears data governance and excellence are limited to the enterprise data teams. The application teams do not pay much attention to it. Any advice?18:52 - What AWS course do you suggest for begginers?19:42 - Can you suggest some projects tha can help us to get a data enginner job? And how to prepare a resume? 22:20 - I have just started upwork freelancing around B.I. I want to figure out how to make na offer24:15 - How do I stand out as a DE as na applicant or as a worker in general? What makes one DE better than the Other?26:21 - I have no background in it but completed aws sol architect asso and aws dev associate. A company hired e as aws data engineer, idk even know how to code. What do you suggest i do?27:30 - How much domain knowledge would you say is required to get into Consulting and get a Project?29:35 -Is there like blind 75 leetcode for sql that i need to study to prepare for a Technical interview for DE?Trying to transition from web developer to DE.31:03 - I don't have system design skills, i have interview in coming monday, any suggestions?33:00 - In na enterprise seeting do analysts usually have acess to the dw and query the data they need? Or the DE teams just give them flat files such as csv/parquet for them to Work on?34:18 - For a few 10s of gb of data, do you recommend big data tools like spark or Athena?. Can we achiev the same with more relational databases ike mysql? Which one is better in terms of cost and future scale?36:05 - I have na interview on Monday with a startup working on customer engagement like attribution and events data.Any advise on how to not blow this?37:10 - Have you ever worked as a geospatial DE with satellite data?37:42 - How can I learn DE, I have some background as web dev with php and mysql, but idk how to become DE38:28 - We are implementing data contracts. We need to drive cultural change. Any tips on how to get an org to focus on data in sources. In a scandi country that has flat orgs so cant rely on mgmt43:32 - Im a flutter dev, do you recommend me learning data sicence or should I be full stack data?44:40 - Hi, 3 years back I just landed into DE after graduation, and directly started with snowflake and i am good with it. I want to become a complete DE, what should i do46:24 - Got a job from chat gpt, but idk anything what to do?49:35 - As a DE do we need to understand DSA to land Jobs? And there are barries for e to land International remote from Nigeria? even thought I have 3 years exp, what do you suggest?51:15- DE and MLE need to know leetcode?52:11- How do you make sure your pipeline is idempotente? We ingest Gbs of files on hourly-basis, partition of table is on day-basis. How do i make sure I ingest corrected hourly file without overwriitng daypartition?53:30 - Why are there not more DE Jobs Towards apache superste Analytics?55:50- Starting DE in 2025, is it Worth?58:10 - Suggestions on changing schema? How do you change a column datatype r add a newcolumn/remove a column withtout overwriting the table? Table is in PB of size1:00:10 - AS a entry level DE, do I need to know Kafka?1:00:56 - Any suggestion on dealing with legacy systems?1:01:25 - Should MLE learn Databricks or snowflake? 1:01:57 - How much did you spend upskilling that got you to where you are right now?1:03:13 - Did you ever go through the whole imposter syndrome phase?1:05:50 - I Work as DE and i fell like i am Always 3 to 5 years behind in current tech. What are your main sources to be up to date with your tech stack?
Mistakes I Have Made As A Data Engineering Consultant - Avoid These Mistakes As A Consultant Seattle Data Guy 2024-10-11 | When you first start your own company, you'll make a few mistakes.Some will be minor, others could be costly.In this video I'll talk about my experiences as a data engineering consultant and the mistakes I have made along the way.Also, if you'd like to join a community of consultants with hours of free content covering topics like how to market, sell and so much more, then check out the TFA community - https://the-technical-freelancer-academy.circle.so/c/resources/If you enjoyed this video, check out some of my other top videos.Top Courses To Become A Data Engineeryoutube.com/watch?v=kW8_l57w74gWhat Is The Modern Data Stack - Intro To Data Infrastructure Part 1youtube.com/watch?v=-ClWgwC0SbwIf you would like to learn more about data engineering, then check out Googles GCP certificatebit.ly/3NQVn7VIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Extracting Data From APIs As Data Engineers - The Basics And Challenges Youll Run Into Seattle Data Guy 2024-10-04 | If you've had to build any data pipelines for analytics, then you're likely very familiar with the extract phase of an ELT or ETL.As the name suggests the extract phase is when you connect to a data source and "extract" data from it. The most common data sources you'll be interacting with being databases, APIs, and file servers(via FTP or SFTP).With my recent focus on going back to the basics, it occurred to me that I have never written about APIs and how we interact with them as data engineers.Now, there are plenty of APIs that have caused me a lot of heartburn in my career and there are others that have been a piece of cake to handle.But it all comes down to how the API is set up and the design choices made when it was built.If you're looking for an out of the box solution to handle your API data extraction. You can check out the two below:Portable For APIs - portable.ioEstuary For Real Time Data Extraction - bit.ly/4eQC3oQDisclosure - I have a financial stake in bothAlso, if you'd like to dive deeper into data strategy and infrastructure and you'd like to support me, you can consider becoming a paid member of my Substack. I have over 100 articles that cover everything from data engineering 101 to leading data teams. Sign up with the link below and get 30% off. - seattledataguy.substack.com/148e9023If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Setting Up Data teams - In The Age Of AI With Aaron Wilkerson Seattle Data Guy 2024-10-03 | As 2025 approaches, I am sure there are plenty of data teams being pushed to add AI to their data strategy roadmaps. Even if it's not clear where it actually fits. So I wanted to talk to the one and only Aaron Wilkerson about his experiences leading teams, especially now, in this world where it can be hard to sift between hype and reality. As someone who has been working leading data teams and initiatives Aaron is a straight shooter, and if you haven't seen some of his past interviews, you should go check them out! If you want to connect with him, you can find him here - linkedin.com/in/aaron-wilkerson-81bb21a
From Hackathons to Co-Founding - Analyzing Unstructured Data With SQL Seattle Data Guy 2024-10-01 | One of the benefits of working in big tech is you often build or see toolings and solutions internally that could benefit a broader audience. Whether it's a query engine or a data storage format. Richard Meng has been working as a Staff Software Engineer at companies like Linkedin and Snowflake with data as a particular focus. Most recently his experiences at Snowflake provided him with several insights into how LLMs can better be used to process data. In particular, unstructured data. For example, instead of going the standard method of chunking the documents and embedding them into vectors to allow future users to ask questions of said data. Richard's team believes in directly using vision LLM on unstructured data in their raw format, which preserves the original context of said unstructured data as much as possible. In this live chat, I'll be focusing on Richard's experiences that led up to this point and how he is viewing LLMs impacting the work of data engineers. If you'd like to reach out to Richard and ask some questions, you can reach out here: linkedin.com/in/berkeleymengAlso, if you are looking to analyze PDFs or Images with SQL, you can try out Roe here.getroe.aiDisclosure: Seattle Data Guy does have a stake in Roe.AI
Finding The Right Data Projects To Work On And Getting Business Buy-In Seattle Data Guy 2024-09-26 | What does it mean to work on impactful data projects? How in the world do you find them? And once you find them, how do you work with the business to get buy-in? These are just a few of the questions I'll be asking Eric Gonzalez tomorrow. Let me know if you have any further questions!
Understanding SQL Query Execution - What Happens When You Run A SQL Command Seattle Data Guy 2024-09-20 | Here is something school probably didn't teach you about SQLOr at the very least you likely forgot.When you write a query, hit submit, and then run the query or that little triangle in DBBeaver……what exactly happens?Sure, you likely understand that data is pulled from multiple tables, data is filtered, and aggregations occur.But behind the scenes, what is going on?Read The Full Article Hereseattledataguy.substack.com/p/behind-the-scenes-of-sql-understandingIf you enjoyed this video, check out some of my other top videos.Top Courses To Become A Data Engineer youtube.com/watch?v=kW8_l57w74gWhat Is The Modern Data Stack - Intro To Data Infrastructure youtube.com/watch?v=-ClWgwC0SbwIf you would like to learn more about data engineering, then check out Googles GCP certificatebit.ly/3NQVn7VIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
This Isnt A Good Time To Become A Consultant - Can You Actually Quit And Start Your Own Thing Seattle Data Guy 2024-09-17 | When is it a good time to become a consultant? Recently I read Richard's article below about how this isn't a good time to become a consultant, where he discussed when it's a good vs bad time to become a consultant.So I wanted to add my own thoughtsYou can read Richard's article here richardmillington.com/p/whentobecomeaconsultantYou can also join the TFA community here for free - https://the-technical-freelancer-academy.circle.so/c/start-here/You can also read my consulting newsletter here:dcubed.substack.comIf you enjoyed this video, check out some of my other top videos.The Ultimate Guide To Starting An Independent Consulting Companyyoutu.be/RYjLU2N8K70If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribe_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Starting A Technical Consulting Company - Ask Me Anything Seattle Data Guy 2024-09-14 | Have you considered starting a consulting? But you're not 100% sure where to start? Perhaps you're stuck on marketing, pricing, sales, or just what admin tools you should use? Well that's what this AMA is for! I want to answer your questions! So please feel free to share some of them below. Also, if you're interested in joining a growing community of technical consultants. Then you should check out the TFA community here - https://the-technical-freelancer-academy.circle.so/c/resources/ There are already hours of a free content you can check out that will likely answer many of your questions.
Reviewing The Last 10 Years Of Data Engineering - With Daniel Palma Seattle Data Guy 2024-09-06 | Nearly 10 years ago I got my first job in data analytics. In some ways a lot has changed since then. In other ways, not much has changed at all. I wanted to talk to Daniel Palma who has also been working as a data engineer for about 10 years what he has seen as he has worked as a data engineer, consulting and leading teams.You can look up Daniel here - linkedin.com/in/danthelionAlso, Daniel has recently switched roles from working as a manager in data engineering to partnering with Estuary. You can learn more belowbit.ly/3Svmeen
Build A Data Stack That Lasts - How To Ensure Your Data Infrastructure Is Maintainable Seattle Data Guy 2024-08-29 | As a consultant, I have been called in to review and, in many cases, replace dozens of half-finished, abandoned, and sometimes forgotten data infrastructure projects.The data infrastructure in a few cases may just need a little tweaking to operate effectively, but other times the project is either so incomplete or so lacking in a central design that the best thing to do is replace the old system.Trust me, I’d love it if I could come into a project and simply change a few lines of code, and then everything would just work. However, so many projects are filled with unclear design decisions or resume-driven development that were never rooted in good planning.Of course, business stakeholders may have also push to get things done quickly. Forcing data teams to take on tech debt that will never be fixed. Don’t get me wrong, you want to get things done and move projects forward. But taking on technical debt is a decision that needs to be made intentionally. Otherwise, like in resume driven development, your data infrastructure might disappear.This begs the question.How do you ensure the data infrastructure you’re building doesn’t get replaced as soon as you leave in the future?In this article I wanted to dive into the problems I often come into that require me to replace the current data infrastructure and how you can avoid it.So let’s dive in.If you enjoyed this video, check out some of my other top videos.Top Courses To Become A Data Engineeryoutube.com/watch?v=kW8_l57w74gIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
What Skills Do You Need To Lead A Data Team - With Ana Zapata Seattle Data Guy 2024-08-17 | I have been talking with data leaders from multiple industries lately. In particular, I have been diving into what it takes to lead a data team. This week I’ll be speaking with Ana Zapata she has been leading data teams for the past near decade in healthcare, and hospitality. Let me know if you have any questions you’d like to ask!
Snowflake vs Databricks - And the Battle For Iceberg Seattle Data Guy 2024-08-15 | Can Iceberg become the single data layer where all our analytical processing can occur on top of?If you enjoyed this video, check out some of my other top videos.Don't Lead A Data Team Before Watching This - 5 Lessons You Need To Know As A Head of Datayoutu.be/n74ke0Wjmi4Data Modeling Where Theory Meets Reality - How Different Companies I Worked At Modeled Their Datayoutu.be/rqLbn1PQPKAAlso, if you're looking to ingest data into your data warehouse, consider checking out Estuary!bit.ly/3SvmeenIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Big Data Vs Thick Data - Talking To Alyce Ge About Real Estate Data Seattle Data Guy 2024-08-09 | Data can be big, it can be small, and it can also be deep and rich. Regardless, how do you manage and process data of varying shapes? That's what I am going to talk to Alyce Ge about, specifically around Real-Estate data which she has been working in for the last few years. She'll share her experiences melding data from different data sources that range from public to third party data providers. What questions should I ask?
Navigating The Data Leadership Landscape - From IC To Director Seattle Data Guy 2024-08-07 | How do you go from analyst or data engineer to being a senior director of data analytics? And what skills do you need? Those are just a few of the questions I’ll be asking Celina Wong who has worked as a director of analytics for multiple companies and now runs her own consulting company. What other questions should I ask?
Can Data Engineers Do Better Than SQL? - Should We Replace It? Seattle Data Guy 2024-08-01 | I have seen plenty of arguments for and against SQL.After all, it's been around for decades, shouldn't we replace it with something new and better?Yet it remains. So i wanted to read through an article that discusses arguments against SQL.Article Sourceedgedb.com/blog/we-can-do-better-than-sqlIf you enjoyed this video, check out some of my other top videos.Top Courses To Become A Data Engineer In 2022youtube.com/watch?v=kW8_l57w74gWhat Is The Modern Data Stack - Intro To Data Infrastructure Part 1youtube.com/watch?v=-ClWgwC0SbwIf you would like to learn more about data engineering, then check out Googles GCP certificatebit.ly/3NQVn7VIf you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.seattledataguy.substack.com/Or check out my blogtheseattledataguy.comAnd if you want to support the channel, then you can become a paid member of my newsletterseattledataguy.substack.com/subscribeTags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio_____________________________________________________________Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1_____________________________________________________________About me:I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Leading Data Science Teams In Oil And Gas With Jessica Iriarte Seattle Data Guy 2024-08-01 | There are so many other industries out there besides the big tech. Many of who deal with very different data sets that interact with the real world. Jessica Iriarte has been leading these types of teams for several years and I wanted to learn more. What are the challenges you face when leading this types of teams and what type of projects do you take on! And of course, how do you bridge the gap between the business and data. I am super excited for this talk. Let me know if you have any questions.
Data Modeling - How Do Different Companies Model Their Data With Lenny Ardiles Seattle Data Guy 2024-07-23 | What is the reality of data modeling? Plenty of companies and individuals say it’s important. But what does data modeling look like at various companies? If you missed my last live where we talked about data modeling, here is another with Lenny Ardiles. He has worked at companies like Facebook, Amazon and Home Depot and I'll be asking him about his various experiences and lessons he has learned along the way!
