Back To The Basics Of Data Engineering - Why SFTP Remains Essential for Companies @SeattleDataGuy
Back To The Basics Of Data Engineering - Why SFTP Remains Essential for Companies  @SeattleDataGuy
Uploaded March 2025 | Updated November 2026, 3 hours ago
Every day, companies share data sets of users, patient claims, financial transactions, and more with each other.

Most people might assume this would be via API. However, companies have been sharing data for decades using CSVs, TSVs, positional files, and other formats you might not be familiar with. Not via API, but SFTP.

I know I wouldn’t have guessed that’s how companies send data back and forth when I was in school.

If you’ve been in the industry for a while, you’ve probably come across automated SFTP jobs that do just that. You’ve also likely had to encrypt or decrypt a CSV and had to interpret a schema file with parsing instructions that—somehow—are always a bit off the first time.

Now, sure, we all want to build real-time systems that use LLMs and other flashy new tools and solutions. Sometimes, that’s not what is called for.

After all, companies of all sizes—even tech giants like Facebook and Airbnb—still use SFTP to share critical information for analytical purposes(as well as operational). So, let’s dig into what SFTP is and how you will likely work with it.

If you want to read the full article, you can find it here-
seattledataguy.substack.com/p/data-sharing-in-the-real-world-why

Or check out my blog
theseattledataguy.com

And if you want to support the channel, then you can become a paid member of my newsletter
seattledataguy.substack.com/subscribe


Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio

_____________________________________________________________
Subscribe: youtube.com/channel/UCmLGJ3VYBcfRaWbP6JLJcpA?sub_confirmation=1
_____________________________________________________________
About me:
I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.

*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
Back To The Basics Of Data Engineering - Why SFTP Remains Essential for CompaniesData Architecture And Leadership - Whats Changed In The Last DecadeWill Data Engineering Exist In 5 Years - Is Data Engineering A Good Career Choice?Operationalizing Data - Moving Beyond Analytics, Turning Data Into ActionData Engineering Vs Machine Learning Pipelines - What Is The DifferenceData Modeling - Walking Through How To Data Model As A Data Engineer - Dimensional Modeling 101Seattle Data Guy LiveThe Realities Of Airflow - The Mistakes New Data Engineers Make Using Apache AirflowWhy Are Data Teams Still Struggling To Answer Basic QuestionsWhy Data Engineers LOVE/HATE Airflow (FT. @mehdio , @startdataengineering  and more!)Growing As A Data Engineering - And Open Q&AHow To Deliver Value To Clients - Talking To Matt Chung About Quitting Amazon To Consult
Seattle Data Guy |

Back To The Basics Of Data Engineering - Why SFTP Remains Essential for Companies

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER