Uploaded February 2023 | Updated September 2026, 6 hours ago

---

Building data pipelines with #python is an important skill for data engineers and data scientists. But what's the best library to use? In this video we look at three options: pandas, polars, and spark (pyspark).

Timeline:
00:00 Data Pipelines
01:11 The Data
02:32 Pandas
04:34 Polars
06:15 PySpark
09:15 Spark SQL

Follow me on twitch for live coding streams: twitch.tv/medallionstallion_

My other videos:

Speed Up Your Pandas Code: youtube.com/watch?v=SAFmrTnEHLg
Intro to Pandas video: youtube.com/watch?v=_Eb0utIRdkw
Exploratory Data Analysis Video: youtube.com/watch?v=xi0vhXFPegw

Working with Audio data in Python: youtube.com/watch?v=ZqpSb5p1xQo
Efficient Pandas Dataframes: youtube.com/watch?v=u4_c2LDi4b8

* Youtube: youtube.com/@robmulla?sub_confirmation=1
* Discord: discord.gg/HZszek7DQc
* Twitch: twitch.tv/medallionstallion_
* Twitter: twitter.com/Rob_Mulla
* Kaggle: kaggle.com/robikscube

#python #polars #spark #dataengineering