Automate CSV to Parquet with Buckets, Webhooks & Jobs @HuggingFace
Automate CSV to Parquet with Buckets, Webhooks & Jobs  @HuggingFace
Uploaded October 2026 | Updated November 2026, 3 days ago
Files keep landing in your Hugging Face Storage Bucket. Here is how to process each one automatically, without downloading anything.

In this live-coding video we set up a webhook on a bucket that triggers a Hugging Face Job whenever a new file arrives. The Job runs the `optimize-parquet.py` uv script, converts the new CSV to optimized Parquet, and writes it to a second bucket. We upload a 10,000-row slice of the NYC taxi dataset, and the triggered Job finishes in about 8 seconds. For data engineers already using Buckets who want to automate what happens next.

Topics covered:
- Hugging Face Storage Buckets with the hf CLI
- Running a uv script as a Hugging Face Job
- Webhooks that trigger Jobs on bucket changes
- Converting CSV to Parquet automatically
- Checking Job status, logs and output

Resources:
- Webhooks guide (bucket to Jobs): huggingface.co/docs/hub/webhooks-guide-bucket-jobs
- optimize-parquet.py script: huggingface.co/datasets/uv-scripts/data-processing/raw/main/optimize-parquet.py
- My notes and demo code: github.com/prpatel/simple-buckets-demo
- Hands-on with Buckets (article): huggingface.co/blog/prpatel/hands-on-with-buckets

Chapters:
0:00 Intro
0:16 Why automate bucket processing
0:39 Input and output buckets
1:39 Define the Job
2:20 Create a webhook on the bucket
3:21 Upload a new CSV
4:01 Check the Job and its output
5:04 Wrap up
Automate CSV to Parquet with Buckets, Webhooks & JobsResults of the Open Source AI Game JamThe Context Course: a FREE course on building agents with agents #learning #ai #agentsAgentic Evaluations Workshop - Deep Dive on the Future on Evals for Agents.VQ-BeT: LeRobot Research Presentation #4 by Nur Muhammad Mahi Shafiullah & Sengjae (Jay) LeeπŸ€— Hugging Cast S2E2 - Accelerating AI with NVIDIA!Train AI Robots Without Writing Code! (Introducing LeLab)Are World Models the Next Big Thing? | Merve NoyanIntroducing Paper Pages on the Hugging Face Hub πŸ€— #shortsOne Minute Gradio #1: Dynamic RenderingWhat are Mixture-of-Experts Models | ft. AritraHugging Face Journal Club: Scaling Laws for Pre-training & RL
Hugging Face |

Automate CSV to Parquet with Buckets, Webhooks & Jobs

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER