Uploaded June 2026 | Updated November 2026, 5 days ago
Training Agents, Session 1: from raw agent traces to a supervised fine-tuning baseline.
In this live session, I’ll set up the first rung of an agentic post-training workflow: SFT. We’ll take public coding-agent traces, turn them into prompt/completion training data, run a small TRL + LoRA fine-tune, and inspect what the first metrics can and cannot tell us.
What we’ll cover:
- Why start with SFT before GRPO or environment RL
- How agent traces become training examples
- Completion-only loss for chat/tool traces
- Running TRL SFT on Hugging Face Jobs
- Keeping experiments reproducible without checking in logs or checkpoints
- What the first eval metrics do and do not prove
0:00 - Introduction and welcome to the first class
4:42 - Meet the team: Introduction of Sergio and the TRL maintainers
5:36 - The starting point: Exploring an agent trace
6:22 - Using agents to train agents (the "meta" approach)
8:03 - Defining the prompt and training parameters
8:45 - Selecting the dataset: Real-world agent traces
10:19 - Deep dive into Supervised Fine-Tuning (SFT) for agents
16:42 - Understanding the agent "contract" and planning
18:42 - Why train on traces? Learning the agent language
21:56 - Anatomy of a trace: Multi-turn tool calls and logic
24:32 - Agent execution: Loading skills and interacting with the Hub
26:05 - Exploring the repository and custom skills
29:39 - Reviewing the agent-generated training script
41:10 - Educational overview: What is SFT at a code level?
44:34 - Data formatting: Prompt completions and masking
51:39 - Monitoring progress: Using the Trackio dashboard for metrics
55:45 - Evaluating performance: Benchmarks and human eval
58:00 - Summary of resources and next steps for the series
Repo:
github.com/burtenshaw/training-agents
Guide:
https://x.com/ben_burtenshaw/status/2067615361428545566?s=20
Slides:
docs.google.com/presentation/d/1hcGZ4U9TjZZzcGNbH2K6wYD45qwZTyo_gosCQsnHlnc/edit?usp=sharing
Agent Trace:
huggingface.co/buckets/burtenshaw/sft-on-traces/tree/example.jsonl
TrackIO:
huggingface.co/spaces/burtenshaw/youtube-livestream-1-gemma4-pi-mono-trackio
This is part of the Training Agents series: using coding agents to design, run, monitor, and review post-training experiments, while training models to become better agents.
#TRL #HuggingFace #PostTraining #AIAgents #FineTuning
Training Agents, Session 1: from raw agent traces to a supervised fine-tuning baseline.
In this live session, I’ll set up the first rung of an agentic post-training workflow: SFT. We’ll take public coding-agent traces, turn them into prompt/completion training data, run a small TRL + LoRA fine-tune, and inspect what the first metrics can and cannot tell us.
What we’ll cover:
- Why start with SFT before GRPO or environment RL
- How agent traces become training examples
- Completion-only loss for chat/tool traces
- Running TRL SFT on Hugging Face Jobs
- Keeping experiments reproducible without checking in logs or checkpoints
- What the first eval metrics do and do not prove
0:00 - Introduction and welcome to the first class
4:42 - Meet the team: Introduction of Sergio and the TRL maintainers
5:36 - The starting point: Exploring an agent trace
6:22 - Using agents to train agents (the "meta" approach)
8:03 - Defining the prompt and training parameters
8:45 - Selecting the dataset: Real-world agent traces
10:19 - Deep dive into Supervised Fine-Tuning (SFT) for agents
16:42 - Understanding the agent "contract" and planning
18:42 - Why train on traces? Learning the agent language
21:56 - Anatomy of a trace: Multi-turn tool calls and logic
24:32 - Agent execution: Loading skills and interacting with the Hub
26:05 - Exploring the repository and custom skills
29:39 - Reviewing the agent-generated training script
41:10 - Educational overview: What is SFT at a code level?
44:34 - Data formatting: Prompt completions and masking
51:39 - Monitoring progress: Using the Trackio dashboard for metrics
55:45 - Evaluating performance: Benchmarks and human eval
58:00 - Summary of resources and next steps for the series
Repo:
github.com/burtenshaw/training-agents
Guide:
https://x.com/ben_burtenshaw/status/2067615361428545566?s=20
Slides:
docs.google.com/presentation/d/1hcGZ4U9TjZZzcGNbH2K6wYD45qwZTyo_gosCQsnHlnc/edit?usp=sharing
Agent Trace:
huggingface.co/buckets/burtenshaw/sft-on-traces/tree/example.jsonl
TrackIO:
huggingface.co/spaces/burtenshaw/youtube-livestream-1-gemma4-pi-mono-trackio
This is part of the Training Agents series: using coding agents to design, run, monitor, and review post-training experiments, while training models to become better agents.
#TRL #HuggingFace #PostTraining #AIAgents #FineTuning










