Uploaded April 2026 | Updated September 2026, 1 week ago
Talk by David vonThenen
socallinuxexpo.org/scale/23x/presentations/practical-guide-training-small-language-model-tokenizers-training-and-real
This session walks through how to build and train your own Small Language Model using open source tools. You'll learn how to select datasets, reuse BERT's tokenizer, design a smaller student model, and apply knowledge distillation to preserve accuracy. We'll share code, show common pitfalls, and help you deploy an efficient, real-world SLM that runs fast on modest hardware.
Talk by David vonThenen
socallinuxexpo.org/scale/23x/presentations/practical-guide-training-small-language-model-tokenizers-training-and-real
This session walks through how to build and train your own Small Language Model using open source tools. You'll learn how to select datasets, reuse BERT's tokenizer, design a smaller student model, and apply knowledge distillation to preserve accuracy. We'll share code, show common pitfalls, and help you deploy an efficient, real-world SLM that runs fast on modest hardware.
