Small Language Models (SLMs) Are the Future: Fine-Tuning AI That Runs on Your iPhone @mrdbourke
Small Language Models (SLMs) Are the Future: Fine-Tuning AI That Runs on Your iPhone  @mrdbourke
Uploaded March 2026 | Updated November 2026, 1 day ago
In this talk, I go over the rise of small language models (SLMs) and how they can benefit your business or day to day life.

Talk date: March 12, 2026 at the Queensland AI Meetup.

We'll look at case studies such as Sunny, an iOS application which uses a fine-tuned version of MedGemma to privately track skin health on-device.

We'll break down why on-device inference matters (privacy, offline access, zero ongoing cost) and compare the economics of local models versus cloud API pricing at scale.

Then we'll discuss the hardware and software optimizations required to run a model on a compute constrained device.

Finally, we'll get hands-on and fine-tune a small language model live.

We'll walk through how to build a custom dataset, set up supervised fine-tuning using Hugging Face's SFT Trainer, and fine-tune a small model, Gemma 3 270M, in about two minutes on an RTX 6000 Blackwell GPU on Google Colab.

We'll see how the base model's outputs to the fine-tuned version side by side, showing how even a small model can be customized to know specific people, handle edge cases, and refuse to answer questions it shouldn't.

Links:
Colab Notebook we used - dbourke.link/qldai-colab-notebook-llm-fine-tune-march-2026
End-to-end Small Language Model Fine-tuning Tutorial - learnhuggingface.com/notebooks/hugging_face_llm_full_fine_tune_tutorial

Courses I teach:
Learn AI/ML (beginner-friendly course) - dbourke.link/ZTMMLcourse
Learn Hugging Face - dbourke.link/ZTMHuggingFace
Learn TensorFlow - dbourke.link/ZTMTFcourse
Learn PyTorch - dbourke.link/ZTMPyTorch

Connect elsewhere:
Download Nutrify (my startup) - apple.co/4ahM7Wc
My website - mrdbourke.com
X/Twitter - twitter.com/mrdbourke
LinkedIn - linkedin.com/in/mrdbourke
Get email updates on my work - dbourke.link/newsletter
Read my novel Charlie Walks - charliewalks.com

Timestamps:
0:00 - Intro
2:19 - About me
4:09 - Case study: Sunny
7:55 - Benefits of small language models running on-device
8:44 - Cost savings of on-device models
9:29 - Case study: Sunny (hardware overview)
10:55 - Current best practice for running VLMs on iPhone
12:35 - Case study: Sunny (memory usage in Xcode)
13:56 - Case study: Sunny (workflow overview)
15:25 - Jeff Dean on precision
16:46 - Precision breakdown
17:28 - Effects of quantization on model size footprint
17:48 - Saving memory by reducing token usage
19:28 - Before and after different on-device experiments
20:29 - Case studies for other small but useful language models
24:00 - Case study for private VLM-based surveillance
25:04 - Small language models features and benefits
25:47 - How to pick a model for your use case
26:06 - Question: What hardware is required for getting started?
27:21 - Prompting vs fine-tuning vs RAG
28:08 - Live LLM fine-tuning problem overview
28:50 - How I made a dataset for fine-tuning Gemma 3
33:08 - Live fine-tuning code begins in Google Colab
36:56 - Data = a guide for what you want your model to do
40:37 - Question: How do you know if your fine-tuned model is performing well?
44:24 - Comparing the base model to the fine-tuned model
53:23 - Demo'ing our fine-tuned model on Hugging Face Spaces
56:00 - Haiku
57:33 - Contact me
59:05 - Q&A
Small Language Models (SLMs) Are the Future: Fine-Tuning AI That Runs on Your iPhoneIts my birthday! Come and ask me anything! (livestream party)Benchmarking: NVIDIA DGX Spark & NVIDIA RTX 4090 | Part 4 - Comparing the resultsHow to learn data science with Coursera in 2022 (beginner-friendly)Got 20-minutes? This whole body workout will help you get shreddedHow I made my first $1500 online: The 2-Year Self ApprenticeshipFix your tight hips and shoulders in 10-minutes (ball joint mobility session)Y Combinator Startup School week 1 progress10-minute Lower Body Bodyweight Workout for BeginnersStudy with me: Data Science (25-minute session)5 Beginner Friendly Steps to Learn Machine LearningArtificial Intelligence in Business Keynote at The University of Queensland
Daniel Bourke |

Small Language Models (SLMs) Are the Future: Fine-Tuning AI That Runs on Your iPhone

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER