Uploaded October 2026 | Updated November 2026, 2 days ago
Fine-tune an LLM for search relevance on a MacBook: OLMo 2 1B Instruct with a classification head, LoRA and the Hugging Face Trainer on PyTorch MPS, no NVIDIA GPU. It judges (search, product) pairs from Amazon's real shopping searches as Exact, Substitute, Complement or Irrelevant, and re-ranks results by that judgement: the order of the top 10 products (NDCG@10) goes 0.59 → 0.75, ahead of plain keyword search (BM25, 0.69), after 17 minutes of training on the Mac's GPU. Lesson 7 of the series Fine-tune LLMs on your Mac.
Full series playlist: youtube.com/playlist?list=PLbDRDyy1iJ_c
My full course, Fine Tuning LLM with Hugging Face Transformers for NLP: kgptalkie.com/fine-tuning-llm
All my courses: kgptalkie.com/udemy-courses
Dataset: huggingface.co/datasets/tasksource/esci
Amazon Shopping Queries Dataset (ESCI): github.com/amazon-science/esci-data
Paper: Reddy et al. 2022, Shopping Queries Dataset, arxiv.org/abs/2206.06588
00:00 The result
00:45 The job: re-rank search results
01:02 The model: OLMo 2 1B Instruct
01:25 The four labels: ESCI
01:46 The data: Amazon's shopping queries
02:00 Load the data: 23,884 pairs
02:35 One test search: psp go
03:05 The base model as a judge
03:30 Scores: accuracy and NDCG@10
04:14 The classification head
04:40 LoRA on OLMo 2
05:11 Training with the Hugging Face Trainer
05:48 Merge and use the model
06:17 Before vs after: 0.59 → 0.75
06:40 An honest note
07:09 Re-ranking psp go
07:29 Where it runs and Hugging Face
07:50 Course and recap
08:18 Next lesson
Lesson 6 (prompt-injection firewall with Falcon-H1): youtu.be/bzf0nMvzYpc
Lesson 5 (brand-voice support bot with MiniCPM5): youtu.be/VL10O4wkFu8
Lesson 4 (sort support tickets with Gemma 3): youtu.be/HsYpr0kttg4
Lesson 3 (System 1 decision model with Qwen 3.5): youtu.be/wr3dMotYFMU
Lesson 2 (invoice PDF to JSON with Granite 4.0): youtu.be/1U3u7kqz4zQ
Lesson 1 (text to SQL with MLX): youtube.com/watch?v=tsShl5GSGLE
Base model: huggingface.co/allenai/OLMo-2-0425-1B-Instruct
Machine: MacBook Pro, Apple M5 Max, 36 GB unified memory. LoRA rank 16 + a 4-label classification head, 1 epoch, 1,493 steps × 16 pairs (23,884 pairs from 1,200 searches), 16.6 min, 14.3 GB GPU memory, ≈ 12 ms per pair. Accuracy 41.8 → 52.2 %, NDCG@10 0.591 → 0.746 (BM25 0.692) on 4,703 test pairs from 250 searches.
Fine-tune an LLM for search relevance on a MacBook: OLMo 2 1B Instruct with a classification head, LoRA and the Hugging Face Trainer on PyTorch MPS, no NVIDIA GPU. It judges (search, product) pairs from Amazon's real shopping searches as Exact, Substitute, Complement or Irrelevant, and re-ranks results by that judgement: the order of the top 10 products (NDCG@10) goes 0.59 → 0.75, ahead of plain keyword search (BM25, 0.69), after 17 minutes of training on the Mac's GPU. Lesson 7 of the series Fine-tune LLMs on your Mac.
Full series playlist: youtube.com/playlist?list=PLbDRDyy1iJ_c
My full course, Fine Tuning LLM with Hugging Face Transformers for NLP: kgptalkie.com/fine-tuning-llm
All my courses: kgptalkie.com/udemy-courses
Dataset: huggingface.co/datasets/tasksource/esci
Amazon Shopping Queries Dataset (ESCI): github.com/amazon-science/esci-data
Paper: Reddy et al. 2022, Shopping Queries Dataset, arxiv.org/abs/2206.06588
00:00 The result
00:45 The job: re-rank search results
01:02 The model: OLMo 2 1B Instruct
01:25 The four labels: ESCI
01:46 The data: Amazon's shopping queries
02:00 Load the data: 23,884 pairs
02:35 One test search: psp go
03:05 The base model as a judge
03:30 Scores: accuracy and NDCG@10
04:14 The classification head
04:40 LoRA on OLMo 2
05:11 Training with the Hugging Face Trainer
05:48 Merge and use the model
06:17 Before vs after: 0.59 → 0.75
06:40 An honest note
07:09 Re-ranking psp go
07:29 Where it runs and Hugging Face
07:50 Course and recap
08:18 Next lesson
Lesson 6 (prompt-injection firewall with Falcon-H1): youtu.be/bzf0nMvzYpc
Lesson 5 (brand-voice support bot with MiniCPM5): youtu.be/VL10O4wkFu8
Lesson 4 (sort support tickets with Gemma 3): youtu.be/HsYpr0kttg4
Lesson 3 (System 1 decision model with Qwen 3.5): youtu.be/wr3dMotYFMU
Lesson 2 (invoice PDF to JSON with Granite 4.0): youtu.be/1U3u7kqz4zQ
Lesson 1 (text to SQL with MLX): youtube.com/watch?v=tsShl5GSGLE
Base model: huggingface.co/allenai/OLMo-2-0425-1B-Instruct
Machine: MacBook Pro, Apple M5 Max, 36 GB unified memory. LoRA rank 16 + a 4-label classification head, 1 epoch, 1,493 steps × 16 pairs (23,884 pairs from 1,200 searches), 16.6 min, 14.3 GB GPU memory, ≈ 12 ms per pair. Accuracy 41.8 → 52.2 %, NDCG@10 0.591 → 0.746 (BM25 0.692) on 4,703 test pairs from 250 searches.










