Build A Reasoning Model From Scratch 5: Inference Scaling 2 (Logprob Scoring, Self-Refinement) @SebastianRaschka
Build A Reasoning Model From Scratch 5: Inference Scaling 2 (Logprob Scoring, Self-Refinement)  @SebastianRaschka
Uploaded September 2026 | Updated November 2026, 4 hours ago
This second video on inference-time scaling, introduces different scoring functions to break ties in majority voting and also introduces self-refinement.

- Reasoning playlist: youtube.com/watch?v=Kh9mqTzjuEQ&list=PLfrHAQCRxtUs
- Reasoning Book: amzn.to/4aAKiFY
- Reasoning GitHub repo: github.com/rasbt/reasoning-from-scratch

- LLMs from Scratch book: amzn.to/4fqvn0D
- LLMs from Scratch repo: github.com/rasbt/LLMs-from-scratch
- LLMs from Scratch playlist: youtube.com/watch?v=yAcWnfsZhzo&list=PLTKMiZHVd_2IIEsoJrWACkIxLRdfMlw11&index=1

00:00 Introduction and inference-time scaling recap
05:02 Loading the pretrained LLM
08:00 Comparing and scoring model answers
10:18 Building a rule-based scorer
17:53 Token probabilities and sequence likelihood
26:47 Computing token probabilities in PyTorch
30:12 Token indexing and shifted targets
37:27 Log probabilities and numerical stability
45:57 Scoring answers with average log probabilities
56:24 How self-refinement works
59:07 Generating critiques and revised answers
1:01:00 Implementing the self-refinement loop
1:05:57 MATH-500 evaluation results
1:07:35 Takeaways and next steps
Build A Reasoning Model From Scratch 5: Inference Scaling 2 (Logprob Scoring, Self-Refinement)What I Learned From Implementing LLM Architectures From Scratch (And How to Get Started)Introduction to Generative Adversarial Networks (Tutorial Recording at ISSDL 2021)L16.1 Dimensionality ReductionL17.0 Intro to Variational Autoencoders   Lecture OverviewL5.0 Gradient Descent   Lecture Overview13.4.2 Feature Permutation Importance (L13: Feature Selection)L10.0 Regularization Methods for Neural Networks   Lecture OverviewL6.3 Automatic Differentiation in PyTorch   Code ExampleBuild A Reasoning Model From Scratch 7: Reinforcement Learning 2 (Tracking, KL Terms, And More)How majority voting works for LLMsL14.3: Architecture Overview
Sebastian Raschka |

Build A Reasoning Model From Scratch 5: Inference Scaling 2 (Logprob Scoring, Self-Refinement)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER