LLMs: A Journey Through Time and Architecture Sebastian Raschka 2024-09-24 | REFERENCES:- Step-by-step guide converting GPT to Llama: github.com/rasbt/LLMs-from-scratch/blob/main/ch05/07_gpt_to_llama/converting-gpt-to-llama2.ipynb- Build a Large Language Model (From Scratch): http://mng.bz/M96o- LitGPT: github.com/Lightning-AI/litgpt- The Llama 3 Herd of Models (31 July 2024), arxiv.org/abs/2407.21783- Qwen2 Technical Report (15 July 2024), arxiv.org/abs/2407.10671- Apple Intelligence Foundation Language Models (29 July 2024), arxiv.org/abs/2407.21075- Gemma 2: Improving Open Language Models at a Practical Size (31 July 2024), arxiv.org/abs/2408.0011DESCRIPTION:In this video, you'll learn about the architectural difference between the original GPT model and the various Llama models. Moreover, you'll also learn about new pre-training recipes used for Qwen 2, Gemma 2, Apple's Foundation Models, and Llama 3, as well as some of the efficiency tweaks introduced by Mixtral, Llama 3, and Gemma 2.---To support this channel, please consider purchasing a copy of my books: sebastianraschka.com/books---twitter.com/rasbtlinkedin.com/in/sebastianraschkamagazine.sebastianraschka.com---OUTLINE:0:00 Introduction2:05 Pre-training in 20246:11 GPT-architecture vs Llama11:27 GPT and other architectures16:47 Takeaways
Building LLMs from the Ground Up: A 3-hour Coding Workshop Sebastian Raschka 2024-08-31 | REFERENCES:1. Build an LLM from Scratch book: amzn.to/4fqvn0D2. Build an LLM from Scratch repo: github.com/rasbt/LLMs-from-scratch3. GitHub repository with workshop code: github.com/rasbt/LLM-workshop-20244. Lightning Studio for this workshop: lightning.ai/lightning-ai/studios/llms-from-the-ground-up-workshop?view=public5. LitGPT: github.com/Lightning-AI/litgptDESCRIPTION:This tutorial is aimed at coders interested in understanding the building blocks of large language models (LLMs), how LLMs work, and how to code them from the ground up in PyTorch. We will kick off this tutorial with an introduction to LLMs, recent milestones, and their use cases. Then, we will code a small GPT-like LLM, including its data input pipeline, core architecture components, and pretraining code ourselves. After understanding how everything fits together and how to pretrain an LLM, we will learn how to load pretrained weights and finetune LLMs using open-source libraries.---To support this channel, please consider purchasing a copy of my books: sebastianraschka.com/books---twitter.com/rasbtlinkedin.com/in/sebastianraschkamagazine.sebastianraschka.com---OUTLINE:0:00 – Workshop overview2:17 – Part 1: Intro to LLMs9:14 – Workshop materials10:48 – Part 2: Understanding LLM input data23:25 – A simple tokenizer class41:03 – Part 3: Coding an LLM architecture45:01 – GPT-2 and Llama 21:07:11 – Part 4: Pretraining1:29:37 – Part 5.1: Loading pretrained weights1:45:12 – Part 5.2: Pretrained weights via LitGPT1:53:09 – Part 6.1: Instruction finetuning2:08:21 – Part 6.2: Instruction finetuning via LitGPT02:26:45 – Part 6.3: Benchmark evaluation02:36:55 – Part 6.4: Evaluating conversational performance02:42:40 – Conclusion
Understanding PyTorch Buffers Sebastian Raschka 2024-07-27 | Sebastian's books: sebastianraschka.com/booksThis video explains what PyTorch buffers are, a concept that is particularly useful when dealing with GPU computations and implement large models like LLMs.Code notebook: github.com/rasbt/LLMs-from-scratch/blob/main/ch03/03_understanding-buffers/understanding-buffers.ipynbGitHub discussion about "triu" in the forward pass: github.com/rasbt/LLMs-from-scratch/discussions/282Link to the Studio GPU environment to follow along: lightning.ai/seraschka/studios/understanding-pytorch-buffers?section=recent---To support this channel, please consider purchasing a copy of my books: sebastianraschka.com/books---twitter.com/rasbtlinkedin.com/in/sebastianraschkamagazine.sebastianraschka.com
Developing an LLM: Building, Training, Finetuning Sebastian Raschka 2024-06-06 | REFERENCES:1. Build an LLM from Scratch book: amzn.to/4fqvn0D2. Build an LLM from Scratch repo: github.com/rasbt/LLMs-from-scratch3. Slides: sebastianraschka.com/pdf/slides/2024-build-llms.pdf4. LitGPT: github.com/Lightning-AI/litgpt5. TinyLlama pretraining: lightning.ai/lightning-ai/studios/pretrain-llms-tinyllama-1-1bDESCRIPTION:This video provides an overview of the three stages of developing an LLM: Building, Training, and Finetuning. The focus is on explaining how LLMs work by describing how each step works.---To support this channel, please consider purchasing a copy of my books: sebastianraschka.com/books---twitter.com/rasbtlinkedin.com/in/sebastianraschkamagazine.sebastianraschka.com---OUTLINE:00:00 – Using LLMs02:50 – The stages of developing an LLM05:26 – The dataset10:15 – Generating multi-word outputs12:30 – Tokenization15:35 – Pretraining datasets21:53 – LLM architecture27:20 – Pretraining35:21 – Classification finetuning39:48 – Instruction finetuning43:06 – Preference finetuning46:04 – Evaluating LLMs53:59 – Pretraining & finetuning rules of thumb
Managing Sources of Randomness When Training Deep Neural Networks Sebastian Raschka 2024-04-16 | Sebastian's books: sebastianraschka.com/booksREFERENCES:1. Link to the code on GitHub: github.com/rasbt/MachineLearning-QandAI-book/tree/main/supplementary/q10-random-sources2. Link to the book mentioned at the end of the video: nostarch.com/machine-learning-q-and-aiDESCRIPTION:In this video, we managing common sources of randomness when training deep neural networks. We cover sources of randomness, including model weight initialization, dataset sampling and shuffling, nondeterministic algorithms, runtime algorithm differences, hardware and driver variations, and generative AI sampling.---To support this channel, please consider purchasing a copy of my books: sebastianraschka.com/books---twitter.com/rasbtlinkedin.com/in/sebastianraschkamagazine.sebastianraschka.com---OUTLINE:00:00 – Introduction01:14 – 1. Model Weight Initialization04:28 – 2. Dataset Sampling and Shuffling07:45 – 3. Nondeterministic Algorithms11:13 – 4. Different Runtime Algorithms14:30 – 5. Hardware and Drivers15:39 – 6. Randomness and Generative AI20:56 – Recap22:34 – Surprise
Insights from Finetuning LLMs with Low-Rank Adaptation Sebastian Raschka 2023-12-17 | Sebastian's books: sebastianraschka.com/booksLinks:- LoRA: Low-Rank Adaptation of Large Language Models, arxiv.org/abs/2106.09685- LitGPT: github.com/Lightning-AI/lit-gpt- LitGPT LoRA Tutorial: github.com/Lightning-AI/lit-gpt/blob/main/tutorials/finetune_lora.md Low-rank adaptation (LoRA) stands as one of the most popular and effective methods for efficiently training custom Large Language Models (LLMs). As practitioners of open-source LLMs, we regard LoRA as a crucial technique in our toolkit.In this talk, I will delve into some practical insights gained from running hundreds of experiments with LoRA, addressing questions such as: How much can I save with quantized LoRA? Are Adam optimizers memory-intensive? Should we train for multiple epochs? How do we choose the LoRA rank?---To support this channel, please consider purchasing a copy of my books: sebastianraschka.com/books---twitter.com/rasbtlinkedin.com/in/sebastianraschkamagazine.sebastianraschka.com
Finetuning Open-Source LLMs Sebastian Raschka 2023-10-14 | Sebastian's books: sebastianraschka.com/booksThis video offers a quick dive into the world of finetuning Large Language Models (LLMs). This video covers - common usage scenarios for pretrained LLMs- parameter-efficient finetuning- a hands-on guide to using the 'lit-GPT' open-source repository for LLM finetuning#FineTuning #LargeLanguageModels #LLMs #OpenAI #DeepLearningUseful links to resources discussed in this video: Code for the LLM classifier: github.com/rasbt/LLM-finetuning-scripts/tree/main/conventional/distilbert-movie-reviewLit-GPT repository: github.com/Lightning-AI/lit-gptNeurIPS LLM efficiency challenge: llm-efficiency-challenge.github.ioMy latest articles on LLM research: magazine.sebastianraschka.com---To support this channel, please consider purchasing a copy of my books: sebastianraschka.com/books---twitter.com/rasbtlinkedin.com/in/sebastianraschka
Scaling PyTorch Model Training With Minimal Code Changes Sebastian Raschka 2023-06-22 | Sebastian's books: sebastianraschka.com/booksCode examples: github.com/rasbt/cvpr2023In this short tutorial, I will show you how to accelerate the training of LLMs and Vision Transformers with minimal code changes using open-source libraries.---To support this channel, please consider purchasing a copy of my books: sebastianraschka.com/books---https://x.com/rasbtlinkedin.com/in/sebastianraschkamagazine.sebastianraschka.com
L13.5 Whats The Difference Between Cross-Correlation And Convolution? Sebastian Raschka 2022-08-03 | Sebastian's books: sebastianraschka.com/booksThis replaces a previous video where the video & audio were out of sync.Slides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L13_intro-cnn__slides.pdfCode: github.com/rasbt/stat453-deep-learning-ss21/blob/main/L13/code/notes/cross-correlation.ipynb
Conditional Ordinal Regression for Neural Networks (CORN) With Examples in PyTorch Sebastian Raschka 2022-07-15 | Sebastian's books: sebastianraschka.com/booksUsing deep neural networks for prediction problems where the labels have a natural order.Link to the code and slides: github.com/rasbt/scipy2022-talk
The Three Elements of PyTorch Sebastian Raschka 2022-05-10 | Sebastian's books: sebastianraschka.com/booksCode: github.com/rasbt/machine-learning-notes/blob/main/demos/basic-pytorch-cnn-for-3-ele-pytorch-video.ipynbSlides: sebastianraschka.com/pdf/slides/2022-05_three-elements-pytorch.pdf00:00 Three elements of PyTorch02:10 (1) Tensor library05:56 (2) Automatic differentiation engine13:32 (3) Deep learning library14:27 PyTorch in 3 Steps15:17 Step 1: defining the model23:32 Step 2: defining the training loop30:20 Step 3: defining the dataset39:34 Why do I like PyTorch?42:25 Hands-on code demoThis talk is an hour long introduction to PyTorch focusing on its three core elements: tensor (array) computing, automatic differentiation, and deep learning utilities.
Ratings and Rankings Using Deep Learning When Class Labels Have A Natural Order Sebastian Raschka 2022-02-21 | Sebastian's books: sebastianraschka.com/booksDeep learning offers state-of-the-art results for classifying images and text. Common deep learning architectures and training procedures focus on predicting unordered categories, such as recognizing a positive and negative sentiment from written text or indicating whether images contain cats, dogs, or airplanes. However, in many real-world problems, we deal with prediction problems where the target variable has an intrinsic ordering. For example, think of customer ratings (e.g., 1 to 5 stars) or medical diagnoses (e.g., disease severity labels such as none, mild, moderate, and severe). This talk will describe the core concepts behind working with ordered class labels, so-called ordinal data. We will cover hands-on PyTorch examples showing how to take existing deep learning architectures for classification and outfit them with loss functions better suited for ordinal data while only making minimal changes to the core architecture.Slides: sebastianraschka.com/pdf/slides/2022-02_rework-coral-lightning.pdfCode: raschka-research-group.github.io/coral-pytorch0:00 Introduction0:32 Many Real-World Predictions Problems Have Ordered Labels0:57 Ordered Labels? Tell Me More!3:59 Can't we just use regular classifiers for ordered labels?5:47 How? Let's (Re)Use What We Already know: An Extended Binary Classification Framework8:07 Problem: rank inconsistency10:53 Converting a Classifier into a CORN Model in 3 Lines of Code13:09 Acknowledgements
13.4.5 Sequential Feature Selection Code Examples (L13: Feature Selection) Sebastian Raschka 2022-01-06 | Sebastian's books: sebastianraschka.com/booksThis final video in the "Feature Selection" series shows you how to use Sequential Feature Selection in Python using both mlxtend and scikit-learn.Jupyter notebook: github.com/rasbt/stat451-machine-learning-fs21/blob/main/13-feature-selection/08_sequential-feature-selection.ipynbTimestamps: 00:00 Dataset setup and KNN baseline04:08 Selecting the best 5 features10:18 Inspecting the results13:40 Selecting the best subset of any size17:29 Exhaustive search21:12 Sequential feature selection in scikit-learn -------This video is part of my Introduction of Machine Learning course.The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3A handy overview page with links to the materials: sebastianraschka.com/blog/2021/ml-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
13.4.4 Sequential Feature Selection (L13: Feature Selection) Sebastian Raschka 2022-01-05 | Sebastian's books: sebastianraschka.com/booksThis video explains how sequential feature selection works. Sequential feature selection is a wrapper method for feature selection that uses the performance (e.g., accuracy) of a classifier to select good feature subsets in an iterative fashion. You can think of sequential feature selection method as an efficient approximation to an exhaustive feature subset search. Slides: sebastianraschka.com/pdf/lecture-notes/stat451fs21/13_feat-sele__slides.pdfSequential feature selection paper: Ferri, F. J., Pudil P., Hatef, M., Kittler, J. (1994). "Comparative study of techniques for large-scale feature selection." Pattern Recognition in Practice IV : 403-413. sciencedirect.com/science/article/pii/B9780444818928500407 -------This video is part of my Introduction of Machine Learning course.Next video: youtube.com/watch?v=KYypVSwqqHI&list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3&index=95The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3A handy overview page with links to the materials: sebastianraschka.com/blog/2021/ml-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
13.4.3 Feature Permutation Importance Code Examples (L13: Feature Selection) Sebastian Raschka 2021-12-31 | Sebastian's books: sebastianraschka.com/booksThis video shows code examples for computing permutation importance in mlxtend and scikit-learn. Permutation importance is a model-agnostic, versatile way for computing the importance of features based on a machine learning classifier or regression model. Code notebooks:Wine data example: github.com/rasbt/stat451-machine-learning-fs21/blob/main/13-feature-selection/05_permutation-importance.ipynbUsing a random feature as a control: github.com/rasbt/stat451-machine-learning-fs21/blob/main/13-feature-selection/06_random_feature_as_control.ipynbChecking correlated features: github.com/rasbt/stat451-machine-learning-fs21/blob/main/13-feature-selection/07_perm-imp-with-correlated-feats.ipynb Slides: sebastianraschka.com/pdf/lecture-notes/stat451fs21/13_feat-sele__slides.pdfRandom forest importance video: youtu.be/ycyCtxZ0a9w-------This video is part of my Introduction of Machine Learning course.Next video: youtu.be/0vCXcGJg5BoThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3A handy overview page with links to the materials: sebastianraschka.com/blog/2021/ml-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
13.4.2 Feature Permutation Importance (L13: Feature Selection) Sebastian Raschka 2021-12-29 | Sebastian's books: sebastianraschka.com/booksThis video introduces permutation importance, which is a model-agnostic, versatile way for computing the importance of features based on a machine learning classifier or regression model. Slides: sebastianraschka.com/pdf/lecture-notes/stat451fs21/13_feat-sele__slides.pdf Random forest importance video: youtu.be/ycyCtxZ0a9w-------This video is part of my Introduction of Machine Learning course.Next video: youtu.be/meTXOuFV-s8The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3A handy overview page with links to the materials: sebastianraschka.com/blog/2021/ml-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
13.4.1 Recursive Feature Elimination (L13: Feature Selection) Sebastian Raschka 2021-12-27 | Sebastian's books: sebastianraschka.com/booksIn this video, we start our discussion of wrapper methods for feature selection. In particular, we cover Recursive Feature Elimination (RFE) and see how we can use it in scikit-learn to select features based on linear model coefficients. Slides: sebastianraschka.com/pdf/lecture-notes/stat451fs21/13_feat-sele__slides.pdf Code: github.com/rasbt/stat451-machine-learning-fs21/blob/main/13-feature-selection/04_recursive-feature-elimination.ipynb Logistic regression lectures: L8.0 Logistic Regression – Lecture Overview (06:28)youtube.com/watch?v=10PTpRRpRk0 L8.1 Logistic Regression as a Single-Layer Neural Network (09:15)youtube.com/watch?v=ncZ5iSZekVQ L8.2 Logistic Regression Loss Function (12:57)youtube.com/watch?v=GxJe0DZvydM L8.3 Logistic Regression Loss Derivative and Training (19:57)youtube.com/watch?v=7rR1L7t2EnA L8.4 Logits and Cross Entropy (06:47)youtube.com/watch?v=icQaFxKa_J0 L8.5 Logistic Regression in PyTorch – Code Example (19:02) youtube.com/watch?v=6igMArA6k3A L8.6 Multinomial Logistic Regression / Softmax Regression (17:31) youtube.com/watch?v=L0FU8NFpx4E L8.7.1 OneHot Encoding and Multi-category Cross Entropy (15:34)youtube.com/watch?v=4n71-tZ94yk L8.7.2 OneHot Encoding and Multi-category Cross Entropy Code Example (15:04)youtube.com/watch?v=5bW0vn4ISqs L8.8 Softmax Regression Derivatives for Gradient Descent (19:38) youtube.com/watch?v=aeM-fmcdkXU L8.9 Softmax Regression Code Example Using PyTorch (25:39)youtube.com/watch?v=mM6apVBXGEA-------This video is part of my Introduction of Machine Learning course.Next video: youtu.be/VUvShOEFdQoThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3A handy overview page with links to the materials: sebastianraschka.com/blog/2021/ml-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
13.3.2 Decision Trees & Random Forest Feature Importance (L13: Feature Selection) Sebastian Raschka 2021-12-22 | Sebastian's books: sebastianraschka.com/booksThis video explains how decision trees training can be regarded as an embedded method for feature selection. Then, we will also look at random forest feature importance and go over two different ways it's computed: (a) impurity-based and (b) permutation-based. Slide link: sebastianraschka.com/pdf/lecture-notes/stat451fs21/13_feat-sele__slides.pdf Code link: github.com/rasbt/stat451-machine-learning-fs21/blob/main/13-feature-selection/03_random-forest.ipynb-------This video is part of my Introduction of Machine Learning course.Next video: youtu.be/SljoN0cO95QThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3A handy overview page with links to the materials: sebastianraschka.com/blog/2021/ml-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
13.3.1 L1-regularized Logistic Regression as Embedded Feature Selection (L13: Feature Selection) Sebastian Raschka 2021-12-14 | Sebastian's books: sebastianraschka.com/booksWithout going into the nitty-gritty details behind logistic regression, this lecture explains how/why we can consider an L1 penalty --- a modification of the loss function -- as an embedded feature selection method. Slides: sebastianraschka.com/pdf/lecture-notes/stat451fs21/13_feat-sele__slides.pdf Code: github.com/rasbt/stat451-machine-learning-fs21/blob/main/13-feature-selection/02_lasso-path.ipynb Links to the logistic regression videos I referenced: sebastianraschka.com/blog/2021/dl-course.html#l08-multinomial-logistic-regression--softmax-regression-------This video is part of my Introduction of Machine Learning course.Next video: youtu.be/ycyCtxZ0a9wThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3A handy overview page with links to the materials: sebastianraschka.com/blog/2021/ml-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
13.2 Filter Methods for Feature Selection Variance Threshold (L13: Feature Selection) Sebastian Raschka 2021-12-11 | Sebastian's books: sebastianraschka.com/booksSorry, I had some issues with the microphone (a too aggressive filter to remove background noise). Should be better in the next vids!Description: This video dives into "filter methods" for feature selection. In particular, we focus on using a variance threshold to select informative features.Code: github.com/rasbt/stat451-machine-learning-fs21/blob/main/13-feature-selection/01_variance-threshold.ipynbSlides: sebastianraschka.com/pdf/lecture-notes/stat451fs20/13_feat-sele__slides.pdf-------This video is part of my Introduction of Machine Learning course.Next video: youtu.be/_aGWjt7GKBEThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3A handy overview page with links to the materials: sebastianraschka.com/blog/2021/ml-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
13.1 The Different Categories of Feature Selection (L13: Feature Selection) Sebastian Raschka 2021-12-10 | Sebastian's books: sebastianraschka.com/booksIn this video, I am introducing the three main categories of feature selection: filter methods, embedded methods, and wrapper methods.Slides: sebastianraschka.com/pdf/lecture-notes/stat451fs20/13_feat-sele__slides.pdf-------This video is part of my Introduction of Machine Learning course.Next video: youtu.be/l40H2-XdrEcThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3A handy overview page with links to the materials: sebastianraschka.com/blog/2021/ml-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
13.0 Introduction to Feature Selection (L13: Feature Selection) Sebastian Raschka 2021-12-09 | Sebastian's books: sebastianraschka.com/booksThis video gives a brief intro of how we care about dimensionality reduction and introduces feature selection as a subcategory that we will cover in more detail in the upcoming videos.Slides: sebastianraschka.com/pdf/lecture-notes/stat451fs20/13_feat-sele__slides.pdf-------This video is part of my Introduction of Machine Learning course.Next video: youtu.be/hV_I4Xb9fRgThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KyGirGEvKlniaWeLOHhUF3A handy overview page with links to the materials: sebastianraschka.com/blog/2021/ml-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
Introduction to Generative Adversarial Networks (Tutorial Recording at ISSDL 2021) Sebastian Raschka 2021-09-17 | Sebastian's books: sebastianraschka.com/booksJuly 2021. Invited tutorial lecture at the International Summer School on Deep Learning, Gdansk. Slides: sebastianraschka.com/pdf/slides/2021-07_issdl-gdansk-intro-to-gans.pdfCode: github.com/rasbt/2021-issdl-gdansk===================This lecture introduces the main concepts behind Generative Adversarial Networks (GANs) and explains the main ideas behind the objective function for optimizing the generator and discriminator subnetworks. Hands-on examples include GANs for handwrittten digit and face generation, implemented in PyTorch. Lastly, this talks summarizes some of the main milestone GAN architectures that emerged in recent years.
Designing Generative Adversarial Networks for Privacy-enhanced Face Recognition (Conference rec.) Sebastian Raschka 2021-09-13 | Sebastian's books: sebastianraschka.com/booksJuly 2021. Invited keynote talk at the 14th International Conference on Human System Interaction (HSI 2021). Slides: sebastianraschka.com/pdf/slides/2021-07_HSI_talk.pdfAfter introducing the main concepts behind face recognition and soft-biometric attribute mining (i.e., the extraction of information such as age, gender, race, health information, and others), this talk discusses different methods for hiding soft-biometric information from facial recognition systems. After introducing the main methodologies, the talk focuses on the PrivacyNet architecture, which is a GAN-based approach to collective and selective facial privacy.
L19.5.2.2 GPT-v1: Generative Pre-Trained Transformer Sebastian Raschka 2021-05-14 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/_BFp4kjSB-IThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.5.2.4 GPT-v2: Language Models are Unsupervised Multitask Learners Sebastian Raschka 2021-05-14 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/wYdKn-X4MhYThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.5.2.7: Closing Words The Recent Growth of Language Transformers Sebastian Raschka 2021-05-14 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/emDmznRlsWwThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.5.2.6 BART: Combining Bidirectional and Auto-Regressive Transformers Sebastian Raschka 2021-05-14 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf0:00 Introduction0:33 BART. Combining Bidirectional and Auto-Regressive Transformers2:14 BART. BERT Encoder + GPT Decoder - Noise Transformations4:39 Noise Transformations in BART for Pre-Training on Unlabeled Data6:19 BART Performance Under Different Noise Transformations7:04 Fine-Tuning on Labeled Data8:21 BART Performance for Discriminative Tasks9:26 BART Performance for Generative Tasks-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/OyqIuxMmLRgThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.5.2.5 GPT-v3: Language Models are Few-Shot Learners Sebastian Raschka 2021-05-14 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/1JBMCG8rW18The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.6 DistilBert Movie Review Classifier in PyTorch Code Example Sebastian Raschka 2021-05-14 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdfCode: github.com/rasbt/stat453-deep-learning-ss21/tree/main/L19/distilbert-classifier-------This video is part of my Introduction of Deep Learning course.The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.5.2.3 BERT: Bidirectional Encoder Representations from Transformers Sebastian Raschka 2021-05-14 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf0:00 Introduction0:29 BERT (Bidirectional Encoder Representations from Transformers)1:44 BERT Inputs4:18 BERT Pre-Training Task #19:47 BERT Pre-Training & Downstream Tasks11:51 Transformer Training Approach12:16 BERT Pre-Training & Fine-Tuning Approach13:49 BERT vs GPT-v1 Performance14:59 BERT Pre-Training & Feature-based Training-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/BXv1m9Asl7IThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.5.2.1 Some Popular Transformer Models: BERT, GPT, and BART Overview Sebastian Raschka 2021-05-14 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/LOCzBgSV4tQThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.5.1 The Transformer Architecture Sebastian Raschka 2021-05-14 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/iFhYwEi03EwThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.4.3 Multi-Head Attention Sebastian Raschka 2021-05-04 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/tstbZXNCfLYThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.4.2 Self-Attention and Scaled Dot-Product Attention Sebastian Raschka 2021-05-04 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/A1eUVxscNq8The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.4.1 Using Attention Without the RNN A Basic Form of Self-Attention Sebastian Raschka 2021-05-04 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/0PjHri8tc1cThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.3 RNNs with an Attention Mechanism Sebastian Raschka 2021-05-04 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/i_pfHD4P_wgThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.2.1 Implementing a Character RNN in PyTorch (Concepts) Sebastian Raschka 2021-04-29 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/tL5puCeDr-oThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.2.2 Implementing a Character RNN in PyTorch Code Example Sebastian Raschka 2021-04-29 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdfCode: github.com/rasbt/stat453-deep-learning-ss21/tree/main/L19/character-rnn-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/mDZil99CtSUThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.1 Sequence Generation with Word and Character RNNs Sebastian Raschka 2021-04-29 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/PFcWQkGP4lUThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L19.0 RNNs & Transformers for Sequence-to-Sequence Modeling Lecture Overview Sebastian Raschka 2021-04-29 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/fSBw6TrePPgThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L18.6: A DCGAN for Generating Face Images in PyTorch Code Example Sebastian Raschka 2021-04-27 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L18_gan__slides.pdfCode: github.com/rasbt/stat453-deep-learning-ss21/tree/main/L18-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/DlWTTrHa8bIThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L18.5: Tips and Tricks to Make GANs Work Sebastian Raschka 2021-04-27 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L18_gan__slides.pdfCode: github.com/rasbt/stat453-deep-learning-ss21/tree/main/L18GAN Tips repo: github.com/soumith/ganhacks-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/5fs9PMzrVigThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L18.4: A GAN for Generating Handwritten Digits in PyTorch Code Example Sebastian Raschka 2021-04-22 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L18_gan__slides.pdfCode: github.com/rasbt/stat453-deep-learning-ss21/blob/main/L18This video discusses 04_01_gan-mnist.ipynb-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/_cUdjPdbldQThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L18.3: Modifying the GAN Loss Function for Practical Use Sebastian Raschka 2021-04-22 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L18_gan__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/cTlxZ1FO1mYThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L18.2: The GAN Objective Sebastian Raschka 2021-04-22 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L18_gan__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/ILpC3b-819QThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L18.1: The Main Idea Behind GANs Sebastian Raschka 2021-04-22 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L18_gan__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/m_H6viKCTEEThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L18.0: Introduction to Generative Adversarial Networks Lecture Overview Sebastian Raschka 2021-04-22 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L18_gan__slides.pdf-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/-Zi5SReze6UThe complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L17.7 VAE Latent Space Arithmetic in PyTorch Making People Smile (Code Example) Sebastian Raschka 2021-04-21 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L17_vae__slides.pdfL17 code: github.com/rasbt/stat453-deep-learning-ss21/tree/main/L17Discussing 5_VAE_celeba_latent-arithmetic.ipynb-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/OnoPaZaKoS8The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L17.6 A Variational Autoencoder for Face Images in PyTorch Code Example Sebastian Raschka 2021-04-21 | Sebastian's books: sebastianraschka.com/booksSlides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L17_vae__slides.pdfL17 code: github.com/rasbt/stat453-deep-learning-ss21/tree/main/L17Discussing 2_VAE_celeba-sigmoid_mse.ipynb, 3_VAE_nearest-neighbor-upsampling.ipynb & 4_VAE_celeba-inspect-latent.ipynb-------This video is part of my Introduction of Deep Learning course.Next video: youtu.be/EfFr87ARDF0The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html-------If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
