Reinforcement Learning Made Simple - Q-Values Edan Meyer 2026-09-29 | Reinforcement Learning Made Simple - Q-Values
2 Years of My Research Explained in 13 Minutes Edan Meyer 2024-07-13 | This is my research into representation learning and model learning in the reinforcement learning setting. Two years in the making, and I finally get to talk about my Master's research! The paper has been accepted to the Reinforcement Learning Conference (RLC) 2024. You can find the full paper at arxiv.org/abs/2312.01203.Outline0:00 - Intro0:34 - DreamerV31:23 - State representation3:27 - Research problem4:26 - World model experiments9:21 - Policy learning experiments13:22 - The paperSocial MediaYouTube - youtube.com/c/EdanMeyerX - https://x.com/ejmejm1Sources:Paper - arxiv.org/abs/2312.01203Thesis - era.library.ualberta.ca/items/d9bc72bd-cb8c-4ca9-a978-e97e8e16abf0
The AI/ML Masters Experience Edan Meyer 2024-05-11 | I did a 2 year Master's degree in computer science at the University of Alberta. My studies focused on research in machine learning (ML) and reinforcement learning (RL). In this video I talk about what you do for an MSc, coursework and research, and I also talk about how much it costs / how much you can get. I mainly focus on Master's degrees in Canada because that is what I did.Outline0:00 - Intro0:31 - Overview0:58 - My experience3:55 - Different types of Master'sSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:My Master's research paper: arxiv.org/abs/2312.01203My thesis: era.library.ualberta.ca/items/d9bc72bd-cb8c-4ca9-a978-e97e8e16abf0
This is What Limits Current LLMs Edan Meyer 2024-05-04 | Recent advances in large language models (LLMs) have centered around more data, larger models, and larger context lengths. The ability of LLMs to learn in-context (i.e. in-context learning) makes longer context lengths extremely valuable. However, there are some problems with relying on just in-context learning to learn during inference.Social MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1
I Talked with Rich Sutton Edan Meyer 2024-01-08 | After a year of anticipation, I finally got to talk with Rich Sutton for the channel! Rich wrote the book on reinforcement learning, has contributed vastly to the literature, and has a unique perspective on AI. We talk about topics like reinforcement learning, the OpenMind Research Institute (OMRI), Keen Technologies, how to do good research, the problem of scale, superintelligence, and so much more.Outline0:00 - Intro1:33 - Interview start2:04 - OpenMind Research Institute4:32 - History of AI7:13 - Is scaling easy?10:49 - The problem with backprop & representations21:22 - Rant on tunnel vision23:43 - New exciting things32:00 - Memory35:34 - Coming up with ideas43:47 - STOMP45:30 - Keen Technologies50:39 - The next stage of humanity & emotions1:06:25 - Extraterrestrial AI1:08:00 - A different approach to research1:21:30 - Rich's advice1:26:00 - Beef with RL1:27:07 - Bringing it all togetherAmii's website: amii.caRich's website: http://www.incompleteideas.netSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter/X - twitter.com/ejmejm1
GPT-4 Outperforms RL by Studying and Reasoning... 🤔 Edan Meyer 2023-08-14 | Let's take a look at "SPRING: GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning". The authors use GPT-4 to learn an agent that can play Crafter, getting it's information from a specification paper. They claim it's better than reinforcement learning, but is that really the case? Should we be using LLMs instead of RL?Outline0:00 - Intro1:02 - Crafter2:39 - LLM Learning7:40 - Similar Projects8:40 - Results13:50 - Ablation16:10 - Other LLMs17:18 - TakeawaysSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:Paper - arxiv.org/abs/2305.15486
Training RL From YouTube Videos Edan Meyer 2023-06-04 | Reinforcement learning is great, but environment interaction can be expensive. This paper proposes an RL algorithm based of successor features that takes advantage of passive data to learn about the world without acting itself.Outline0:00 - Intro1:41 - Offline-RL2:50 - Successor Features13:34 - Algorithm21:11 - Results27:50 - Criticisms & ThoughtsSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:Paper - arxiv.org/pdf/2304.04782.pdf
Sparks of AGI: What to Know Edan Meyer 2023-04-22 | Spark of AGI, a new paper from Microsoft, takes us through over 100 pages of GPT-4 experiments to argue that it shows early signs of artificial general intelligence. The results are impressive, but there are also still clear areas for improvement and indications of the limitations of LLMs like GPT-4.Outline0:00 - Intro1:01 - Experiment Speed Run 13:51 - ClearML4:54 - GPT-4 Flaws & Math8:31 - Tool Usage11:13 - Experiment Speed Run 213:12 - Limitations19:03 - AGIClearML - bit.ly/3GtCsj5 Social MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:Paper - arxiv.org/pdf/2303.12712.pdf
This Embodied LLM is... Edan Meyer 2023-04-03 | PaLM-E is a new LLM from Google that is both embodied in multimodal. Excitingly, it shows positive transfer across different robotics tasks. While the prospects are exciting, it is unclear what other conclusions can be drawn from the work.Outline0:00 - Intro1:34 - How It Works6:47 - Robotics Tasks11:25 - Results26:50 - TakeawaysSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:PaLM-E paper - arxiv.org/abs/2303.03378
Automating Research With GPT API [Livestream] Edan Meyer 2023-03-17 | The title says it all, I'm figuring this out as I go! Currently I'm using GPT-3.5 turbo to do all this testing.The current plan is to create a system that:1. Can generate research ideas in a target area2. Use the idea to make a list of proof of concept experiments3. Write the code for the experiments4. Debug
GPT-4: What, Why, How? Edan Meyer 2023-03-16 | OpenAI's GPT-4 makes significant improvements over GPT-3 and 3.5, but does it live up to the hype? It will soon support images, and is already multi-lingual, supporting context lengths up to ~32,000 tokens. It works great but we know next to nothing about it. I also talk about how to get access to GPT-4, along with a demo using ChatGPT.GTC Signup: nvda.ws/408jS7wRaffle Signup: https://forms.gle/NyxKRw2tF6Un9CaT6Outline0:00 - Intro0:50 - New Features1:42 - Evaluations5:32 - Multilingual7:07 - Images8:19 - Hallucination9:19 - Training & RL11:40 - Malicious Use14:40 - How it Works18:20 - Getting Access19:56 - Demo24:06 - Script Generation26:51 - Programming Demo32:58 - Takeaways34:29 - GTC & Raffle35:57 - Wrapping it UpSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:GPT-4: openai.com/research/gpt-4GTC sessions I'm attending:- bit.ly/nvidia-science- bit.ly/nvidia-real-world-rl
ChatGPT Is a Dead End Edan Meyer 2023-03-08 | ChatGPT can only go so far, let's talk about the limitations of LLMs and topics like grounding and exploration that are important research directions for future work.GTC Signup: nvda.ws/408jS7wRaffle Signup: https://forms.gle/NyxKRw2tF6Un9CaT6Sessions I'm attending:- bit.ly/nvidia-science- bit.ly/nvidia-real-world-rlOutline0:00 - Intro0:48 - GPU Giveaway1:06 - Human-Level AI2:38 - My Thoughts on ChatGPT3:52 - ChatGPT a Dead End5:23 - Path Forward - AI Feedback8:17 - Path Forward - Grounding9:11 - Beyond LLMs10:23 - GTC & Giveaway11:37 - OutroSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:ChatGPT: openai.com/blog/chatgptAnthropic AI Paper: anthropic.com/constitutional.pdf
RL Foundation Models Are Coming! Edan Meyer 2023-02-26 | AdA is a new algorithm out from DeepMind that combines interesting ideas like curriculum learning, meta reinforcement learning (via RL^2), model-based reinforcement learning, attention, and memory models to develop a prototype for a reinforcement learning foundation model. The results look promising, and the future of this area looks bright!Outline0:00 - Intro1:07 - Example Video2:40 - ClearML3:48 - How It Works Overview4:20 - Meta-Learning & RL8:20 - Attention & Memory9:55 - Distillation12:01 - Auto-Curriculum Learning15:18 - Results27:14 - Takeaways & Future WorkClearML - bit.ly/3GtCsj5 Social Media:YouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:AdA Paper: arxiv.org/abs/2301.07608Museli Paper: arxiv.org/abs/2104.06159Primacy Bias Paper: arxiv.org/abs/2205.07802
This Algorithm Could Make a GPT-4 Toaster Possible Edan Meyer 2023-01-27 | The Forward-Forward algorithm from Geoffry Hinton is a backpropagation alternative inspired by learning in the cortex. It tackles several issues with backprop that would allow it to be run much more efficiently. Hopefully research like this continues to pave the way toward full-hardware integrated AI chips in the future.Outline0:00 - Intro1:13 - ClearML2:17 - Motivation5:40 - Forward-Forward Explained13:54 - MNIST Example18:54 - Top-Down Interactions26:00 - More Examples / Results27:41 - Sleep & Phased Learning29:36 - Related Ideas30:38 - Learning Fast & Slow32:35 - Mortal ComputationClearML - bit.ly/3GtCsj5 Social Media:YouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:Paper - https://www.cs.toronto.edu/~hinton/FFA13.pdf
Model Based RL Finally Works! Edan Meyer 2023-01-13 | Dreamer v3 is a model based reinforcement learning (MBRL) algorithm that performs well over a wide variety of environments, from 2D to 3D, simple to complex. It can get diamonds in a simplified version of Minecraft and work well out of the box without tuning.Outline0:00 - Intro1:32 - World Model13:22 - Actor Critic18:32 - Results22:52 - Minecraft Results25:16 - Thoughts & ConclusionSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:Dreamer v3 - arxiv.org/abs/2301.04104v1
Using ChatGPT to Write My PhD Apps Edan Meyer 2022-12-12 | Computer science / ML PhD apps are boring, so let's use ChatGPT to write one for us! It works kinda well I guess. Not perfect, but pretty impressive I'd say! I got some better results with ChatGPT in the past, so it looks like the quality of the outputs can have a bit of a range.ClearML: bit.ly/3GtCsj5Outline0:00 - Intro1:27 - ChatGPT Example2:07 - ClearML3:13 - Writing my SoP8:46 - Giving it my resume15:25 - What university can I get into?Social MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:ChatGPT Blog - openai.com/blog/chatgpt
The Best of NeurIPS 2022 Edan Meyer 2022-12-10 | Let's talk about all the NeurIPS 2022 outstanding paper awards. There were a good mix of papers, but especially a lot of work in diffusion models and optimization with both empirical work and theory. You can find links to all of these papers at: blog.neurips.cc/2022/11/21/announcing-the-neurips-2022-awardsClearML: bit.ly/3GtCsj5Outline0:00 - Intro1:03 - ClearML2:08 - OOD Detection4:21 - Imagen5:19 - Diffusion Model Design7:07 - ProcTHOR8:58 - Instilling Human Biases10:46 - Neural Document Retrieval12:35 - High Dimensional SGD13:14 - Gradient Descent: The Ultimate Optimizer15:00 - Riemannian Diffusion Models16:09 - Gradient Estimation for Discrete Values17:27 - Chinchilla19:03 - Beyond Neural Scaling Laws via Pruning20:23 - On-Demand Multi-Distribution Learning22:18 - LAION-5B23:15 - MineDOJOSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:Outstanding Papers - blog.neurips.cc/2022/11/21/announcing-the-neurips-2022-awards
Agent Learns to do Reinforcement Learning Edan Meyer 2022-11-19 | "In-context Reinforcement Learning with Algorithm Distillation" is a new paper from DeepMind about learning how to learn how to do Reinforcement Learning (RL) using behavior cloning over a learning history with a Transformer. The idea is simple, but I think the implications could be big for the future.Outline0:00 - Intro0:30 - Why I like this paper2:08 - ClearML3:17 - Algorithm Overview7:50 - Bandits9:06 - Robustness Results15:08 - Speedup Results22:00 - Other Results23:00 - ConclusionClearML - bit.ly/3GtCsj5 Social MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1RL AD Paper - arxiv.org/abs/2210.14215
Whats New In Machine Learning? Edan Meyer 2022-10-03 | I decided to take a look through the open review papers submitted to ICLR 2023. ICLR is the International Conference on Learning Representations, and is one of the most prestigious conferences in machine learning (ML). Let's see what new papers people are submitting this year :)Social MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:Reddit post: reddit.com/r/MachineLearning/comments/xtabkh/p_interactive_map_of_iclr_2023_submissionsInteractive map: atlas.nomic.ai/map/01ff9510-d771-47db-b6a0-2108c9fe8ad1/3ceb455b-7971-4495-bb81-8291dc2d8f37My New To Read List:openreview.net/pdf/cd54eb1f885125e291f889b1d9351e3de4eee92e.pdfopenreview.net/pdf/b8a3fe6e051c07d56858ee86ab6ec1aa0ce02f04.pdfopenreview.net/pdf/c440e635f8faa6ec68634bbf9d60962a8bb11e48.pdfopenreview.net/pdf/fb51a9a71d3de1574503bd9794077610ff6e3515.pdfopenreview.net/pdf/70887668eb5abc0f227a5efe6a95e91c29dbdcb0.pdfopenreview.net/pdf/a93e0bc5de0936e182608ae7f4b1015ce58fcf2d.pdfopenreview.net/pdf/a9ea37dc3d4fbb37dd18c3942b3d4c2b4399a0d9.pdf
12 Steps to AGI Edan Meyer 2022-09-16 | The Alberta Plan is a new paper by the team from DeepMind Alberta that details their plans and core focuses over the next 5-10 years. While it puts a focus on Reinforcement Learning (RL) and touches on many areas that are already being explored, it takes a different perspective and focuses on a slightly different problem, which shifts how some of the mentioned problems may be tackled. The plan primarily focuses on continual learning in a vastly complex world where the agent needs to learn, and learn to learn (meta-learning) to achieve it's goals.Outline0:00 - Intro1:58 - Core problems5:33 - Alberta Plan tenets11:33 - The Common Model13:55 - 12 step overview15:57 - Steps 1-630:14 - Steps 7-1239:38 - Intelligence amplification / Singularity41:14 - ThoughtsSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:Alberta Plan Paper - arxiv.org/abs/2208.11173CBP Paper - arxiv.org/abs/2108.06325My video on CBP - youtu.be/zEMOX3Di2TcSTOMP Paper - arxiv.org/pdf/2202.03466.pdf
Stable Diffusion - What, Why, How? Edan Meyer 2022-09-05 | Stable Diffusion is a text-based image generation machine learning model released by Stability.AI. It's default ability generated image from text, but the model is open source which means that it can also do much more. In this video I explain how Stable Diffusion works at a high level, briefly talk about how it is different from other Diffusion-based models, compare it to DALL-E 2, and mess around with the code.Outline0:00 - Intro1:58 - Stable Diffusion Explained4:32 - Why The Buzz?6:46 - Starting The Code12:45 - DALL-E 2 Comparison20:15 - Custom Pipeline32:01 - Making Diffusion Videos37:30 - Generating Similar Images40:23 - Image-to-Image53:42 - Wrapping it UpSocial MediaYouTube - youtube.com/c/EdanMeyerTwitter - twitter.com/ejmejm1Sources:Colab Notebook - colab.research.google.com/drive/1_kbRZPTjnFgViPrmGcUsaszEdYa8XTpq?usp=sharingBlog post - stability.ai/blog/stable-diffusion-public-releaseSource Code - github.com/CompVis/stable-diffusionHugging Face Models - huggingface.co/spaces/stabilityai/stable-diffusionPaper - arxiv.org/pdf/2112.10752.pdf
Googles New Model Learns College-Level Math Edan Meyer 2022-07-29 | Minerva is Google's new large language model (LLM) that can solve math questions. It is built on top of their PaLM model, with the biggest variant having 540 billion parameters. The paper uses these previous model, but with new math-specific data and analysis to test the current limits of LLMs when it comes to quantitative reasoning tasks. The results of the paper are impressive, but in several areas, still raises the question of what the next significant development in Machine Learning will be to push forward generalization.----- Outline -----It’s 5am and I need to sleep, will add tomorrow----- Socials -----YouTube: youtube.com/c/EdanMeyerTwitter: twitter.com/ejmejm1----- References -----Paper (Solving Quantitative Reasoning Problems with Language Models): arxiv.org/abs/2206.14858Blog Post: ai.googleblog.com/2022/06/minerva-solving-quantitative-reasoning.html
This AI Learns from YouTube! Edan Meyer 2022-06-26 | OpenAI's new paper, "Video PreTraining(VPT): Learning to Act by Watching Unlabeled Online Videos" trains an agent to play Minecraft using a mixture of imitation learning (in the form of behavior cloning) and reinforcement learning (RL). The results are very impressive, training a 500 million parameter model that can obtain diamonds and occasionally craft diamond tools. I cover the paper in this video, talking about their approach of using and fine-tuning a foundational model with semi-supervised learning.Outline0:00 - Intro & Abstract2:10 - Paper Goal3:55 - Methods5:30 - Initial Results8:06 - Fine-tuning w/ BC9:15 - Reinforcement Learning12:02 - Sample Video13:14 - Method Efficiency15:30 - Text Conditioning16:33 - Summary18:00 - Criticisms & ThoughtsSourcesPaper: openai.com/blog/vptBlog post: arxiv.org/abs/2206.11795Source code & models: github.com/openai/Video-Pre-TrainingAgent videos:youtube.com/playlist?list=PLNAOIb_agjf3U3rSvG_BCWqJ869NdBhcPyoutube.com/playlist?list=PLNAOIb_agjf2yDSs4AqcoyPv4z_eWUiKmyoutube.com/playlist?list=PLNAOIb_agjf3e_UKweM5pQUSfTw8r-Wfc
AGI is NOT coming soon Edan Meyer 2022-06-14 | Artificial General Intelligence (AGI) is popularly defined as AI that can do anything a human can do. More specifically in this video I talk about my thoughts on when we will reach human level AI. Given recent successes in Machine Learning, and especially with scaling Deep Learning, there has been a lot of talk about AGI recently. I have heard a lot of wild opinions, so with the video I try to take a research-centric approach, talking about results from recent papers and models like Gopher, Chinchilla, Gato, "Scaling Laws for Neural Language Models," and more. I talk about this in the context of other recent works like GPT-3 and Dall-E 2.Outline:0:00 - Intro & AGI Buzz1:41 - What is AGI?2:13 - Pro AGI Argument4:09 - Scaling Isn't Enough6:51 - "Level 2 Intelligence"9:14 - Counterarguments12:12 - Supervised Learning Isn't Enough13:40 - An Optimistic PerspectiveSources (other than the ones cited in the video)Refrenced AI/AGI articles:venturebeat.com/2021/06/26/deepmind-agi-paper-adds-urgency-to-ethical-aiventurebeat.com/2022/06/04/is-deepminds-gato-the-worlds-first-agithenextweb.com/news/deepminds-astounding-new-gato-ai-makes-fear-humans-will-never-achieve-agitechnologyreview.com/2022/05/23/1052627/deepmind-gato-ai-model-hypeImage segmentation example: towardsdatascience.com/image-segmentation-with-six-lines-0f-code-acb870a462e8Trackmania footage: youtube.com/watch?v=dLKAlWHn2vo
Is Gato Really the Future of AI? Edan Meyer 2022-05-15 | DeepMind has released "A Generalist Agent", a paper that introduces their new multi-modal model Gato. But is Gato truly a generalist agent? It is a transformer based model with the goal of generalizing over new tasks. It is trained fully autoregressively with supervised learning (no reinforcement learning) on a total of 603 different tasks. The tasks include robotics, Atari, DM Lab, Procgen, and a lot more. It also includes text and image tasks. This video is a paper review / explanation where I also give my thoughts on the paper.Outline:0:00 - Intro & Gato Overview1:20 - Paper Motivation2:56 - Model Explanation6:13 - Data & Tasks7:45 - Initial Results9:38 - Zero Shot Transfer12:58 - Generalization Ablation17:05 - Fine-tuning19:38 - My ThoughtsGato blog post: deepmind.com/publications/a-generalist-agentGato paper: arxiv.org/pdf/2205.06175.pdf
DALL-E 2 is… meh Edan Meyer 2022-04-26 | DALL-E 2 is a new, multi-billion parameter computer vision model for image generation by OpenAI. It is a successor to the original DALL E, but this time uses a diffusion model. Diffusion models have been gaining traction and are now at the top for image generation. It can also perform tasks like inpainting, upscaling, and interpolation between images. In this video we take a look at the research paper and see how DALL-E has progresses from the original paper to this new DALL-E 2.Outline:0:00 - Overview0:43 - DALL-E 1 Primer2:16 - DALL-E 2 Overview4:17 - Example Images & Decoder Discussion5:45 - DALL-E 2 Training8:53 - Diffusion Models9:48 - Experiments13:27 - My Thoughts15:01 - Looking At Generated ImagesDALL-E 2 blog post: openai.com/dall-e-2DALL-E 2 paper: arxiv.org/abs/2204.06125Great video on diffusion models: youtube.com/watch?v=fbLgFrlTnGU
Chinchilla Explained: Compute-Optimal Massive Language Models Edan Meyer 2022-04-09 | Chinchilla is a massive language released by DeepMind as part of a recent paper that focuses on scaling large language models in a compute-optimal manner. It outperforms recent models like GPT-3, Gopher, and Megatron-Turing NLG that use hundreds of billions of parameters with only 70 billion parameters. They achieve this by training 400 large models to find the optimal ratio of parameters and amount of training data to train a model given a computation budget.Outline:0:00 - Overview1:51 - Paper Intro6:15 - Methods18:14 - Scaling Implications23:43 - Chinchilla Overview25:48 - Chinchilla Performance29:49 - Summary30:07 - Thoughts & CritiquesPaper (Training Compute-Optimal Large Language Models): arxiv.org/abs/2203.15556
Learning Forever, Backprop Is Insufficient Edan Meyer 2022-04-04 | #ai #mlContinual Learning, or Life-long learning, is becoming more popular in Machine Learning (ML). This new research paper talks about plasticity decay, and how normal backpropagation is insufficient for continual learning. The inherent non-stationary property in many problems, especially in Reinforcement Learning (RL), makes it difficult to learn. Continual Backpropagation (CBP) is proposed as a solution to this.Outline:0:00 - Overview2:00 - Paper Intro2:53 - Problems & Environments8:11 - Plasticity Decay Experiments11:45 - Continual Backprop Explained15:54 - Continual Backprop Experiments22:00 - Extra Interesting Experiments25:34 - SummaryPaper source: arxiv.org/abs/2108.06325Abstract:The Backprop algorithm for learning in neural networks utilizes two mechanisms: first, stochastic gradient descent and second, initialization with small random weights, where the latter is essential to the effectiveness of the former. We show that in continual learning setups, Backprop performs well initially, but over time its performance degrades. Stochastic gradient descent alone is insufficient to learn continually; the initial randomness enables only initial learning but not continual learning. To the best of our knowledge, ours is the first result showing this degradation in Backprop's ability to learn. To address this issue, we propose an algorithm that continually injects random features alongside gradient descent using a new generate-and-test process. We call this the Continual Backprop algorithm. We show that, unlike Backprop, Continual Backprop is able to continually adapt in both supervised and reinforcement learning problems. We expect that as continual learning becomes more common in future applications, a method like Continual Backprop will be essential where the advantages of random initialization are present throughout learning.
Is AI Research On the Wrong Path? Edan Meyer 2022-03-21 | Lots of research in Artificial Intelligence (AI) and Machine Learning (ML) in the past few years has focused on supervised learning, and more recently there has been a trend of constructing massive models and focusing on scaling (e.g. GPT-3, Gopher, AlphaCode). These aren't the only two areas of work, but is it perhaps a mistake to focus on these approaches in ML research? Is something like Reinforcement Learning (RL) more promising? I'm not sure, but these are some opinions I've heard from others.Physics simulation paper: arxiv.org/abs/2002.09405Face image generation paper: arxiv.org/pdf/1903.06048.pdfSource of other AI-gen images used (DALL-E): arxiv.org/abs/2102.12092
Doing ML Research as a Graduate Student Edan Meyer 2022-03-07 | I'm a Masters Computer Science student in starting research in AI and Machine Learning. In ML, I am particularly interested in Reinforcement Learning (RL). I'm hoping to share a little more with you all than just the normal ML paper reviews and code that I go over. Let me know what you want to see from this series, whether it be research ideas, code, or something else.Chapters:0:00 The ML journey2:48 My notebook11:11 Code16:43 Feedback
Learning Fast with No Goals - VISR Explained Edan Meyer 2022-02-21 | VISR is a paper about rapid generalization to new tasks in Reinforcement Learning (RL). The full paper was released by DeepMind in 2020 and is called Fast Task Inference with Variational Intrinsic Successor Features. It uses successor features and goal-conditioned policies to rapidly adapt to new tasks after learning within the no-reward regime of RL.Link to paper: openreview.net/forum?id=BJeAHkrYDS
AlphaCode Explained: AI Code Generation Edan Meyer 2022-02-04 | AlphaCode is DeepMind's new massive language model for generating code. It is similar to OpenAI Codex, except for in the paper they provide a bit more analysis. The field of NLP within AI and ML has exploded get a lot more papers all the time. Hopefully this video can help you understand how AlphaCode works and what some of the key takeaways are.AlphaCode blog post: deepmind.com/blog/article/Competitive-programming-with-AlphaCodeAlphaCode paper: storage.googleapis.com/deepmind-media/AlphaCode/competition_level_code_generation_with_alphacode.pdf
Is Machine Learning Research Oversaturated? Edan Meyer 2022-01-15 | Machine Learning (ML) research has grown exponentially popular over the years. With the success of deep learning, many people have aimed to take up research in machine learning, but find the space too overwhelming. Is AI and machine learning research oversaturated? What about areas like NLP and computer vision? I don't think so.Reddit post: reddit.com/r/MachineLearning/comments/s2g1rb/d_is_ai_research_reaching_saturationPhotorealistic image generation: github.com/tkarras/progressive_growing_of_gansArt generation: github.com/rbbrdckybk/ai-art-generatorPapers per day reference: data-mining.philippe-fournier-viger.com/too-many-machine-learning-papers
AI Doesn’t Move Like Humans - Here’s Why Edan Meyer 2022-01-04 | Options in Reinforcement Learning are a type of temporal abstraction of actions. The RL options framework takes into account the hierarchical nature of actions we see in real life. They allow AI / RL agents to better explore and achieve better results. This has been demonstrated in RL for robotics and for games. This is a very interesting area of Machine Learning that I highly recommend looking more into!Original options paper: https://people.cs.umass.edu/~barto/courses/cs687/Sutton-Precup-Singh-AIJ99.pdfExploration visualization reference: openreview.net/pdf?id=SkeIyaVtwB
Generating Photorealistic Images With OpenAI GLIDE Edan Meyer 2021-12-27 | GLIDE is OpenAI's new Computer Vision (CV) model that performs better than DALLE. I show you how to generate images with GLIDE in this video. The images it generates are very high quality, some are nearly photorealistic. You can also make AI generated art with the model just by changing the prompt! The model is 3 billion parameters, though the public model is only 300 million parameters.Paper: arxiv.org/abs/2112.10741GitHub: github.com/openai/glide-text2imColab Notebook: colab.research.google.com/github/woctezuma/glide-text2im-colab/blob/main/text2im.ipynb
Gopher Explained: 280 BILLION Parameter Model Beats GPT-3 Edan Meyer 2021-12-12 | Gopher is DeepMind's new large language model. With 280 billion parameters, it's larger than GPT-3. It gets state-of-the-art (SOTA) results in around 100 tasks. The best part of the Gopher paper is the wide and deep analysis on what scales with model size, performance on many tasks like language modeling, Q&A, logic tasks, and also a look into the ethics of large language models. I think the Gopher paper is a great example of what scaling research in Machine Learning should be about.Gopher Blog Post: deepmind.com/blog/article/language-modelling-at-scaleGopher Paper: storage.googleapis.com/deepmind-media/research/language-research/Training%20Gopher.pdfResource on Transformers: lilianweng.github.io/lil-log/2018/06/24/attention-attention.html#full-architecture
Reinforcement Learning in RTS Games Edan Meyer 2021-11-29 | This video we cover a research paper about Reinforcement Learning in RTS games and how it works! Recently works like OpenAI Five and Alpha Star by DeepMind have been incredible, but what makes these models work is still a mystery in many ways. The ablation study performed by this paper was very insightful in providing more light on what is important in these ML models.Full paper title: Gym-μRTS: Toward Affordable Deep Reinforcement Learning Research in Real-Time Strategy GamesWandb report: wandb.ai/vwxyzjn/gym-microrts-paper/reports/Gym-RTS-Toward-Affordable-Deep-Reinforcement-Learning-Research-in-Real-Time-Strategy-Games--Vmlldzo2MDIzMTgArxiv paper: arxiv.org/pdf/2105.13807.pdf
The Dos and Donts of OpenAI Codex Edan Meyer 2021-10-18 | OpenAI Codex is OpenAI's latest GPT-based language based model for GitHub Copilot that generates code. How to use OpenAI Codex effectively is a question that is up in the air, but I hope to provide some tips to help. Using GitHub Copilot for code completion is great, but still has many pitfalls when it comes to complex code generation.Previous video on OpenAI Codex: youtu.be/V1cpfbCVytgLink to GitHub Copilot: copilot.github.comLink to API post: openai.com/blog/openai-codex
Can OpenAI Codex Compete in an ML Competition? Edan Meyer 2021-10-11 | OpenAI Codex is OpenAI's latest GPT-based language based model for GitHub Copilot that generates code. I test if Codex can be used in competitions like the Kaggle competition with Google Brain ventilator data. While it is able to generate a working LSTM approach, the rank isn't so good. OpenAI Codex is impressive but it still has a ways to go! Check out my channel for more OpenAI Codex content if you find this interesting.Previous video on OpenAI Codex: youtu.be/Pkp1MRFGUVoLink to API post: openai.com/blog/openai-codex
3 Paths To AGI Edan Meyer 2021-10-03 | Artificial General Intelligence (AGI) is the holy grail of Artificial Intelligence (AI) and machine learning (ML), but how will we reach it? I outline 3 distinct possible methods for reaching AGI by using ML. The methods range from using robots, to reinforcement learning, to self-supervised learning. Let me know how you think we may reach AGI in the comments!
Can OpenAI Codex Debug Its Own Code? Edan Meyer 2021-09-29 | OpenAI Codex is OpenAI's latest GPT-based language based model for GitHub Copilot that generates code. Here we test if Codex can debug code, and even correct it's own errors. We look at both fixing error messages and identifying silent errors. The results were far from perfect but quite intriguing! Check out my channel for more OpenAI Codex content if you find this interesting.Previous video on OpenAI Codex: youtu.be/7QWVJ5rWy2sLink to API post: openai.com/blog/openai-codex
A New Theory For How The Brain Learns Edan Meyer 2021-09-22 | How the brain learns has long been a mystery that we have slowly uncovering bit by bit. Plasticity clearly plays a large rule in the process, but it is unclear exactly how it is used. This paper talk about how burst of spike / firings in neurons can send feedback signals to perform multiplexing and solve to credit assignment problem. It is quite an interesting theory!Burst-Dependent Synaptic Plasticity Can Coordinate Learning in Hierarchical Circuits Paper: nature.com/articles/s41593-021-00857-x
Can OpenAI Codex Recreate Itself? Edan Meyer 2021-09-18 | OpenAI Codex is OpenAI's latest GPT-based language based model that generates code. In this video we test if Codex can remake a model like itself, a code generation model. Given that this is similar to the condition to start the singularity, I thought it would be interesting to test how far it could get.Previous video on OpenAI Codex: youtu.be/FC962DmVfSUNext video on OpenAI Codex: youtu.be/Pkp1MRFGUVoLink to API post: openai.com/blog/openai-codex
Can OpenAI Codex Create AI? Edan Meyer 2021-09-12 | OpenAI Codex is the follow-up model to Github Copilot. OpenAI Codex is a GPT based model that generates code. In this video we test if it an write AI / ML code in Python. As it turns out, it works fairly well even for machine learning code!Next video on OpenAI Codex: youtu.be/FC962DmVfSULink to API post: openai.com/blog/openai-codex
AI Assistants Are Actually Stupid Edan Meyer 2021-08-21 | Learn how virtual assistant like Alexa and Google Assistant work. AI assistants have gained a lot of popularity, so I thought I'd talk about how they work. Machine learning virtual assistants are becoming the norm, but there are still many issues with current virtual assistants.Note that I changed/left out some details of how these assistants work to make sure I don't violate NDA. The general concepts are accurate though.Intent Detection SOTA: paperswithcode.com/task/intent-detectionSlot Filling SOTA: paperswithcode.com/task/slot-filling
Computers Have Already Surpassed The Human Brain Edan Meyer 2021-07-24 | Computers and AI hardware may already be faster than the human brain. Though not easy to compare the two, there are actually many ways we can attempt to calculate whether the brain or AI performs more computations. Many people scoff at the human brain vs computer comparison, but there is some good work behind it and it is quite interesting!Source: openphilanthropy.org/brain-computation-reportCheck out the above more a much deeper dive into the details of the brain vs. computer argument.
AI That Codes vs. X-Google Programmer Edan Meyer 2021-07-17 | GitHub Copilot is an AI that can code, released in collaboration between GitHub and OpenAI. This video hopes to answer the question of whether an AI programmer or a Google programmer is better. Even though it can't get everything, Copilot is impressive. Is it a step towards the AI singularity? Or is it a step towards self-programming AI (AI that codes itself). It's impressive, but certainly not taking any jobs away from any programmers yet.GitHub Copilot: copilot.github.comTimestamps:0:00 Challenge Explanation0:54 Problem #13:44 Problem #29:08 Problem #312:33 Problem #416:53 Demonstrations & Afterthoughts
Self-Supervised RL - Learning Without Data [Zero to Paper] Edan Meyer 2021-07-07 | Inverse Reinforcement Learning with Natural Language Goals (LangGoal IRL) offers a way to do sample-efficient IRL and a way to generalize using self-supervised learning. The paper is novel and is a step forward for general AI and ML algorithms. Though it has its cons, I think it is one of the better papers out there that cover RL, IRL, NLP, and generalization.Zero to Paper playlist: youtube.com/playlist?list=PL_49VD9KwQ_ONxENRk11jFEI3_pqAwaugInverse Reinforcement Learning video: youtube.com/watch?v=qo355ALvLRIPaper covered: arxiv.org/pdf/2008.06924.pdf
Learning Language Through Games [Zero to Paper] Edan Meyer 2021-06-23 | Let's talk about natural language in Reinforcement Learning. It's also a form of language grounding because models are trained to learn connections between language and a non-text environments. I think language-conditioned RL is the way forward for making more efficient, general AI training. Let me know what you think of this combination between NLP and RL in the comments!Zero to Paper playlist: youtube.com/playlist?list=PL_49VD9KwQ_ONxENRk11jFEI3_pqAwaugInverse Reinforcement Learning video: youtube.com/watch?v=qo355ALvLRIPapers covered:arxiv.org/pdf/2005.09382.pdfarxiv.org/pdf/1902.07742.pdfarxiv.org/pdf/1806.01946.pdf
TRILLION Parameter Models Are Here Edan Meyer 2021-06-16 | Training a large model with Machine Learning used to be strictly limited by GPU memory, but now with Microsoft's new paper, ZeRO-Infinity, training much larger ML models is possible. ZeRO-Infinity allows training models with trillions of parameters with a modest amount of GPUs and finetuning many billion parameter models with just one GPU. This is a great advancement for anyone who wants to work with big models like GPT-2 and the like! Use ML train large model.ZeRO-Infinity blog post: microsoft.com/en-us/research/blog/zero-infinity-and-deepspeed-unlocking-unprecedented-model-scale-for-deep-learning-trainingZeRO-Infinity paper: arxiv.org/abs/2104.07857DeepSpeed: deepspeed.ai
Bad ML Results? Try This Edan Meyer 2021-06-09 | After getting bad ML results, how to improve ML results is a common question. To get better ML results there are a number of points you can focus on. Improving your model is the obvious one, but there are also many other way to improve ML results (e.g. accuracy, recall, precision, f1 score). Methods include getting more data to assembling. There is no easy answer, but this video will hopefully give you somewhat of an idea of how to get starting with optimizing your Machine Learning approach.
