A Gentle Introduction to Offline Reinforcement Learning RAIL 2026-09-28 | A Gentle Introduction to Offline Reinforcement Learning
Dexterous Robotic Foundation Models RAIL 2025-10-19 | Presentation by Sergey Levine on dexterous robotic foundation models.
HIL-SERL: Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning RAIL 2024-12-18 | Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement LearningJianlan Luo, Charles Xu, Jeffrey Wu, Sergey LevineProject page: hil-serl.github.ioCode: github.com/rail-berkeley/hil-serl
Octo: An Open-Source Generalist Robot Policy RAIL 2024-06-26 | A 5-minute presentation outlining the Octo robotic policy.You can get started with Octo on your own robot, using the Github repo at github.com/octo-models/octo or the main project website at octo-models.github.io
SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning RAIL 2024-01-30 | SERL: A Software Suite for Sample-Efficient Robotic Reinforcement LearningJianlan Luo*, Zheyuan Hu*, Charles Xu, You Liang Tan, Jacob Berg, Archit Sharma, Stefan Schaal, Chelsea Finn, Abhishek Gupta, Sergey LevineProject page: serl-robot.github.ioCode: github.com/rail-berkeley/serl
FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learning RAIL 2024-01-17 | FMB: a Functional Manipulation Benchmark for Generalizable Robotic LearningJianlan Luo*, Charles Xu*, Fangchen Liu, Liam Tan, Zipeng Lin, Jeffrey Wu, Pieter Abbeel, Sergey LevineProject website: functional-manipulation-benchmark.github.io
Making Real-World Reinforcement Learning Practical RAIL 2024-01-03 | Lecture by Sergey Levine about progress on real-world deep RL. Covers these papers:A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning: sites.google.com/berkeley.edu/walk-in-the-parkGrow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion: sites.google.com/berkeley.edu/aprlReset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention: sites.google.com/view/mtrfREBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulation: sites.google.com/view/reboot-dexterousFastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing: sites.google.com/view/fastrlapQ-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions: qtransformer.github.ioDeep RL at Scale: Sorting Waste in Office Buildings with a Fleet of Mobile Manipulators: rl-at-scale.github.io
RLIF: Interactive Imitation Learning as Reinforcement Learning RAIL 2023-12-17 | We present a reinforcement learning algorithm that runs under DAgger-like assumptions, which can improve upon suboptimal experts without knowing ground-truth rewards.RLIF: Interactive Imitation Learning as Reinforcement LearningJianlan Luo*, Perry Dong*, Yuexiang Zhai, Yi Ma, Sergey LevineProject page: rlif-page.github.io
Large-Scale Data-Driven Robotic Learning RAIL 2023-11-11 | Presentation by Sergey Levine prepared for the "Towards Generalist Robots" workshop at CoRL. Covers these works:Bridge v2: rail-berkeley.github.io/bridgedataGRIF: rail-berkeley.github.io/grifSuSIE: rail-berkeley.github.io/susieQ-Transformer: qtransformer.github.ioPTR: sites.google.com/view/ptr-finalICVF: dibyaghosh.com/icvfV-PTR: dibyaghosh.com/vptrRT-X: robotics-transformer-x.github.io
CS 285: Andrea Zanette: Towards a Statistical Foundation for Reinforcement Learning RAIL 2023-11-09 | ...
CS 285: Eric Mitchell: Reinforcement Learning from Human Feedback: Algorithms & Applications RAIL 2023-11-07 | Guest lecture in CS 285 by Eric Mitchell (Stanford)
Reinforcement Learning with Large Datasets: Robotics, Image Generation, and LLMs RAIL 2023-11-02 | Talk by Prof. Sergey Levine on RL with data for robotics, image generation, and LLMs
Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planning (Summary) RAIL 2023-10-17 | We present Language Frontier Guide (LFG), an algorithm for guiding real-world robot exploration by leveraging the semantic knowledge stored in LLMs."Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planning"Dhruv Shah*, Michael Equi*, Blazej Osinski, Fei Xia, Brian Ichter, Sergey LevineUC Berkeley, University of Warsaw, Google DeepMindProject Page:
NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration (Summary Video) RAIL 2023-10-12 | We present Navigation with Goal Masking and Diffusion (NoMaD), a Transformer-based policy trained on data from multiple ground robots, with a diffusion model decoder to flexibly handle both goal-conditioned and goal-agnostic navigation."NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration"Ajay Sridhar, Dhruv Shah, Catherine Glossop, Sergey Levine Berkeley AI Research (UC Berkeley)Project Page: general-navigation-models.github.ioarXiv pre-print: arxiv.org/abs/2310.07896
Multi-Stage Cable Routing Through Hierarchical Imitation Learning RAIL 2023-07-19 | Multi-Stage Cable Routing Through Hierarchical Imitation LearningJianlan Luo*, Charles Xu*, Xinyang Geng*, Gilbert Feng, Kuan Fang, Liam Tan, Stefan Schaal, Sergey LevineProject website: sites.google.com/view/cableroutingPaper: arxiv.org/abs/2307.08927
Data-Driven Reinforcement Learning for Robotic Manipulation RAIL 2023-07-10 | Recording of a talk prepared for the Industrial Assembly Workshop at RSS 2023, covering recent work on robotic manipulation, large datasets, and broadly reusable pretrained models in robotics.
Reinforcement Learning with Large Datasets: a Path to Resourceful Autonomous Agents RAIL 2023-07-07 | Presentation by Prof. Sergey Levine on offline RL, robot pretraining, and the intersection of optimization and data-driven AI.
ViNT: A Foundation Model for Visual Navigation (Summary Video) RAIL 2023-06-27 | We present the Visual Navigation Transformer (ViNT), a foundation model that aims to bring the success of general-purpose pre-trained models to vision-based robotic navigation. ViNT is trained with a general goal-reaching objective that can be used with any navigation dataset, and employs a flexible Transformer-based architecture to learn navigational affordances and enable efficient adaptation to a variety of downstream navigational tasks."ViNT: A Foundation Model for Visual Navigation"Dhruv Shah*, Ajay Sridhar*, Nitish Dashora*, Kyle Stachowicz, Kevin Black, Noriaki Hirose, Sergey LevineBerkeley AI Research (UC Berkeley)Project Page: visualnav-transformer.github.ioarXiv pre-print: arxiv.org/abs/2306.14846Timeline:--------------00:00 Introduction00:35 Model Architecture01:12 Exploration with ViNT01:41 Real-world Deployment03:10 Emergent Behaviors03:27 Adapting ViNT via Prompt-Tuning04:25 The End
A General-Purpose Robotic Navigation Model RAIL 2023-06-25 | Presentation by Sergey Levine on robotic foundation models for visual navigation. Covers the following papers:ViNG: sites.google.com/view/ving-robotRECON: sites.google.com/view/recon-robotViKiNG: sites.google.com/view/viking-releaseLM-Nav: sites.google.com/view/lmnavGNM: sites.google.com/view/drive-any-robotFastRLAP: sites.google.com/view/fastrlapViNT: coming soon :)
Robotic Learning with Large Datasets: Towards General-Purpose Models for All Robots RAIL 2023-06-03 | Lecture by Prof. Sergey LevineIncludes material from these projects:Bridge Data: rail-berkeley.github.io/bridgedataPTR: arxiv.org/abs/2210.05178FLAP: sites.google.com/view/project-flapGNM: sites.google.com/view/drive-any-robotFastRLAP: sites.google.com/view/fastrlapQT-Opt: arxiv.org/abs/1806.10293MT-Opt: karolhausman.github.io/mt-optActionable Models: actionable-models.github.ioRT-1: robotics-transformer.github.ioPaLM-E: palm-e.github.io
Reinforcement Learning Pretraining for Reinforcement Learning Finetuning RAIL 2023-04-23 | Lecture by Prof. Sergey Levine covering two recent works:Cal-QL: arxiv.org/abs/2303.05479ICVF: arxiv.org/abs/2304.04782
FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing (Summary) RAIL 2023-04-20 | "FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing'Kyle Stachowicz*, Arjun Bhorkar*, Dhruv Shah*, Ilya Kostrikov, Sergey LevineProject Page: sites.google.com/view/fastrlaparXiv pre-print: http://arxiv.org/abs/2304.09831
The Bitterest of Lessons: The Role of Data and Optimization in Emergence RAIL 2023-02-12 | A talk by Sergey Levine about emergence
Neural Software Abstractions: Michael Chang Dissertation Talk RAIL 2022-12-15 | Title: Neural Software Abstractions: Learning Abstractions for Automatically Modeling and Manipulating SystemsSpeaker: Michael Chang (mbchang.github.io)Advisors: Sergey Levine, Thomas L. GriffithsAbstract:The way neural networks are studied and built today bears a striking resemblance to how electronic circuits were studied and built 100 years ago: back then we manually designed electronic circuits for specific tasks while now we train neural circuits for specific tasks. The retrieval-augmented transformer, for example, is the deep-learning analogue to the von Neumann architecture of the 1940s. In this talk, I explore the question: what did it take for us to scale electronic circuits to the modern software stack we have today, and how can we apply a similar approach to scaling our neural circuits to create the future of learning software? I will discuss four examples of abstractions humans have invented for scaling electronic circuits to programmed software -- the digital abstraction, data abstraction, function abstraction, and problem abstraction. I will deconstruct the underlying principles that made these abstractions powerful, and show how we can apply these principles to design neural networks that exhibit many of the same properties and benefits that we get from discreteness, variables, reusable computations, and hierarchical organization, without explicitly defining these abstractions ourselves. Similarly to how programmed software enabled humans to manually model and manipulate systems, this line of work, which I call "neural software abstractions," aims to build learning software for automatically modeling and manipulating systems. I conclude by situating neural software abstractions in the broader context of the relationship between AI and humans, arguing for research on both neural software abstractions and adaptive human computer interfaces as two complementary research directions towards building AI that enables humans to do more with less.
CS 285: Guest Lecture: Bo Dai RAIL 2022-11-29 | Guest lecture by Bo Dai in UC Berkeley's CS 285: Deep Reinforcement Learning
CS 285: Lecture 23: Open Problems RAIL 2022-11-29 | Lecture on challenges and open problems in UC Berkeley's CS 285: Deep Reinforcement Learning
Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement RAIL 2022-11-28 | Contributed oral at the All Things Attention (attention-learning-workshop.github.io) NeurIPS 2022 workshopAuthors:Michael Changmbchang@berkeley.edumbchang.github.io Li Dayandayan@berkeley.edualyd.github.ioFranziska Meierfmeier@meta.com fmeier.github.io Thomas L. Griffiths tomg@princeton.eduhttps://cocosci.princeton.edu/tom/index.phpSergey Levine svlevine@eecs.berkeley.eduhttps://people.eecs.berkeley.edu/~svlevine/Amy Zhangamyzhang@meta.com amyzhang.github.io
Object Representations as Fixed Points RAIL 2022-11-28 | Conference publication at NeurIPS 2022.sites.google.com/view/implicit-slot-attention/home.Object Representations as Fixed Points: Training Iterative Inference Algorithms with Implicit DifferentiationAuthors:Michael Changmbchang@berkeley.edumbchang.github.io Thomas L. Griffiths tomg@princeton.eduhttps://cocosci.princeton.edu/tom/index.phpSergey Levine svlevine@eecs.berkeley.eduhttps://people.eecs.berkeley.edu/~svlevine/
Deep Reinforcement Learning with Real-World Data RAIL 2022-11-15 | Lecture by Sergey Levine on how offline reinforcement learning can provide an effective way to leverage previously collected data.Covers:CQL: arxiv.org/abs/2006.04779PTR: arxiv.org/abs/2210.05178FLAP: sites.google.com/view/project-flapILQL: sea-snell.github.io/ILQL_site
