Uploaded April 2023 | Updated September 2026, 4 hours ago
PaLM-E is a new LLM from Google that is both embodied in multimodal. Excitingly, it shows positive transfer across different robotics tasks. While the prospects are exciting, it is unclear what other conclusions can be drawn from the work.
Outline
0:00 - Intro
1:34 - How It Works
6:47 - Robotics Tasks
11:25 - Results
26:50 - Takeaways
Social Media
YouTube - youtube.com/c/EdanMeyer
Twitter - twitter.com/ejmejm1
Sources:
PaLM-E paper - arxiv.org/abs/2303.03378
PaLM-E is a new LLM from Google that is both embodied in multimodal. Excitingly, it shows positive transfer across different robotics tasks. While the prospects are exciting, it is unclear what other conclusions can be drawn from the work.
Outline
0:00 - Intro
1:34 - How It Works
6:47 - Robotics Tasks
11:25 - Results
26:50 - Takeaways
Social Media
YouTube - youtube.com/c/EdanMeyer
Twitter - twitter.com/ejmejm1
Sources:
PaLM-E paper - arxiv.org/abs/2303.03378
![Learning Language Through Games [Zero to Paper]](https://i.ytimg.com/vi/qY0nCUeQlXI/mqdefault.jpg "Learning Language Through Games [Zero to Paper]
Lets talk about natural language in Reinforcement Learning. Its also a form of language grounding because models are trained to learn connections between language and a non-text environments. I think language-conditioned RL is the way forward for making more efficient, general AI training. Let me know what you think of this combination between NLP and RL in the comments!
Zero to Paper playlist: https://www.youtube.com/playlist?list=PL_49VD9KwQ_ONxENRk11jFEI3_pqAwaug
Inverse Reinforcement Learning video: https://www.youtube.com/watch?v=qo355ALvLRI
Papers covered:
https://arxiv.org/pdf/2005.09382.pdf
https://arxiv.org/pdf/1902.07742.pdf
https://arxiv.org/pdf/1806.01946.pdf")
