Uploaded November 2023 | Updated September 2026, 2 weeks ago
#qlearning #qstar #rlhf
What is Q-Learning and how does it work? A brief tour through the background of Q-Learning, Markov Decision Processes, Deep Q-Networks, and other basics necessary to understand Q* ;)
OUTLINE:
0:00 - Introduction
2:00 - Reinforcement Learning
7:00 - Q-Functions
19:00 - The Bellman Equation
26:00 - How to learn the Q-Function?
38:00 - Deep Q-Learning
42:30 - Summary
Paper: arxiv.org/abs/1312.5602
My old video on DQN: youtu.be/rFwQDDbYTm4
Links:
Homepage: ykilcher.com
Merch: ykilcher.com/merch
YouTube: youtube.com/c/yannickilcher
Twitter: twitter.com/ykilcher
Discord: ykilcher.com/discord
LinkedIn: linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: subscribestar.com/yannickilcher
Patreon: patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
#qlearning #qstar #rlhf
What is Q-Learning and how does it work? A brief tour through the background of Q-Learning, Markov Decision Processes, Deep Q-Networks, and other basics necessary to understand Q* ;)
OUTLINE:
0:00 - Introduction
2:00 - Reinforcement Learning
7:00 - Q-Functions
19:00 - The Bellman Equation
26:00 - How to learn the Q-Function?
38:00 - Deep Q-Learning
42:30 - Summary
Paper: arxiv.org/abs/1312.5602
My old video on DQN: youtu.be/rFwQDDbYTm4
Links:
Homepage: ykilcher.com
Merch: ykilcher.com/merch
YouTube: youtube.com/c/yannickilcher
Twitter: twitter.com/ykilcher
Discord: ykilcher.com/discord
LinkedIn: linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: subscribestar.com/yannickilcher
Patreon: patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
![[ML News] Metas OPT 175B language model | DALL-E Mega is training | TorToiSe TTS fakes my voice](https://i.ytimg.com/vi/pwSnC8jlh50/mqdefault.jpg "[ML News] Metas OPT 175B language model | DALL-E Mega is training | TorToiSe TTS fakes my voice
#mlnews #dalle #gpt3
An inside look of whats happening in the ML world!
Sponsor: Weights & Biases
https://wandb.me/yannic
OUTLINE:
0:00 - Intro
0:20 - Sponsor: Weights & Biases
1:40 - Meta AI releases OPT-175B
4:55 - CoCa: New CLIP-Competitor
8:15 - DALL-E Mega is training
10:05 - TorToiSe TTS is amazing!
11:50 - Investigating Vision Transformers
12:50 - Hugging Face Deep RL class launched
13:40 - Helpful Things
17:00 - John Deeres driverless tractors
References:
Meta AI releases OPT-175B
https://ai.facebook.com/blog/democratizing-access-to-large-scale-language-models-with-opt-175b/
https://arxiv.org/abs/2205.01068
https://arxiv.org/pdf/2205.01068.pdf
https://github.com/facebookresearch/metaseq/tree/main/projects/OPT
https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf
https://github.com/facebookresearch/metaseq/tree/main/projects/OPT/chronicles
https://twitter.com/yoavgo/status/1522150063815987201
CoCa: New CLIP-Competitor
https://arxiv.org/abs/2205.01917
https://arxiv.org/pdf/2205.01917.pdf
DALL-E Mega is training
https://twitter.com/borisdayma
https://twitter.com/borisdayma/status/1521891895001112577
https://wandb.ai/dalle-mini/dalle-mini/reports/DALL-E-Mega VmlldzoxODMxMDI2
TorToiSe TTS is amazing!
https://github.com/neonbjb/tortoise-tts
https://nonint.com/static/tortoise_v2_examples.html
https://colab.research.google.com/drive/1wVVqUPqwiDBUVeWWOUNglpGhU3hg_cbR
https://github.com/neonbjb
Investigating Vision Transformers
https://github.com/sayakpaul/probing-vits/?utm_source=pocket_mylist
https://twitter.com/RisingSayak/status/1515918406171914240?utm_source=pocket_mylist
https://keras.io/examples/vision/probing_vits/
https://github.com/sayakpaul/probing-vits/tree/main/notebooks?utm_source=pocket_mylist
Hugging Face Deep RL class launched
https://github.com/huggingface/deep-rl-class
Helpful Things
https://merantix-momentum.com/technology/squirrel/?utm_source=pocket_mylist
https://github.com/merantix-momentum/squirrel-core?utm_source=pocket_mylist
https://pyscript.net/?utm_source=pocket_mylist
https://github.com/google-research/big_vision
https://deepsportradar.github.io/challenge.html
https://github.com/DeepSportRadar/camera-calibration-challenge
https://twitter.com/alekseykorshuk/status/1515989357961920514?utm_source=pocket_mylist
https://github.com/AlekseyKorshuk/huggingnft
John Deeres driverless tractors
https://thenextweb.com/news/john-deere-slowly-becoming-one-worlds-most-important-ai-companies
https://tractorhacking.github.io/
Links:
Merch: https://ykilcher.com/merch
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n")
![[ML News] Devin AI Software Engineer | GPT-4.5-Turbo LEAKED | US Govt Report: Total Extinction](https://i.ytimg.com/vi/q1LrXH5_Oy0/mqdefault.jpg "[ML News] Devin AI Software Engineer | GPT-4.5-Turbo LEAKED | US Govt Report: Total Extinction
Your weekly dose of ML News
OUTLINE:
0:00 - Intro
0:15 - Devin: AI software engineer
5:50 - Mira Murati on Sora training data
6:50 - Inflection accused of copying Claude
9:00 - Tools & papers
16:30 - GPT-4.5-turbo mystery
17:30 - US government report: total extinction by AI
19:20 - Various other news
References:
https://www.cognition-labs.com/introducing-devin
https://twitter.com/cognition_labs/status/1767548763134964000?t=ZECIn-uqbguwHtY8X_Gvtw&s=09
https://news.google.com/stories/CAAqNggKIjBDQklTSGpvSmMzUnZjbmt0TXpZd1NoRUtEd2lWMUwyU0N4RnVWM3pSRWhWX01pZ0FQAQ?hl=en-US&gl=US&ceid=US%3Aen
https://www.bloomberg.com/news/articles/2024-03-12/cognition-ai-is-a-peter-thiel-backed-coding-assistant?embedded-checkout=true
https://www.bloomberg.com/authors/AQWHkoPod9g/ashlee-vance
https://www.bloomberg.com/news/articles/2024-03-12/cognition-ai-is-a-peter-thiel-backed-coding-assistant?srnd=undefined&embedded-checkout=true
https://www.bloomberg.com/news/newsletters/2024-03-12/cognition-ai-s-devin-assistant-can-build-websites-videos-from-a-prompt?srnd=undefined&embedded-checkout=true
https://archive.ph/5LZV9
https://github.com/opendevin/opendevin
https://twitter.com/MetaGPT_/status/1767965444579692832?t=dsYKmPfOBVGCFCwvPtZVWQ&s=09
https://docs.deepwisdom.ai/main/en/DataInterpreter/detail.html?id=AppleStockPriceAnalysisAndPrediction
https://docs.deepwisdom.ai/main/en/guide/use_cases/agent/interpreter/intro.html
https://github.com/geekan/MetaGPT/tree/main/examples/di
https://inflection.ai/inflection-2-5
https://twitter.com/seshubon/status/1765870717844050221
https://twitter.com/inflectionAI/status/1766173427441049684
https://www.mlxserver.com/
https://huggingface.co/spaces/mlabonne/AutoMerger
https://github.com/microsoft/aici
https://github.com/google-research/google-research/tree/master/fax
https://github.com/stanfordnlp/pyvene
https://arxiv.org/pdf/2403.06634.pdf
https://twitter.com/mattshumer_/status/1767606938538295757?t=1dYect5ylg9xrWSS4sL38Q&s=09
https://time.com/6898967/ai-extinction-national-security-risks-report/
https://venturebeat.com/ai/hugging-face-is-launching-an-open-source-robotics-project-led-by-former-tesla-scientist/
https://twitter.com/gcabanac/status/1767574447337124290?t=MnzwEbf_Zx0yQthe0RQ8hw&s=09
https://twitter.com/AnthropicAI/status/1768018310615151002?t=3ieMvNZxaoTXGGZttBsBvQ&s=09
https://huggingface.co/CohereForAI/c4ai-command-r-v01
https://twitter.com/Yampeleg/status/1765707714473197729?t=p3zOXUqKdqS-RzYjTNo65g&s=09
https://huggingface.co/yam-peleg/Hebrew-Gemma-11B
https://enriccorona.github.io/vlogger/
https://huggingface.co/NousResearch/Genstruct-7B
https://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/
https://arxiv.org/abs/2403.04652
https://twitter.com/corry_wang/status/1766949316394897851?t=i0ndsef_I_b3BDkVmyHgYw&s=09
https://twitter.com/sama/status/1766291001134715207?t=Wgyye9odOfF1Aoo0hZGihg&s=09
https://venturebeat.com/ai/nist-staffers-revolt-against-potential-appointment-of-effective-altruist-ai-researcher-to-us-ai-safety-institute/
https://occiglot.github.io/occiglot/posts/occiglot-announcement/
https://twitter.com/EMostaque/status/1767199048337932719?t=tYB3KeabfLlB90XhUX0R7A&s=09
Links:
Homepage: https://ykilcher.com
Merch: https://ykilcher.com/merch
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
LinkedIn: https://www.linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n")
