Uploaded March 2023 | Updated September 2026, 2 weeks ago
#mlnews #chatgpt #llama
ChatGPT goes around the world and is finally available via API. Stunning mind-reading performed using fMRI and Stable Diffusion. LLaMA weights leak and hilarity ensues. GTC23 is around the corner!
ERRATA: It's a 4090, not a 4090 ti 🙃
OUTLINE:
0:00 - Introduction
0:20 - GTC 23 on March 20
1:55 - ChatGPT API is out!
4:50 - OpenAI becomes more business-friendly
7:15 - OpenAI plans for AGI
10:00 - ChatGPT influencers
12:15 - Open-Source Prompting Course
12:35 - Flan UL2 20B
13:30 - LLaMA weights leaked
15:50 - Mind-Reading from fMRI
20:10 - Random News / Helpful Things
25:30 - Interview with Bryan Catanzaro
Participate in the GTC Raffle: ykilcher.com/gtc
References:
GTC 23 on March 20
nvidia.com/gtc
ykilcher.com/gtc
ChatGPT API is out!
twitter.com/gdb/status/1630991925984755714
openai.com/blog/introducing-chatgpt-and-whisper-apis
twitter.com/greggyb/status/1631121912679002112
haihai.ai/chatgpt-api
OpenAI becomes more business-friendly
twitter.com/sama/status/1631002519311888385
techcrunch.com/2023/02/21/openai-foundry-will-let-customers-buy-dedicated-capacity-to-run-its-ai-models/?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAFL1O8s22qBsEtytYZWR7O2VlTa9nAGhdZPFfeQfZCDWjkNBIac7WlDikRNLEH1tqSszUN02ouqRyyCsShDa1kQyUbiApD1IUPfgmHXZxgIMFxr8bwr8BuBa7sK55dYqMRFFbE7YILuBn_rmj7aJI1tp7GAXubODfCUaKvOkoOYj
bain.com/vector-digital/partnerships-alliance-ecosystem/openai-alliance
OpenAI plans for AGI
openai.com/blog/planning-for-agi-and-beyond
ChatGPT influencers
youtube.com/watch?v=4kp7oVTu9Ck
youtube.com/watch?v=k13v8jp8H5o
linkedin.com/posts/eniascailliau_create-an-online-course-100-ai-ugcPost-7036969935796891648-H_uj
linkedin.com/posts/linasbeliunas_must-know-ai-tools-ugcPost-7035700089947836416-Qri4
twitter.com/LinusEkenstam/status/1629879567514238976
linkedin.com/posts/imarpit_50-awesome-chatgpt-prompts-ugcPost-7036905788631646209-2CU-
Open-Source Prompting Course
learnprompting.org
Flan UL2 20B
yitay.net/blog/flan-ul2-20b
huggingface.co/google/flan-ul2
LLaMA weights leaked
github.com/facebookresearch/llama/pull/73
github.com/facebookresearch/llama/pull/73/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5
github.com/ChristopherKing42
open-assistant.io/dashboard
Mind-Reading from fMRI
sites.google.com/view/stablediffusion-with-brain/?s=09
nature.com/articles/s41562-022-01516-2?utm_content=animation
Random News
wired.com/story/alphabet-layoffs-hit-trash-sorting-robots
huggingface.co/blog/fast-mac-diffusers
pyribs.org
twitter.com/rowancheung/status/1630569844654460928
pimeyes.com/en
cacti-framework.github.io
twitter.com/bhutanisanyam1/status/1630980866775330819
linkedin.com/in/bryancatanzaro
Links:
Homepage: ykilcher.com
Merch: ykilcher.com/merch
YouTube: youtube.com/c/yannickilcher
Twitter: twitter.com/ykilcher
Discord: ykilcher.com/discord
LinkedIn: linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: subscribestar.com/yannickilcher
Patreon: patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
#mlnews #chatgpt #llama
ChatGPT goes around the world and is finally available via API. Stunning mind-reading performed using fMRI and Stable Diffusion. LLaMA weights leak and hilarity ensues. GTC23 is around the corner!
ERRATA: It's a 4090, not a 4090 ti 🙃
OUTLINE:
0:00 - Introduction
0:20 - GTC 23 on March 20
1:55 - ChatGPT API is out!
4:50 - OpenAI becomes more business-friendly
7:15 - OpenAI plans for AGI
10:00 - ChatGPT influencers
12:15 - Open-Source Prompting Course
12:35 - Flan UL2 20B
13:30 - LLaMA weights leaked
15:50 - Mind-Reading from fMRI
20:10 - Random News / Helpful Things
25:30 - Interview with Bryan Catanzaro
Participate in the GTC Raffle: ykilcher.com/gtc
References:
GTC 23 on March 20
nvidia.com/gtc
ykilcher.com/gtc
ChatGPT API is out!
twitter.com/gdb/status/1630991925984755714
openai.com/blog/introducing-chatgpt-and-whisper-apis
twitter.com/greggyb/status/1631121912679002112
haihai.ai/chatgpt-api
OpenAI becomes more business-friendly
twitter.com/sama/status/1631002519311888385
techcrunch.com/2023/02/21/openai-foundry-will-let-customers-buy-dedicated-capacity-to-run-its-ai-models/?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAFL1O8s22qBsEtytYZWR7O2VlTa9nAGhdZPFfeQfZCDWjkNBIac7WlDikRNLEH1tqSszUN02ouqRyyCsShDa1kQyUbiApD1IUPfgmHXZxgIMFxr8bwr8BuBa7sK55dYqMRFFbE7YILuBn_rmj7aJI1tp7GAXubODfCUaKvOkoOYj
bain.com/vector-digital/partnerships-alliance-ecosystem/openai-alliance
OpenAI plans for AGI
openai.com/blog/planning-for-agi-and-beyond
ChatGPT influencers
youtube.com/watch?v=4kp7oVTu9Ck
youtube.com/watch?v=k13v8jp8H5o
linkedin.com/posts/eniascailliau_create-an-online-course-100-ai-ugcPost-7036969935796891648-H_uj
linkedin.com/posts/linasbeliunas_must-know-ai-tools-ugcPost-7035700089947836416-Qri4
twitter.com/LinusEkenstam/status/1629879567514238976
linkedin.com/posts/imarpit_50-awesome-chatgpt-prompts-ugcPost-7036905788631646209-2CU-
Open-Source Prompting Course
learnprompting.org
Flan UL2 20B
yitay.net/blog/flan-ul2-20b
huggingface.co/google/flan-ul2
LLaMA weights leaked
github.com/facebookresearch/llama/pull/73
github.com/facebookresearch/llama/pull/73/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5
github.com/ChristopherKing42
open-assistant.io/dashboard
Mind-Reading from fMRI
sites.google.com/view/stablediffusion-with-brain/?s=09
nature.com/articles/s41562-022-01516-2?utm_content=animation
Random News
wired.com/story/alphabet-layoffs-hit-trash-sorting-robots
huggingface.co/blog/fast-mac-diffusers
pyribs.org
twitter.com/rowancheung/status/1630569844654460928
pimeyes.com/en
cacti-framework.github.io
twitter.com/bhutanisanyam1/status/1630980866775330819
linkedin.com/in/bryancatanzaro
Links:
Homepage: ykilcher.com
Merch: ykilcher.com/merch
YouTube: youtube.com/c/yannickilcher
Twitter: twitter.com/ykilcher
Discord: ykilcher.com/discord
LinkedIn: linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: subscribestar.com/yannickilcher
Patreon: patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
![[ML News] ConvNeXt: Convolutions return | China regulates algorithms | Saliency cropping examined](https://i.ytimg.com/vi/yVKiMh2vEWQ/mqdefault.jpg "[ML News] ConvNeXt: Convolutions return | China regulates algorithms | Saliency cropping examined
#mlnews #convnext #mt3
Your update on whats new in the Machine Learning world!
OUTLINE:
0:00 - Intro
0:15 - ConvNeXt: Return of the Convolutions
2:50 - Investigating Saliency Cropping Algorithms
9:40 - YourTTS: SOTA zero-shot Text-to-Speech
10:40 - MT3: Multi-Track Music Transcription
11:35 - China regulates addictive algorithms
13:00 - A collection of Deep Learning interview questions & solutions
13:35 - Helpful Things
16:05 - AlphaZero explained blog post
16:45 - Ru-DOLPH: HyperModal Text-to-Image-to-Text model
17:45 - Google AI 2021 Review
References:
ConvNeXt: Return of the Convolutions
https://arxiv.org/abs/2201.03545
https://github.com/facebookresearch/ConvNeXt
https://twitter.com/giffmana/status/1481054929573888005
https://twitter.com/wightmanr/status/1481150080765739009
https://twitter.com/tanmingxing/status/1481362887272636417
Investigating Saliency Cropping Algorithms
https://openaccess.thecvf.com/content/WACV2022/papers/Birhane_Auditing_Saliency_Cropping_Algorithms_WACV_2022_paper.pdf
https://vinayprabhu.github.io/Saliency_Image_Cropping/paper_html/main.html
https://vinayprabhu.medium.com/on-the-twitter-cropping-controversy-critique-clarifications-and-comments-7ac66154f687
https://vinayprabhu.github.io/Saliency_Image_Cropping/
YourTTS: SOTA zero-shot Text-to-Speech
https://github.com/coqui-ai/TTS?utm_source=pocket_mylist
https://arxiv.org/abs/2112.02418?utm_source=pocket_mylist
https://coqui.ai/?utm_source=pocket_mylist
https://coqui.ai/blog/tts/yourtts-zero-shot-text-synthesis-low-resource-languages
MT3: Multi-Track Music Transcription
https://arxiv.org/abs/2111.03017
https://github.com/magenta/mt3
https://huggingface.co/spaces/akhaliq/MT3
https://www.reddit.com/r/MachineLearning/comments/rtlx0r/r_mt3_multitask_multitrack_music_transcription/
China regulates addictive algorithms
https://technode.com/2022/01/05/china-issues-new-rules-to-regulate-algorithms-targeting-addiction-monopolies-and-overspending/
https://qz.com/2109618/china-reveals-new-algorithm-rules-to-weaken-platforms-control-of-users/
A collection of Deep Learning interview questions & solutions
https://arxiv.org/abs/2201.00650?utm_source=pocket_mylist
https://arxiv.org/pdf/2201.00650.pdf
Helpful Things
https://docs.deepchecks.com/en/stable/index.html
https://github.com/deepchecks/deepchecks
https://docs.deepchecks.com/en/stable/examples/guides/quickstart_in_5_minutes.html
https://www.dagshub.com/
https://www.dagshub.com/docs/index.html
https://www.dagshub.com/blog/launching-dagshub-2-0/
https://bayesiancomputationbook.com/welcome.html
https://mlcontests.com/
https://github.com/Yard1/ray-skorch
https://github.com/skorch-dev/skorch
https://www.rumbledb.org/?utm_source=pocket_mylist
https://github.com/DarshanDeshpande/jax-models
https://github.com/s3prl/s3prl
AlphaZero explained blog post
https://joshvarty.github.io/AlphaZero/?utm_source=pocket_mylist
Ru-DOLPH: HyperModal Text-to-Image-to-Text model
https://github.com/sberbank-ai/ru-dolph
https://colab.research.google.com/drive/1gmTDA13u709OXiAeXWGm7sPixRhEJCga?usp=sharing
Google AI 2021 Review
https://ai.googleblog.com/2022/01/google-research-themes-from-2021-and.html
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n")
![[Paper Analysis] On the Theoretical Limitations of Embedding-Based Retrieval (Warning: Rant)](https://i.ytimg.com/vi/zKohTkN0Fyk/mqdefault.jpg "[Paper Analysis] On the Theoretical Limitations of Embedding-Based Retrieval (Warning: Rant)
Paper: https://arxiv.org/abs/2508.21038
Abstract:
Vector embeddings have been tasked with an ever-increasing set of retrieval tasks over the years, with a nascent rise in using them for reasoning, instruction-following, coding, and more. These new benchmarks push embeddings to work for any query and any notion of relevance that could be given. While prior works have pointed out theoretical limitations of vector embeddings, there is a common assumption that these difficulties are exclusively due to unrealistic queries, and those that are not can be overcome with better training data and larger models. In this work, we demonstrate that we may encounter these theoretical limitations in realistic settings with extremely simple queries. We connect known results in learning theory, showing that the number of top-k subsets of documents capable of being returned as the result of some query is limited by the dimension of the embedding. We empirically show that this holds true even if we restrict to k=2, and directly optimize on the test set with free parameterized embeddings. We then create a realistic dataset called LIMIT that stress tests models based on these theoretical results, and observe that even state-of-the-art models fail on this dataset despite the simple nature of the task. Our work shows the limits of embedding models under the existing single vector paradigm and calls for future research to develop methods that can resolve this fundamental limitation.
Authors: Orion Weller, Michael Boratko, Iftekhar Naim, Jinhyuk Lee
Links:
Homepage: https://ykilcher.com
Merch: https://ykilcher.com/merch
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
LinkedIn: https://www.linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n")
