Uploaded August 2023 | Updated September 2026, 2 weeks ago
#mlnews #llama2 #openai
Your regular irregular update on the world of Machine Learning.
References:
twitter.com/ylecun/status/1681336284453781505
ai.meta.com/llama
about.fb.com/news/2023/07/llama-2-statement-of-support
247wallst.com/special-report/2023/08/12/this-is-the-biggest-social-media-platform-ranking-the-worlds-largest-networking-sites/4
github.com/Alpha-VLLM/LLaMA2-Accessory
together.ai/blog/llama-2-7b-32k?s=09&utm_source=pocket_saves
github.com/imoneoi/openchat
twitter.com/lmsysorg/status/1686794639469371393?s=09&t=sS3awkbavmSMSmwp64Ef4A&utm_source=pocket_saves
huggingface.co/lmsys/vicuna-13b-v1.5-16k
https://blog.google/outreach-initiatives/public-policy/google-microsoft-openai-anthropic-frontier-model-forum/
earthdata.nasa.gov/news/impact-ibm-hls-foundation-model?utm_source=pocket_reader
huggingface.co/ibm-nasa-geospatial/Prithvi-100M
ai.meta.com/blog/generative-ai-text-images-cm3leon
deepmind.com/blog/rt-2-new-model-translates-vision-and-language-into-action?utm_source=twitter&utm_medium=social&utm_campaign=rt2
arxiv.org/abs/2307.14334
https://sites.research.google/med-palm/
open-catalyst.metademolab.com/?utm_source=twitter&utm_medium=organic_social&utm_campaign=opencatalyst&utm_content=card
open-catalyst.metademolab.com/demo
anthropic.com/index/claude-2?utm_source=pocket_reader
claude.ai/login
audiocraft.metademolab.com/?utm_source=pocket_saves
venturebeat.com/programming-development/stability-ai-launches-stablecode-an-llm-for-code-generation
stability.ai/blog/stablecode-llm-generative-ai-coding
twitter.com/JeffDean/status/1686806525862608896?s=09&t=LG2z9ok9QExHbSy0fvBsxA&utm_source=pocket_saves
https://sites.research.google/open-buildings/
twitter.com/deliprao/status/1687283117873106946?s=09&t=1NmC-B55Z8IuF_HTuGOo7w&utm_source=pocket_saves
arxiv.org/pdf/2308.01320.pdf
twitter.com/javilopen/status/1687795349719547905?utm_source=pocket_saves
research.nvidia.com/labs/par/Perfusion
ar5iv.labs.arxiv.org/html/2307.14936
linkedin.com/feed/update/urn:li:activity:7093463974750371840/?utm_source=pocket_saves
huggingface.co/syzymon/long_llama_3b_instruct
arxiv.org/abs/2307.03170
dynalang.github.io
github.com/mlfoundations/open_flamingo
twitter.com/akshay_pachaar/status/1687079353937698816?s=09&t=fos8QSCsGEEM6dMflhq0Mg&utm_source=pocket_saves
github.com/OpenBMB/ToolBench
llm-attacks.org
arstechnica.com/information-technology/2023/07/openai-discontinues-its-ai-writing-detector-due-to-low-rate-of-accuracy
sites.google.com/view/steve-1
github.com/Shalev-Lifshitz/STEVE-1
erichartford.com/dolphin
huggingface.co/ehartford/dolphin-llama-13b
mosaicml.com/blog/long-context-mpt-7b-8k
twitter.com/camenduru/status/1688045780244848640?s=09&t=ubJ2Qtz-TG6Xo3_GMtt2Cw&utm_source=pocket_saves
github.com/IDEA-Research/DWPose
twitter.com/tri_dao/status/1680987577913065472?s=09&t=Q181vFmM6d3nDq-5BwfDeg&utm_source=pocket_saves
tridao.me/publications/flash2/flash2.pdf
thehackernews.com/2023/07/wormgpt-new-ai-tool-allows.html
tomshardware.com/news/ai-steals-data-with-keystroke-audio
arxiv.org/pdf/2308.01074.pdf
foxnews.com/politics/ai-test-flight-air-force-unmanned-wingman-aircraft
theverge.com/2023/8/2/23817406/white-castle-soundhound-ai-sliders
google.com/search?sca_esv=556495916&q=food+delivery+bot+kicked&tbm=vid&source=lnms&sa=X&ved=2ahUKEwjZ6PDPrdmAAxUThf0HHWzrBGgQ0pQJegQIChAB&cshid=1691920142432720&biw=2327&bih=1180&dpr=2.2
youtube.com/watch?v=--n_NhmXnfc
thesun.co.uk/tech/20793591/coop-delivery-robots-cambridge-kicked-by-workers-tiktok
ktla.com/news/local-news/food-delivery-robots-under-attack-from-vandals-thieves-local-businesses-starting-to-be-affected
youtube.com/watch?v=xxzS9qaARv0
psypost.org/2023/08/chatgpt-is-much-better-than-humans-at-accurately-identifying-emotions-in-fictional-textual-scenarios-167380
theverge.com/2023/8/1/23815287/meta-ai-persona-generative-llama-instagram-facebook
cnbc.com/2023/07/28/microsoft-annual-report-highlights-importance-of-gpus.html
Links:
Homepage: ykilcher.com
Merch: ykilcher.com/merch
YouTube: youtube.com/c/yannickilcher
Twitter: twitter.com/ykilcher
Discord: ykilcher.com/discord
LinkedIn: linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
#mlnews #llama2 #openai
Your regular irregular update on the world of Machine Learning.
References:
twitter.com/ylecun/status/1681336284453781505
ai.meta.com/llama
about.fb.com/news/2023/07/llama-2-statement-of-support
247wallst.com/special-report/2023/08/12/this-is-the-biggest-social-media-platform-ranking-the-worlds-largest-networking-sites/4
github.com/Alpha-VLLM/LLaMA2-Accessory
together.ai/blog/llama-2-7b-32k?s=09&utm_source=pocket_saves
github.com/imoneoi/openchat
twitter.com/lmsysorg/status/1686794639469371393?s=09&t=sS3awkbavmSMSmwp64Ef4A&utm_source=pocket_saves
huggingface.co/lmsys/vicuna-13b-v1.5-16k
https://blog.google/outreach-initiatives/public-policy/google-microsoft-openai-anthropic-frontier-model-forum/
earthdata.nasa.gov/news/impact-ibm-hls-foundation-model?utm_source=pocket_reader
huggingface.co/ibm-nasa-geospatial/Prithvi-100M
ai.meta.com/blog/generative-ai-text-images-cm3leon
deepmind.com/blog/rt-2-new-model-translates-vision-and-language-into-action?utm_source=twitter&utm_medium=social&utm_campaign=rt2
arxiv.org/abs/2307.14334
https://sites.research.google/med-palm/
open-catalyst.metademolab.com/?utm_source=twitter&utm_medium=organic_social&utm_campaign=opencatalyst&utm_content=card
open-catalyst.metademolab.com/demo
anthropic.com/index/claude-2?utm_source=pocket_reader
claude.ai/login
audiocraft.metademolab.com/?utm_source=pocket_saves
venturebeat.com/programming-development/stability-ai-launches-stablecode-an-llm-for-code-generation
stability.ai/blog/stablecode-llm-generative-ai-coding
twitter.com/JeffDean/status/1686806525862608896?s=09&t=LG2z9ok9QExHbSy0fvBsxA&utm_source=pocket_saves
https://sites.research.google/open-buildings/
twitter.com/deliprao/status/1687283117873106946?s=09&t=1NmC-B55Z8IuF_HTuGOo7w&utm_source=pocket_saves
arxiv.org/pdf/2308.01320.pdf
twitter.com/javilopen/status/1687795349719547905?utm_source=pocket_saves
research.nvidia.com/labs/par/Perfusion
ar5iv.labs.arxiv.org/html/2307.14936
linkedin.com/feed/update/urn:li:activity:7093463974750371840/?utm_source=pocket_saves
huggingface.co/syzymon/long_llama_3b_instruct
arxiv.org/abs/2307.03170
dynalang.github.io
github.com/mlfoundations/open_flamingo
twitter.com/akshay_pachaar/status/1687079353937698816?s=09&t=fos8QSCsGEEM6dMflhq0Mg&utm_source=pocket_saves
github.com/OpenBMB/ToolBench
llm-attacks.org
arstechnica.com/information-technology/2023/07/openai-discontinues-its-ai-writing-detector-due-to-low-rate-of-accuracy
sites.google.com/view/steve-1
github.com/Shalev-Lifshitz/STEVE-1
erichartford.com/dolphin
huggingface.co/ehartford/dolphin-llama-13b
mosaicml.com/blog/long-context-mpt-7b-8k
twitter.com/camenduru/status/1688045780244848640?s=09&t=ubJ2Qtz-TG6Xo3_GMtt2Cw&utm_source=pocket_saves
github.com/IDEA-Research/DWPose
twitter.com/tri_dao/status/1680987577913065472?s=09&t=Q181vFmM6d3nDq-5BwfDeg&utm_source=pocket_saves
tridao.me/publications/flash2/flash2.pdf
thehackernews.com/2023/07/wormgpt-new-ai-tool-allows.html
tomshardware.com/news/ai-steals-data-with-keystroke-audio
arxiv.org/pdf/2308.01074.pdf
foxnews.com/politics/ai-test-flight-air-force-unmanned-wingman-aircraft
theverge.com/2023/8/2/23817406/white-castle-soundhound-ai-sliders
google.com/search?sca_esv=556495916&q=food+delivery+bot+kicked&tbm=vid&source=lnms&sa=X&ved=2ahUKEwjZ6PDPrdmAAxUThf0HHWzrBGgQ0pQJegQIChAB&cshid=1691920142432720&biw=2327&bih=1180&dpr=2.2
youtube.com/watch?v=--n_NhmXnfc
thesun.co.uk/tech/20793591/coop-delivery-robots-cambridge-kicked-by-workers-tiktok
ktla.com/news/local-news/food-delivery-robots-under-attack-from-vandals-thieves-local-businesses-starting-to-be-affected
youtube.com/watch?v=xxzS9qaARv0
psypost.org/2023/08/chatgpt-is-much-better-than-humans-at-accurately-identifying-emotions-in-fictional-textual-scenarios-167380
theverge.com/2023/8/1/23815287/meta-ai-persona-generative-llama-instagram-facebook
cnbc.com/2023/07/28/microsoft-annual-report-highlights-importance-of-gpus.html
Links:
Homepage: ykilcher.com
Merch: ykilcher.com/merch
YouTube: youtube.com/c/yannickilcher
Twitter: twitter.com/ykilcher
Discord: ykilcher.com/discord
LinkedIn: linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)

![[ML News] ConvNeXt: Convolutions return | China regulates algorithms | Saliency cropping examined
#mlnews #convnext #mt3
Your update on whats new in the Machine Learning world!
OUTLINE:
0:00 - Intro
0:15 - ConvNeXt: Return of the Convolutions
2:50 - Investigating Saliency Cropping Algorithms
9:40 - YourTTS: SOTA zero-shot Text-to-Speech
10:40 - MT3: Multi-Track Music Transcription
11:35 - China regulates addictive algorithms
13:00 - A collection of Deep Learning interview questions & solutions
13:35 - Helpful Things
16:05 - AlphaZero explained blog post
16:45 - Ru-DOLPH: HyperModal Text-to-Image-to-Text model
17:45 - Google AI 2021 Review
References:
ConvNeXt: Return of the Convolutions
https://arxiv.org/abs/2201.03545
https://github.com/facebookresearch/ConvNeXt
https://twitter.com/giffmana/status/1481054929573888005
https://twitter.com/wightmanr/status/1481150080765739009
https://twitter.com/tanmingxing/status/1481362887272636417
Investigating Saliency Cropping Algorithms
https://openaccess.thecvf.com/content/WACV2022/papers/Birhane_Auditing_Saliency_Cropping_Algorithms_WACV_2022_paper.pdf
https://vinayprabhu.github.io/Saliency_Image_Cropping/paper_html/main.html
https://vinayprabhu.medium.com/on-the-twitter-cropping-controversy-critique-clarifications-and-comments-7ac66154f687
https://vinayprabhu.github.io/Saliency_Image_Cropping/
YourTTS: SOTA zero-shot Text-to-Speech
https://github.com/coqui-ai/TTS?utm_source=pocket_mylist
https://arxiv.org/abs/2112.02418?utm_source=pocket_mylist
https://coqui.ai/?utm_source=pocket_mylist
https://coqui.ai/blog/tts/yourtts-zero-shot-text-synthesis-low-resource-languages
MT3: Multi-Track Music Transcription
https://arxiv.org/abs/2111.03017
https://github.com/magenta/mt3
https://huggingface.co/spaces/akhaliq/MT3
https://www.reddit.com/r/MachineLearning/comments/rtlx0r/r_mt3_multitask_multitrack_music_transcription/
China regulates addictive algorithms
https://technode.com/2022/01/05/china-issues-new-rules-to-regulate-algorithms-targeting-addiction-monopolies-and-overspending/
https://qz.com/2109618/china-reveals-new-algorithm-rules-to-weaken-platforms-control-of-users/
A collection of Deep Learning interview questions & solutions
https://arxiv.org/abs/2201.00650?utm_source=pocket_mylist
https://arxiv.org/pdf/2201.00650.pdf
Helpful Things
https://docs.deepchecks.com/en/stable/index.html
https://github.com/deepchecks/deepchecks
https://docs.deepchecks.com/en/stable/examples/guides/quickstart_in_5_minutes.html
https://www.dagshub.com/
https://www.dagshub.com/docs/index.html
https://www.dagshub.com/blog/launching-dagshub-2-0/
https://bayesiancomputationbook.com/welcome.html
https://mlcontests.com/
https://github.com/Yard1/ray-skorch
https://github.com/skorch-dev/skorch
https://www.rumbledb.org/?utm_source=pocket_mylist
https://github.com/DarshanDeshpande/jax-models
https://github.com/s3prl/s3prl
AlphaZero explained blog post
https://joshvarty.github.io/AlphaZero/?utm_source=pocket_mylist
Ru-DOLPH: HyperModal Text-to-Image-to-Text model
https://github.com/sberbank-ai/ru-dolph
https://colab.research.google.com/drive/1gmTDA13u709OXiAeXWGm7sPixRhEJCga?usp=sharing
Google AI 2021 Review
https://ai.googleblog.com/2022/01/google-research-themes-from-2021-and.html
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n [ML News] ConvNeXt: Convolutions return | China regulates algorithms | Saliency cropping examined](https://i.ytimg.com/vi/yVKiMh2vEWQ/mqdefault.jpg)

![[Paper Analysis] On the Theoretical Limitations of Embedding-Based Retrieval (Warning: Rant)
Paper: https://arxiv.org/abs/2508.21038
Abstract:
Vector embeddings have been tasked with an ever-increasing set of retrieval tasks over the years, with a nascent rise in using them for reasoning, instruction-following, coding, and more. These new benchmarks push embeddings to work for any query and any notion of relevance that could be given. While prior works have pointed out theoretical limitations of vector embeddings, there is a common assumption that these difficulties are exclusively due to unrealistic queries, and those that are not can be overcome with better training data and larger models. In this work, we demonstrate that we may encounter these theoretical limitations in realistic settings with extremely simple queries. We connect known results in learning theory, showing that the number of top-k subsets of documents capable of being returned as the result of some query is limited by the dimension of the embedding. We empirically show that this holds true even if we restrict to k=2, and directly optimize on the test set with free parameterized embeddings. We then create a realistic dataset called LIMIT that stress tests models based on these theoretical results, and observe that even state-of-the-art models fail on this dataset despite the simple nature of the task. Our work shows the limits of embedding models under the existing single vector paradigm and calls for future research to develop methods that can resolve this fundamental limitation.
Authors: Orion Weller, Michael Boratko, Iftekhar Naim, Jinhyuk Lee
Links:
Homepage: https://ykilcher.com
Merch: https://ykilcher.com/merch
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
LinkedIn: https://www.linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n [Paper Analysis] On the Theoretical Limitations of Embedding-Based Retrieval (Warning: Rant)](https://i.ytimg.com/vi/zKohTkN0Fyk/mqdefault.jpg)



