Full Self-Driving is HARD! Analyzing Elon Musk re: Tesla Autopilot on Lex Fridmans Podcast Yannic Kilcher 2026-09-20 | Full Self-Driving is HARD! Analyzing Elon Musk re: Tesla Autopilot on Lex Fridmans Podcast
Traditional Holiday Live Stream Yannic Kilcher 2024-12-27 | ykilcher.com/discordLinks:TabNine Code Completion (Referral): http://bit.ly/tabnine-yannickYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: discord.gg/4H8xxDFBitChute: bitchute.com/channel/yannic-kilcherMinds: minds.com/ykilcherParler: parler.com/profile/YannicKilcherLinkedIn: linkedin.com/in/yannic-kilcher-488534136BiliBili: space.bilibili.com/1824646584If you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Byte Latent Transformer: Patches Scale Better Than Tokens (Paper Explained) Yannic Kilcher 2024-12-24 | #tokenization #llm #metaThis paper does away with tokenization and creates an LLM architecture that operates on dynamically sized "patches" instead of tokens. By controlling the patch size, they gain a level of control over the tradeoff between model size and FLOPs and use that to achieve more favorable scaling behavior than classically tokenized LLMs.Paper: ai.meta.com/research/publications/byte-latent-transformer-patches-scale-better-than-tokensCode: github.com/facebookresearch/bltAbstract:We introduce the Byte Latent Transformer (BLT), a new byte-level LLM architecture that, for the first time, matches tokenization-based LLM performance at scale with significant improvements in inference efficiency and robustness. BLT encodes bytes into dynamically sized patches, which serve as the primary units of computation. Patches are segmented dynamically based on the entropy of the next byte, allocating more compute and model capacity where increased data complexity demands it. We present the first flop controlled scaling study of byte-level models up to 8B parameters with 4T training bytes. Our results demonstrate the feasibility of scaling models trained on raw bytes without a fixed-vocabulary. Both training and inference efficiency improve due to dynamically selecting long patches when data is predictable, along with qualitative improvements on reasoning and long tail generalization. Overall, for fixed inference costs, BLT shows significantly better scaling than tokenization-based models, by simultaneously growing both patch and model size.Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Safety Alignment Should be Made More Than Just a Few Tokens Deep (Paper Explained) Yannic Kilcher 2024-12-10 | This paper demonstrates in a series of experiments that current safety alignment techniques of LLMs, as well as corresponding jailbreaking attacks, are in large part focusing on modulating the distribution of the first few tokens of the LLM response.Paper: openreview.net/forum?id=6Mxhg9PtDE&s=09Abstract:The safety alignment of current Large Language Models (LLMs) is vulnerable. Simple attacks, or even benign fine-tuning, can jailbreak aligned models. We note that many of these vulnerabilities are related to a shared underlying issue: safety alignment can take shortcuts, wherein the alignment adapts a model's generative distribution primarily over only its very first few output tokens. We unifiedly refer to this issue as shallow safety alignment. In this paper, we present case studies to explain why shallow safety alignment can exist and show how this issue universally contributes to multiple recently discovered vulnerabilities in LLMs, including the susceptibility to adversarial suffix attacks, prefilling attacks, decoding parameter attacks, and fine-tuning attacks. The key contribution of this work is that we demonstrate how this consolidated notion of shallow safety alignment sheds light on promising research directions for mitigating these vulnerabilities. We show that deepening the safety alignment beyond the first few tokens can meaningfully improve robustness against some common exploits. We also design a regularized fine-tuning objective that makes the safety alignment more persistent against fine-tuning attacks by constraining updates on initial tokens. Overall, we advocate that future safety alignment should be made more than just a few tokens deep.Authors: AnonymousLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters (Paper Explained) Yannic Kilcher 2024-11-23 | A deep dive into the TokenFormer and an opinion about its impact, novelty, and relation to prior work.Paper: arxiv.org/abs/2410.23168Abstract:Transformers have become the predominant architecture in foundation models due to their excellent performance across various domains. However, the substantial cost of scaling these models remains a significant concern. This problem arises primarily from their dependence on a fixed number of parameters within linear projections. When architectural modifications (e.g., channel dimensions) are introduced, the entire model typically requires retraining from scratch. As model sizes continue growing, this strategy results in increasingly high computational costs and becomes unsustainable. To overcome this problem, we introduce TokenFormer, a natively scalable architecture that leverages the attention mechanism not only for computations among input tokens but also for interactions between tokens and model parameters, thereby enhancing architectural flexibility. By treating model parameters as tokens, we replace all the linear projections in Transformers with our token-parameter attention layer, where input tokens act as queries and model parameters as keys and values. This reformulation allows for progressive and efficient scaling without necessitating retraining from scratch. Our model scales from 124M to 1.4B parameters by incrementally adding new key-value parameter pairs, achieving performance comparable to Transformers trained from scratch while greatly reducing training costs. Code and models are available at \url{this https URL}.Authors: Haiyang Wang, Yue Fan, Muhammad Ferjad Naeem, Yongqin Xian, Jan Eric Lenssen, Liwei Wang, Federico Tombari, Bernt SchieleLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models Yannic Kilcher 2024-10-19 | This paper (by Apple) questions the mathematical reasoning abilities of current LLMs and designs a synthetic template-based dataset distribution to investigate various aspects around LLM performance of high-school level math questions.Paper: arxiv.org/abs/2410.05229Abstract:Recent advancements in Large Language Models (LLMs) have sparked interest in their formal reasoning capabilities, particularly in mathematics. The GSM8K benchmark is widely used to assess the mathematical reasoning of models on grade-school-level questions. While the performance of LLMs on GSM8K has significantly improved in recent years, it remains unclear whether their mathematical reasoning capabilities have genuinely advanced, raising questions about the reliability of the reported metrics. To address these concerns, we conduct a large-scale study on several SOTA open and closed models. To overcome the limitations of existing evaluations, we introduce GSM-Symbolic, an improved benchmark created from symbolic templates that allow for the generation of a diverse set of questions. GSM-Symbolic enables more controllable evaluations, providing key insights and more reliable metrics for measuring the reasoning capabilities of this http URL findings reveal that LLMs exhibit noticeable variance when responding to different instantiations of the same question. Specifically, the performance of all models declines when only the numerical values in the question are altered in the GSM-Symbolic benchmark. Furthermore, we investigate the fragility of mathematical reasoning in these models and show that their performance significantly deteriorates as the number of clauses in a question increases. We hypothesize that this decline is because current LLMs cannot perform genuine logical reasoning; they replicate reasoning steps from their training data. Adding a single clause that seems relevant to the question causes significant performance drops (up to 65%) across all state-of-the-art models, even though the clause doesn't contribute to the reasoning chain needed for the final answer. Overall, our work offers a more nuanced understanding of LLMs' capabilities and limitations in mathematical reasoning.Authors: Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, Mehrdad FarajtabarLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Were RNNs All We Needed? (Paper Explained) Yannic Kilcher 2024-10-12 | This paper posits the interesting question: How much of the performance of Mamba, S4, and other state-space-like models is actually just attributable to some very core concepts - rather than their elaborate architectures. The authors construct minimal versions of GRUs and LSTMs and report competitive performance.Paper: arxiv.org/abs/2410.01201Abstract:The scalability limitations of Transformers regarding sequence length have renewed interest in recurrent sequence models that are parallelizable during training. As a result, many novel recurrent architectures, such as S4, Mamba, and Aaren, have been proposed that achieve comparable performance. In this work, we revisit traditional recurrent neural networks (RNNs) from over a decade ago: LSTMs (1997) and GRUs (2014). While these models were slow due to requiring to backpropagate through time (BPTT), we show that by removing their hidden state dependencies from their input, forget, and update gates, LSTMs and GRUs no longer need to BPTT and can be efficiently trained in parallel. Building on this, we introduce minimal versions (minLSTMs and minGRUs) that (1) use significantly fewer parameters than their traditional counterparts and (2) are fully parallelizable during training (175x faster for a sequence of length 512). Lastly, we show that these stripped-down versions of decade-old RNNs match the empirical performance of recent sequence models.Authors: Leo Feng, Frederick Tung, Mohamed Osama Ahmed, Yoshua Bengio, Hossein HajimirsadeghLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (Paper) Yannic Kilcher 2024-10-05 | How can one best use extra FLOPS at test time?Paper: arxiv.org/abs/2408.03314Abstract:Enabling LLMs to improve their outputs by using more test-time computation is a critical step towards building generally self-improving agents that can operate on open-ended natural language. In this paper, we study the scaling of inference-time computation in LLMs, with a focus on answering the question: if an LLM is allowed to use a fixed but non-trivial amount of inference-time compute, how much can it improve its performance on a challenging prompt? Answering this question has implications not only on the achievable performance of LLMs, but also on the future of LLM pretraining and how one should tradeoff inference-time and pre-training compute. Despite its importance, little research attempted to understand the scaling behaviors of various test-time inference methods. Moreover, current work largely provides negative results for a number of these strategies. In this work, we analyze two primary mechanisms to scale test-time computation: (1) searching against dense, process-based verifier reward models; and (2) updating the model's distribution over a response adaptively, given the prompt at test time. We find that in both cases, the effectiveness of different approaches to scaling test-time compute critically varies depending on the difficulty of the prompt. This observation motivates applying a "compute-optimal" scaling strategy, which acts to most effectively allocate test-time compute adaptively per prompt. Using this compute-optimal strategy, we can improve the efficiency of test-time compute scaling by more than 4x compared to a best-of-N baseline. Additionally, in a FLOPs-matched evaluation, we find that on problems where a smaller base model attains somewhat non-trivial success rates, test-time compute can be used to outperform a 14x larger model.Authors: Charlie Snell, Jaehoon Lee, Kelvin Xu, Aviral KumarLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Privacy Backdoors: Stealing Data with Corrupted Pretrained Models (Paper Explained) Yannic Kilcher 2024-08-04 | #llm #privacy #finetuning Can you tamper with a base model in such a way that it will exactly remember its fine-tuning data? This paper presents a method of doing exactly that, and implements it in modern transformers.OUTLINE:0:00 - Intro & Overview10:50 -Core idea: single-use data traps44:30 - Backdoors in transformer models58:00 - Additional numerical tricks1:00:35 - Experimental results & conclusionPaper: arxiv.org/abs/2404.00473Code: github.com/ShanglunFengatETHZ/PrivacyBackdoorAbstract:Practitioners commonly download pretrained machine learning models from open repositories and finetune them to fit specific applications. We show that this practice introduces a new risk of privacy backdoors. By tampering with a pretrained model's weights, an attacker can fully compromise the privacy of the finetuning data. We show how to build privacy backdoors for a variety of models, including transformers, which enable an attacker to reconstruct individual finetuning samples, with a guaranteed success! We further show that backdoored models allow for tight privacy attacks on models trained with differential privacy (DP). The common optimistic practice of training DP models with loose privacy guarantees is thus insecure if the model is not trusted. Overall, our work highlights a crucial and overlooked supply chain attack on machine learning privacy.Authors: Shanglun Feng, Florian TramèrLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Scalable MatMul-free Language Modeling (Paper Explained) Yannic Kilcher 2024-07-08 | Matrix multiplications (MatMuls) are pervasive throughout modern machine learning architectures. However, they are also very resource intensive and require special accelerators (GPUs). This paper explores architectures that do away with MatMuls and use quantization and recurrence to keep performance up.OUTLINE:0:00 - Intro2:30 - MatMul is everywhere5:55 - Ternary accumulation as a substitute for matrix multiplication16:35 - Replacing attention layers with recurrent layers32:40 - Replacing dense layers with ternary channel mixing38:30 - Language modelling results & scaling laws45:00 - Other experimental results48:20 - ConclusionPaper: arxiv.org/abs/2406.02528Code: github.com/ridgerchu/matmulfreellmAbstract:Matrix multiplication (MatMul) typically dominates the overall computational cost of large language models (LLMs). This cost only grows as LLMs scale to larger embedding dimensions and context lengths. In this work, we show that MatMul operations can be completely eliminated from LLMs while maintaining strong performance at billion-parameter scales. Our experiments show that our proposed MatMul-free models achieve performance on-par with state-of-the-art Transformers that require far more memory during inference at a scale up to at least 2.7B parameters. We investigate the scaling laws and find that the performance gap between our MatMul-free models and full precision Transformers narrows as the model size increases. We also provide a GPU-efficient implementation of this model which reduces memory usage by up to 61% over an unoptimized baseline during training. By utilizing an optimized kernel during inference, our model's memory consumption can be reduced by more than 10x compared to unoptimized models. To properly quantify the efficiency of our architecture, we build a custom hardware solution on an FPGA which exploits lightweight operations beyond what GPUs are capable of. We processed billion-parameter scale models at 13W beyond human readable throughput, moving LLMs closer to brain-like efficiency. This work not only shows how far LLMs can be stripped back while still performing effectively, but also points at the types of operations future accelerators should be optimized for in processing the next generation of lightweight LLMs. Our code implementation is available at this https URL.Authors: Rui-Jie Zhu, Yu Zhang, Ethan Sifferman, Tyler Sheaves, Yiqiao Wang, Dustin Richmond, Peng Zhou, Jason K. EshraghianLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (Paper Explained) Yannic Kilcher 2024-06-26 | #rag #hallucinations #legaltech An in-depth look at a recent Stanford paper examining the degree of hallucinations in various LegalTech tools that incorporate LLMs.OUTLINE:0:00 - Intro1:58 - What are legal research tools and how are large language models used by them?5:30 - Overview and abstract of the paper9:29 - What is a hallucination and why do they occur?15:45 - What is retrieval augmented generation (RAG)?25:00 - Why LLMs are a bad choice when reasoning is involved29:16 - The products that were tested32:00 - Some shady practices by the researchers in the back and forth with the legal research companies37:00 - Legal technology companies’ marketing claims to eliminate or solve hallucination risk45:27 - Researchers evaluation of RAG for legal and requirement to have specialized education to use the research tools55:27 - How the researchers propose to measure accuracy and the problems of measuring accuracy1:09:20 - Researchers conclusionPaper: arxiv.org/abs/2405.20362Abstract:Legal practice has witnessed a sharp rise in products incorporating artificial intelligence (AI). Such tools are designed to assist with a wide range of core legal tasks, from search and summarization of caselaw to document drafting. But the large language models used in these tools are prone to "hallucinate," or make up false information, making their use risky in high-stakes domains. Recently, certain legal research providers have touted methods such as retrieval-augmented generation (RAG) as "eliminating" (Casetext, 2023) or "avoid[ing]" hallucinations (Thomson Reuters, 2023), or guaranteeing "hallucination-free" legal citations (LexisNexis, 2023). Because of the closed nature of these systems, systematically assessing these claims is challenging. In this article, we design and report on the first preregistered empirical evaluation of AI-driven legal research tools. We demonstrate that the providers' claims are overstated. While hallucinations are reduced relative to general-purpose chatbots (GPT-4), we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time. We also document substantial differences between systems in responsiveness and accuracy. Our article makes four key contributions. It is the first to assess and report the performance of RAG-based proprietary legal AI tools. Second, it introduces a comprehensive, preregistered dataset for identifying and understanding vulnerabilities in these systems. Third, it proposes a clear typology for differentiating between hallucinations and accurate legal responses. Last, it provides evidence to inform the responsibilities of legal professionals in supervising and verifying AI outputs, which remains a central open question for the responsible integration of AI into law.Authors: Varun Magesh, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D. Manning, Daniel E. HoLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
xLSTM: Extended Long Short-Term Memory Yannic Kilcher 2024-06-01 | xLSTM is an architecture that combines the recurrency and constant memory requirement of LSTMs with the large-scale training of transformers and achieves impressive results.Paper: arxiv.org/abs/2405.04517Abstract:In the 1990s, the constant error carousel and gating were introduced as the central ideas of the Long Short-Term Memory (LSTM). Since then, LSTMs have stood the test of time and contributed to numerous deep learning success stories, in particular they constituted the first Large Language Models (LLMs). However, the advent of the Transformer technology with parallelizable self-attention at its core marked the dawn of a new era, outpacing LSTMs at scale. We now raise a simple question: How far do we get in language modeling when scaling LSTMs to billions of parameters, leveraging the latest techniques from modern LLMs, but mitigating known limitations of LSTMs? Firstly, we introduce exponential gating with appropriate normalization and stabilization techniques. Secondly, we modify the LSTM memory structure, obtaining: (i) sLSTM with a scalar memory, a scalar update, and new memory mixing, (ii) mLSTM that is fully parallelizable with a matrix memory and a covariance update rule. Integrating these LSTM extensions into residual block backbones yields xLSTM blocks that are then residually stacked into xLSTM architectures. Exponential gating and modified memory structures boost xLSTM capabilities to perform favorably when compared to state-of-the-art Transformers and State Space Models, both in performance and scaling.Authors: Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, Sepp HochreiterLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
[ML News] OpenAI is in hot waters (GPT-4o, Ilya Leaving, Scarlett Johansson legal action) Yannic Kilcher 2024-05-21 | #gpt4o #sky #scarlettjohansson After the release of their flagship model GPT-4o, OpenAI finds itself in multiple controversies and an exodus of senior personnel - notably Ilya SutskeverReferences:openai.com/index/gpt-4o-and-more-tools-to-chatgpt-freeopenai.com/index/hello-gpt-4ohttps://x.com/LiamFedus/status/1790064963966370209?t=rx2YBT9AdDdKPhI6dUH4zA&s=09https://x.com/lmsysorg/status/1790097588399779991?t=rx2YBT9AdDdKPhI6dUH4zA&s=09https://x.com/bindureddy/status/1790127425705120149?t=mMUBqFBRphx-bDuZ1j3mjQ&s=09openai.com/index/improvements-to-data-analysis-in-chatgptopenai.com/index/openai-and-reddit-partnershiphttps://archive.ph/jHlMmvox.com/future-perfect/2024/5/17/24158478/openai-departures-sam-altman-employees-chatgpt-releasehttps://x.com/soumithchintala/status/1791547776804831673?t=pKvy-PHndHFb4QBOpDBHFw&s=09https://x.com/sama/status/1791936857594581428?t=tM0Bi50VmbiIwCypiHS0Gg&s=09https://x.com/ilyasut/status/1790517455628198322?t=4Rb4lY401dfJRjQAF_H5Fw&s=09https://x.com/sama/status/1790518031640347056?t=fgL4bpi2oFwYQHykwIb6Lw&s=09https://x.com/janleike/status/1791498174659715494https://x.com/sama/status/1791543264090472660https://x.com/gdb/status/1791869138132218351?t=87L_tKgBpiFO7o8w_oKS4A&s=09openai.com/index/how-the-voices-for-chatgpt-were-chosenforbes.com/sites/roberthart/2024/05/20/openai-says-its-pulling-chatgpt-voice-sky-that-sounds-like-scarlett-johansson/?sh=593844605725https://x.com/BobbyAllyn/status/1792679435701014908/photo/1https://x.com/stclairashley/status/1792710045668630905?t=HR7-U3hsxhL6XYCXnINisw&s=09Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
ORPO: Monolithic Preference Optimization without Reference Model (Paper Explained) Yannic Kilcher 2024-05-01 | Paper: arxiv.org/abs/2403.07691Abstract:While recent preference alignment algorithms for language models have demonstrated promising results, supervised fine-tuning (SFT) remains imperative for achieving successful convergence. In this paper, we study the crucial role of SFT within the context of preference alignment, emphasizing that a minor penalty for the disfavored generation style is sufficient for preference-aligned SFT. Building on this foundation, we introduce a straightforward and innovative reference model-free monolithic odds ratio preference optimization algorithm, ORPO, eliminating the necessity for an additional preference alignment phase. We demonstrate, both empirically and theoretically, that the odds ratio is a sensible choice for contrasting favored and disfavored styles during SFT across the diverse sizes from 125M to 7B. Specifically, fine-tuning Phi-2 (2.7B), Llama-2 (7B), and Mistral (7B) with ORPO on the UltraFeedback alone surpasses the performance of state-of-the-art language models with more than 7B and 13B parameters: achieving up to 12.20% on AlpacaEval2.0 (Figure 1), 66.19% on IFEval (instruction-level loose, Table 6), and 7.32 in MT-Bench (Figure 12). We release code and model checkpoints for Mistral-ORPO-α (7B) and Mistral-ORPO-β (7B).Authors: Jiwoo Hong, Noah Lee, James ThorneLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
[ML News] Chips, Robots, and Models Yannic Kilcher 2024-04-30 | OUTLINE:0:00 - Intro0:19 - Our next-generation Meta Training and Inference Accelerator01:39 - ALOHA Unleashed03:10 - Apple Inks $50M Deal with Shutterstock for AI Training Data04:28 - OpenAI Researchers, Including Ally of Sutskever, Fired for Alleged Leaking05:01 - Adobe's Ethical Firefly AI was Trained on Midjourney Images05:52 - Trudeau announces $2.4billion for AI-related investments06:48 - RecurrentGemma: Moving Past Transformers for Efficient Open Language Models07:15 - CodeGemma - an official Google release for code LLMs07:24 - Mistral AI: Cheaper, Better, Faster, Stronger08:08 - Vezora/Mistral-22B-v0.109:00 - WizardLM-2, next generation state-of-the-art-LLM09:31 - Idefics2, the strongest Vision-Language-Model (VLM) below 10B!10:14 - BlinkDL/rwkv-6-world10:50 - Pile-T5: Trained T5 on the Pile11:35 - Model Card for Zephyr 141B-A39B12:42 - Parler TTS13:11 - RHO-1: Not all tokens are what you need14:59 - Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMsReferences:twitter.com/ayzwah/status/1780263768968273923ai.meta.com/blog/next-generation-meta-training-inference-accelerator-AI-MTIA/?utm_source=twittertwitter.com/soumithchintala/status/1778087952964374854?t=Mb-mQvm4YIZ35pVpEijs6g&s=09deepnewz.com/tech/apple-inks-50m-deal-shutterstock-ai-training-datatwitter.com/TolgaBilge_/status/1778598047821291793?t=zInlPDRZzozcz7-pjFSnyA&s=09twitter.com/javilopen/status/1778821749792034911?t=oGLiMj6GQdKTuM6GbiYrAg&s=09twitter.com/paulg/status/1781329523155357914?t=vCQT2mJf5BbtjdN1BMFYFQ&s=09twitter.com/RichardSocher/status/1776706907295846628cbc.ca/news/politics/federal-government-ai-investment-1.7166234arxiv.org/pdf/2404.07839huggingface.co/blog/codegemmamistral.ai/news/mixtral-8x22btwitter.com/MistralAILabs/status/1780606904273702932?t=JlSCcYulpJL74pNJbtSZag&s=09huggingface.co/Vezora/Mistral-22B-v0.1huggingface.co/Vezora/Mistral-22B-v0.2twitter.com/WizardLM_AI/status/1779899325868589372?t=l0Fd-4mfdtz3np_gALKaLA&s=09twitter.com/_philschmid/status/1779922877589889400?t=7q1xg1LRy80mV8JGRm4aqA&s=09huggingface.co/BlinkDL/rwkv-6-worldblog.eleuther.ai/pile-t5huggingface.co/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1huggingface.co/MaziyarPanahi/zephyr-orpo-141b-A35b-v0.1-GGUFtwitter.com/reach_vb/status/1778138382633140276?t=Mb-mQvm4YIZ35pVpEijs6g&s=09arxiv.org/pdf/2404.07965arxiv.org/pdf/2404.05719sambanova.ai/blog/samba-coe-the-power-of-routing-ml-models-at-scalemicrosoft.com/en-us/research/project/vasa-1twitter.com/twelve_labs/status/1780939765405065254?t=5ONxSzdwnghsKcwq3IPmEQ&s=09drive.google.com/file/d/1Av5jpsbH3g09TRD1PfRh0nLsYrN_iu7_/viewarxiv.org/pdf/2404.12387arxiv.org/abs/2404.12241arxiv.org/pdf/2404.12241twitter.com/Alon_Jacoby/status/1780650122382049596audiodialogues.github.ioos-world.github.ioai.meta.com/blog/openeqa-embodied-question-answering-robotics-ar-glasses/?utm_source=twitter&utm_medium=organic_social&utm_content=video&utm_campaign=datasetarxiv.org/pdf/2404.07503arxiv.org/pdf/2404.06654twitter.com/amanrsanger/status/1779620682340704386?t=UnOronFwkESwAXiE0i0R4A&s=09huggingface.co/datasets/xai-org/RealworldQAgithub.com/PygmalionAI/aphrodite-enginegithub.com/jina-ai/reader/?tab=readme-ov-filehttps://r.jina.ai/x.com/elonmuskhttps://r.jina.ai/github.com/jina-ai/readergithub.com/rogeriochaves/langstreamtwitter.com/mvpatel2000/status/1777891913313440215?t=m5POrtTTS33tgwmRztQj3w&s=09github.com/databricks/megablocksgithub.com/nus-apr/auto-code-rovergithub.com/nus-apr/auto-code-rover/blob/main/preprint.pdftwitter.com/karpathy/status/1683143097604243456?t=7V_ApJFbjrm4TbxM5n3nXA&s=09twitter.com/karpathy/status/1777427944971083809?t=s6xYQmYkhQyiFU65Fwq9tw&s=09github.com/BasedHardware/Friendtwitter.com/argmaxinc/status/1781382688819282132?t=vCQT2mJf5BbtjdN1BMFYFQ&s=09twitter.com/awnihannun/status/1778519566437794109?t=8N5PjwlKJpGotTx_HXZQrQ&s=09twitter.com/Prince_Canuma/status/1776399292036501898pytorch.org/blog/torchtune-fine-tune-llmsIf you want to support me, the best thing to do is to share out the content :)
TransformerFAM: Feedback attention is working memory Yannic Kilcher 2024-04-28 | Paper: arxiv.org/abs/2404.09173Abstract:While Transformers have revolutionized deep learning, their quadratic attention complexity hinders their ability to process infinitely long inputs. We propose Feedback Attention Memory (FAM), a novel Transformer architecture that leverages a feedback loop to enable the network to attend to its own latent representations. This design fosters the emergence of working memory within the Transformer, allowing it to process indefinitely long sequences. TransformerFAM requires no additional weights, enabling seamless integration with pre-trained models. Our experiments show that TransformerFAM significantly improves Transformer performance on long-context tasks across various model sizes (1B, 8B, and 24B). These results showcase the potential to empower Large Language Models (LLMs) to process sequences of unlimited length.Authors: Dongseong Hwang, Weiran Wang, Zhuoyuan Huo, Khe Chai Sim, Pedro Moreno MengibarLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
[ML News] Devin exposed | NeurIPS track for high school students Yannic Kilcher 2024-04-27 | OUTLINE:0:00 - Intro0:21 - Debunking Devin: "First AI Software Engineer" Upwork lie exposed!07:24 - NeurIPS 2024 will have a track for papers from high schoolers.13:29 - Opus can operate as a Turing machine.13:47 - An AI-Powered, Self-Running Propaganda Machine for $10514:27 - TechScape: How cheap, outsourced labour in Africa is shaping AI English16:25 - Is ChatGPT Transforming Academics' Writing Style?References:news.ycombinator.com/item?id=40008109&s=09youtube.com/watch?v=tNmgmwEtoWEyoutube.com/watch?v=xE2fxcETP5Etwitter.com/itsandrewgao/status/1779369373737668669?t=omW3DvRNmZyce8oo0Ehf1g&s=09twitter.com/0interestrates/status/1779268441226256500?t=tGwngUpChSD2YZ0VQDJHAA&s=09twitter.com/thegautamkamath/status/1778580754785550819?t=Qq1nLUIOyfRfBbZ6BHdXPw&s=09twitter.com/vipul_1011/status/1778619720964419930?t=225aakPnHb-ojIjveaWkkg&s=09twitter.com/avt_im/status/1778913195408626110?t=UPtduAKTX1uvq8Wa_EQOWg&s=09arxiv.org/pdf/2402.05120.pdftwitter.com/ctjlewis/status/1779740038852690393?t=AhIQM4rBUim-IWEkXL7OVQ&s=33wsj.com/politics/how-i-built-an-ai-powered-self-running-propaganda-machine-for-105-e9888705twitter.com/ylecun/status/1780728376283521191?t=rbTfUT7IWzXy83fvr-f4hw&s=09futureofhumanityinstitute.orggoogle.com/search?q=alex+hern+guardian+delve&oq=alex+hern+guardian+delve&gs_lcrp=EgZjaHJvbWUyBggAEEUYOTIHCAEQIRigATIHCAIQIRigATIHCAMQIRigATIHCAQQIRiPAtIBCDQ5NTVqMGo0qAIAsAIB&sourceid=chrome&ie=UTF-8theguardian.com/technology/2024/apr/16/techscape-ai-gadgest-humane-ai-pin-chatgptarxiv.org/pdf/2404.08627.pdfLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention Yannic Kilcher 2024-04-24 | Google researchers achieve supposedly infinite context attention via compressive memory.Paper: arxiv.org/abs/2404.07143Abstract:This work introduces an efficient method to scale Transformer-based Large Language Models (LLMs) to infinitely long inputs with bounded memory and computation. A key component in our proposed approach is a new attention technique dubbed Infini-attention. The Infini-attention incorporates a compressive memory into the vanilla attention mechanism and builds in both masked local attention and long-term linear attention mechanisms in a single Transformer block. We demonstrate the effectiveness of our approach on long-context language modeling benchmarks, 1M sequence length passkey context block retrieval and 500K length book summarization tasks with 1B and 8B LLMs. Our approach introduces minimal bounded memory parameters and enables fast streaming inference for LLMs.Authors: Tsendsuren Munkhdalai, Manaal Faruqui, Siddharth GopalLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
[ML News] Llama 3 changes the game Yannic Kilcher 2024-04-23 | Meta's Llama 3 is out. New model, new license, new opportunities.References:llama.meta.com/llama3ai.meta.com/blog/meta-llama-3github.com/meta-llama/llama3/blob/main/MODEL_CARD.mdllama.meta.com/trust-and-safetyai.meta.com/research/publications/cyberseceval-2-a-wide-ranging-cybersecurity-evaluation-suite-for-large-language-modelsgithub.com/meta-llama/llama-recipes/tree/main/recipes/responsible_aillama.meta.com/llama3/licenseabout.fb.com/news/2024/04/meta-ai-assistant-built-with-llama-3/?utm_source=twitter&utm_medium=organic_social&utm_content=thread&utm_campaign=imagineflashtwitter.com/minchoi/status/1782775792298037639?t=6U7Ob9P0SQmYdyLGUGq0Kg&s=09twitter.com/_akhaliq/status/1782607138952499661?t=osENiISXOhJEf89b9QAjSA&s=09twitter.com/_philschmid/status/1782420712105357616?t=vQQt7O9abWazZ-R3k3l9Kg&s=09twitter.com/lmsysorg/status/1782483699449332144?t=h1EdrbrXi0_03gXXbhXskw&s=09twitter.com/SebastienBubeck/status/1782627991874678809?t=QvZngdG1k0TllAyzT0qAsg&s=09twitter.com/_Mira___Mira_/status/1782595759726354485?t=QvZngdG1k0TllAyzT0qAsg&s=09twitter.com/_philschmid/status/1782358903558205556?t=h1EdrbrXi0_03gXXbhXskw&s=09twitter.com/cHHillee/status/1781060345366503527?t=5ONxSzdwnghsKcwq3IPmEQ&s=09meta.ai/?icebreaker=imaginetwitter.com/OpenAI/status/1777772582680301665?t=DKDx-qwUP3Xr4oFvAM9mOQ&s=09twitter.com/OpenAIDevs/status/1780640119890047475?t=YOJFQ6Ysx7JVDfZ6o3TT6A&s=09twitter.com/OpenAIDevs/status/1779922566091522492?t=KhlVzoXh3NjCld1JiobsTw&s=09twitter.com/CodeByPoonam/status/1776902550811525146?t=3cK96YjTWJnY0RmHLwAPsg&s=09twitter.com/hey_madni/status/1776950057801236933?t=P2x2bXrYgMHm8jX7k2CAaQ&s=09cloud.google.com/blog/products/ai-machine-learning/google-cloud-gemini-image-2-and-mlops-updatestwitter.com/altryne/status/1778522661070475586?t=jdDna4B-45yLez12yuElig&s=09twitter.com/xenovacom/status/1778812177215881395?t=oGLiMj6GQdKTuM6GbiYrAg&s=09twitter.com/minchoi/status/1778074187778683253?t=Mb-mQvm4YIZ35pVpEijs6g&s=09udio.comudio.com/pricingLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Hugging Face got hacked Yannic Kilcher 2024-04-17 | Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
[ML News] Microsoft to spend 100 BILLION DOLLARS on supercomputer (& more industry news) Yannic Kilcher 2024-04-15 | Some updates from industry in the Machine Learning worldLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
[ML News] Jamba, CMD-R+, and other new models (yes, I know this is like a week behind 🙃) Yannic Kilcher 2024-04-13 | A flurry of new models continues to appear.Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Flow Matching for Generative Modeling (Paper Explained) Yannic Kilcher 2024-04-08 | Flow matching is a more general method than diffusion and serves as the basis for models like Stable Diffusion 3.Paper: arxiv.org/abs/2210.02747Abstract:We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. Specifically, we present the notion of Flow Matching (FM), a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths. Flow Matching is compatible with a general family of Gaussian probability paths for transforming between noise and data samples -- which subsumes existing diffusion paths as specific instances. Interestingly, we find that employing FM with diffusion paths results in a more robust and stable alternative for training diffusion models. Furthermore, Flow Matching opens the door to training CNFs with other, non-diffusion probability paths. An instance of particular interest is using Optimal Transport (OT) displacement interpolation to define the conditional probability paths. These paths are more efficient than diffusion paths, provide faster training and sampling, and result in better generalization. Training CNFs using Flow Matching on ImageNet leads to consistently better performance than alternative diffusion-based methods in terms of both likelihood and sample quality, and allows fast and reliable sample generation using off-the-shelf numerical ODE solvers.Authors: Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, Matt LeLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping (Searchformer) Yannic Kilcher 2024-04-06 | Paper: arxiv.org/abs/2402.14083Abstract:While Transformers have enabled tremendous progress in various application settings, such architectures still lag behind traditional symbolic planners for solving complex decision making tasks. In this work, we demonstrate how to train Transformers to solve complex planning tasks and present Searchformer, a Transformer model that optimally solves previously unseen Sokoban puzzles 93.7% of the time, while using up to 26.8% fewer search steps than standard A∗ search. Searchformer is an encoder-decoder Transformer model trained to predict the search dynamics of A∗. This model is then fine-tuned via expert iterations to perform fewer search steps than A∗ search while still generating an optimal plan. In our training method, A∗'s search dynamics are expressed as a token sequence outlining when task states are added and removed into the search tree during symbolic planning. In our ablation studies on maze navigation, we find that Searchformer significantly outperforms baselines that predict the optimal plan directly with a 5-10× smaller model size and a 10× smaller training dataset. We also demonstrate how Searchformer scales to larger and more complex decision making tasks like Sokoban with improved percentage of solved tasks and shortened search dynamics.Authors: Lucas Lehnert, Sainbayar Sukhbaatar, Paul Mcvay, Michael Rabbat, Yuandong TianLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
[ML News] Grok-1 open-sourced | Nvidia GTC | OpenAI leaks model names | AI Act Yannic Kilcher 2024-03-26 | OUTLINE:0:00 - Intro0:15 - XAI releases Grok-12:00 - Nvidia GTC4:45 - Comment of the Week5:35 - Brute-forcing OpenAI model names7:30 - Inflection AI gets eaten by Microsoft9:25 - EU AI Act moving forward11:45 - Advances in Robotics14:00 - India retracts controversial advisory14:30 - OpenSora15:20 - Improved Gemma fine-tuning16:20 - Decoding encrypted LLM traffic17:45 - VariaReferences:https://x.ai/blog/grok-osgithub.com/xai-org/grok-1finance.yahoo.com/news/nvidia-debuts-next-generation-blackwell-ai-chip-at-gtc-2024-205825161.html?guccounter=1&guce_referrer=aHR0cHM6Ly9uZXdzLmdvb2dsZS5jb20v&guce_referrer_sig=AQAAAHYRVePPrDnH3HxPV8smDzUiia_ztWttteAmHKxy-x_Z75lqq2trR4Exwq2sFyjNQojO_95xWvqQFHkV3NI_IKmw9W8XZ7d52qBsdvqaDRkdNzBSzQhnskzUE_E-nDo6OFG0LmrM0ygvjqLgJyhMDnraaGHrUsb98kknjn7-83MJspectrum.ieee.org/nvidia-gr00t-rostwitter.com/anshelsag/status/1769989302552031473?t=DYAFhri4cu55LMwJV4V99A&s=09twitter.com/ibab_ml/status/1769770983924142475twitter.com/arthurmensch/status/1769842867621581299?t=sYPy011kN9KxzdnA11M4yQ&s=09twitter.com/arithmoquine/status/1770136393563378082?t=FgH3-TABR73QVUQuP5wq2g&s=09https://files.catbox.moe/od9pyb.txttechcrunch.com/2024/03/19/after-raising-1-3b-inflection-got-eaten-alive-by-its-biggest-investor-microsofthttps://archive.ph/p4W1N#selection-2463.23-2463.114instagram.com/reel/C4df3DZg1wj/?igsh=MWQ1ZGUxMzBkMA%3D%3Dtechcrunch.com/2024/03/15/mercedes-begins-piloting-apptronik-humanoid-robotsaxios.com/2024/03/14/humanoid-robot-army-agility-digit-amazon-warehousetechcrunch.com/2024/03/15/india-drops-plan-to-require-approval-for-ai-model-launchesgithub.com/hpcaitech/Open-Sorareddit.com/r/LocalLLaMA/comments/1bd18y8/gemma_finetuning_should_be_much_better_nowtwitter.com/felix_red_panda/status/1769363356094230837?t=JMMb3OldqfhhCH8X5e7ljA&s=09twitter.com/imaurer/status/1768386949201408103twitter.com/ollama/status/1768415114724819060?t=Q7opDnL4_anatuoXzATBng&s=09arxiv.org/pdf/2403.09611.pdfgithub.com/lavague-ai/LaVaguehttps://blog.research.google/2024/03/chain-of-table-evolving-tables-in.htmlcnbc.com/2024/03/18/apple-in-talks-to-license-googles-gemini-for-generative-ai-bloomberg.htmlhttps://blog.google/products/search/google-search-update-march-2024/stability.ai/news/introducing-stable-video-3dtwitter.com/Nils_Reimers/status/1769809006762037368?t=XmqKGm1ycjvsz2HAWA6WzQ&s=09Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
[ML News] Devin AI Software Engineer | GPT-4.5-Turbo LEAKED | US Govt Report: Total Extinction Yannic Kilcher 2024-03-17 | Your weekly dose of ML NewsOUTLINE:0:00 - Intro0:15 - Devin: AI software engineer5:50 - Mira Murati on Sora training data6:50 - Inflection accused of copying Claude9:00 - Tools & papers16:30 - GPT-4.5-turbo mystery17:30 - US government report: total extinction by AI19:20 - Various other newsReferences:cognition-labs.com/introducing-devintwitter.com/cognition_labs/status/1767548763134964000?t=ZECIn-uqbguwHtY8X_Gvtw&s=09news.google.com/stories/CAAqNggKIjBDQklTSGpvSmMzUnZjbmt0TXpZd1NoRUtEd2lWMUwyU0N4RnVWM3pSRWhWX01pZ0FQAQ?hl=en-US&gl=US&ceid=US%3Aenbloomberg.com/news/articles/2024-03-12/cognition-ai-is-a-peter-thiel-backed-coding-assistant?embedded-checkout=truebloomberg.com/authors/AQWHkoPod9g/ashlee-vancebloomberg.com/news/articles/2024-03-12/cognition-ai-is-a-peter-thiel-backed-coding-assistant?srnd=undefined&embedded-checkout=truebloomberg.com/news/newsletters/2024-03-12/cognition-ai-s-devin-assistant-can-build-websites-videos-from-a-prompt?srnd=undefined&embedded-checkout=truehttps://archive.ph/5LZV9github.com/opendevin/opendevintwitter.com/MetaGPT_/status/1767965444579692832?t=dsYKmPfOBVGCFCwvPtZVWQ&s=09docs.deepwisdom.ai/main/en/DataInterpreter/detail.html?id=AppleStockPriceAnalysisAndPredictiondocs.deepwisdom.ai/main/en/guide/use_cases/agent/interpreter/intro.htmlgithub.com/geekan/MetaGPT/tree/main/examples/diinflection.ai/inflection-2-5twitter.com/seshubon/status/1765870717844050221twitter.com/inflectionAI/status/1766173427441049684mlxserver.comhuggingface.co/spaces/mlabonne/AutoMergergithub.com/microsoft/aicigithub.com/google-research/google-research/tree/master/faxgithub.com/stanfordnlp/pyvenearxiv.org/pdf/2403.06634.pdftwitter.com/mattshumer_/status/1767606938538295757?t=1dYect5ylg9xrWSS4sL38Q&s=09time.com/6898967/ai-extinction-national-security-risks-reportventurebeat.com/ai/hugging-face-is-launching-an-open-source-robotics-project-led-by-former-tesla-scientisttwitter.com/gcabanac/status/1767574447337124290?t=MnzwEbf_Zx0yQthe0RQ8hw&s=09twitter.com/AnthropicAI/status/1768018310615151002?t=3ieMvNZxaoTXGGZttBsBvQ&s=09huggingface.co/CohereForAI/c4ai-command-r-v01twitter.com/Yampeleg/status/1765707714473197729?t=p3zOXUqKdqS-RzYjTNo65g&s=09huggingface.co/yam-peleg/Hebrew-Gemma-11Benriccorona.github.io/vloggerhuggingface.co/NousResearch/Genstruct-7Bhttps://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/arxiv.org/abs/2403.04652twitter.com/corry_wang/status/1766949316394897851?t=i0ndsef_I_b3BDkVmyHgYw&s=09twitter.com/sama/status/1766291001134715207?t=Wgyye9odOfF1Aoo0hZGihg&s=09venturebeat.com/ai/nist-staffers-revolt-against-potential-appointment-of-effective-altruist-ai-researcher-to-us-ai-safety-instituteocciglot.github.io/occiglot/posts/occiglot-announcementtwitter.com/EMostaque/status/1767199048337932719?t=tYB3KeabfLlB90XhUX0R7A&s=09Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
[ML News] Elon sues OpenAI | Mistral Large | More Gemini Drama Yannic Kilcher 2024-03-10 | #mlnews #ainews #openai OUTLINE:0:00 - Intro0:20 - Elon sues OpenAI14:00 - Mistral Large16:40 - ML Espionage18:30 - More Gemini Drama24:00 - Copilot generates spicy images26:55 - Gemma bugs28:45 - VariaReferences: gist.github.com/yk/0c065cdc8e414738abfaae4f8e417e00Thumbnail pictures: WikipediaLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
No, Anthropics Claude 3 is NOT sentient Yannic Kilcher 2024-03-05 | No, Anthropic's Claude 3 is not conscious or sentient or self-aware.References:anthropic.com/news/claude-3-familytwitter.com/_akhaliq/status/1764673955313459560?t=gkBx2uTXfrxLl-5_mL7Btg&s=09twitter.com/idavidrein/status/1764675668175094169?t=pJfbN3LtKaxsU8egz83Mvg&s=09twitter.com/TolgaBilge_/status/1764754012824314102?t=9bakXDnVMC1oAEyZFoKimA&s=09twitter.com/karinanguyen_/status/1764670019743690757?t=gkBx2uTXfrxLl-5_mL7Btg&s=09twitter.com/alexalbert__/status/1764722513014329620lesswrong.com/posts/pc8uP4S9rDoNpwJDZ/claude-3-claims-its-consciousLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
[ML News] Groq, Gemma, Sora, Gemini, and Air Canadas chatbot troubles Yannic Kilcher 2024-03-01 | Your dose of ML News!OUTLINE:0:00 - Intro0:20 - Gemma & Gemini3:40 - Groq6:30 - Nvidia EOS Supercomputer7:15 - Gpulist.ai8:20 - Demis Hassabis on scale10:10 - Hardware wars12:05 - Sora15:10 - Gemini 1.5 Pro & Long Context18:45 - Air Canada must pay for chatbot mistake23:30 - Giant Rat Balls26:25 - Various NewsReferences:https://blog.google/technology/developers/gemma-open-models/?utm_source=twtwitter.com/altryne/status/1760358916624719938?t=PVZkHQA_p7GxmeUX0hcZ_Q&s=09twitter.com/paulg/status/1760078920135872716?t=PVZkHQA_p7GxmeUX0hcZ_Q&s=09groq.comtwitter.com/mattshumer_/status/1759347920543834117?t=cS5nPvZOsV6iDA1mVabHOg&s=09twitter.com/GroqInc/status/1759483896322781584wow.groq.com/news_press/groq-lpu-inference-engine-leads-in-first-independent-llm-benchmarktwitter.com/tianle_cai/status/1759780363361251828?t=SobcZzLkKufAhKaSK56DoA&s=09twitter.com/DZhang50/status/1759728119005712837twitter.com/felix_red_panda/status/1759720197055791188twitter.com/cHHillee/status/1759704303810519271twitter.com/mascobot/status/1759709223276228825techpowerup.com/319172/nvidia-unveils-eos-to-public-a-top-ten-supercomputerandromeda.aigpulist.aihttps://archive.ph/G6POitomshardware.com/tech-industry/artificial-intelligence/jim-keller-responds-to-sam-altmans-plan-to-raise-dollar7-billion-to-make-ai-chipsfuturism.com/the-byte/ai-destroy-humankind-yudkowskytwitter.com/_akhaliq/status/1758197872716026209?t=P6KPJIJ4Xxr82oMkh_Hd3w&s=09twitter.com/_Borriss_/status/1758206358376050822?t=drmW5Qzs7OuEaV_00uSqHQ&s=09twitter.com/billpeeb/status/1758650919430848991twitter.com/tsarnick/status/1758323312483303443?t=SmELRZbMIH_1hfx-T4RNHA&s=09twitter.com/MartinNebelong/status/1758431263193543080?t=do6FAkgZL8qpblevr8uxeQ&s=09twitter.com/OriolVinyalsML/status/1758148444588319020?t=K2RYfqbLuBvP-viCaPyC-Q&s=09twitter.com/mattshumer_/status/1759804492919275555twitter.com/mattshumer_/status/1759737197710704939?t=nt1vJygjHAUm5ad37MOfOQ&s=09twitter.com/haoliuhl/status/1757828392362389999?t=YnOsR_O52b2D7R3EW-A-0w&s=09github.com/lucidrains/ring-attention-pytorchbc.ctvnews.ca/air-canada-s-chatbot-gave-a-b-c-man-the-wrong-information-now-the-airline-has-to-pay-for-the-mistake-1.6769454arstechnica.com/tech-policy/2024/02/air-canada-must-honor-refund-policy-invented-by-airlines-chatbotcbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416twitter.com/kareem_carr/status/1758148011245371627?t=KLGIAEy0_dXst8WBkPs7KA&s=09arstechnica.com/science/2024/02/scientists-aghast-at-bizarre-ai-rat-with-huge-genitals-in-peer-reviewed-articlevice.com/en/article/4a389b/ai-midjourney-rat-penis-study-retracted-frontierstwitter.com/karpathy/status/1757600075281547344?t=8Sdkzge2gw3NOEjJL1BfaA&s=09twitter.com/karpathy/status/1759996549109776702?t=KCmoCq5o6nTrlagAKn5-JQ&s=09nature.com/articles/d41586-024-00497-8twitter.com/CohereForAI/status/1757359611399532921cohere.com/research/ayatwitter.com/OfirPress/status/1759571109513416768?t=XIXOgbTucAlb7xLq7eLo8Q&s=09github.com/mut-ex/gligen-guiprojectaria.com/datasets/aeatwitter.com/StabilityAI/status/1760656767237656820?t=cPicZ5BVMiNzAS0f7yB79A&s=09twitter.com/gordic_aleksa/status/1760692060439634041?t=5ANBNrMLrU0byDGQtFOhgw&s=09huggingface.co/gordicaleksa/YugoGPThuggingface.co/datasets/nvidia/OpenMathInstruct-1/blob/main/LICENSEinterestingengineering.com/science/ai-spots-dangerous-drug-comboshttps://archive.ph/caW1Y#selection-4955.105-4955.142newatlas.com/robotics/seeing-eye-dog-v2-gamechangeros-copilot.github.iobusinessinsider.com/apple-reportedly-plans-to-launch-an-ai-tool-this-year-2024-2?r=US&IR=Ttechcrunch.com/2024/02/16/anthropic-takes-steps-to-prevent-election-misinformation/?guccounter=1https://archive.ph/GbcgbLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Gemini has a Diversity Problem Yannic Kilcher 2024-02-22 | Google turned the anti-bias dial up to 11 on their new Gemini Pro model.References:developers.googleblog.com/2024/02/gemini-15-available-for-private-preview-in-google-ai-studio.htmlhttps://blog.google/technology/developers/gemma-open-models/?utm_source=twstorage.googleapis.com/deepmind-media/gemma/gemma-report.pdftwitter.com/ClementDelangue/status/1760324815888486668?t=spXd7Oq_cSrRN2A-3r6gnQ&s=09twitter.com/paulg/status/1760078920135872716?t=PVZkHQA_p7GxmeUX0hcZ_Q&s=09twitter.com/yoavgo/status/1760445342691016811/photo/3twitter.com/alex_peys/status/1760327435890135279/photo/2twitter.com/woke8yearold/status/1760310705142558781/photo/1twitter.com/stratejake/status/1760333904857497650?t=Z3BZOBaLI1EYAJ-CBAMNEg&s=09twitter.com/JohnLu0x/status/1760066875583816003?t=Z3BZOBaLI1EYAJ-CBAMNEg&s=09twitter.com/IMAO_/status/1760093853430710557?t=0eNmoTuvYZl9HQRaUBOKNw&s=09twitter.com/WallStreetSilv/status/1760474958151426340?t=6k4VwKFvciw2VoDc70Tl2A&s=09twitter.com/JackK/status/1760334258722250785twitter.com/TRHLofficial/status/1760485063941149100?t=hx48DQd64JbVxZ3OzhD0wg&s=09twitter.com/gordic_aleksa/status/1760266452475494828?t=VZ2lX_v-KrY4Thu4FvDh4w&s=09twitter.com/benthompson/status/1760452419627233610?t=qR9D9KDC1axOx3gDBKKc2Q&s=09twitter.com/altryne/status/1760358916624719938?t=PVZkHQA_p7GxmeUX0hcZ_Q&s=09twitter.com/pmarca/status/1760503344035180601?t=6k4VwKFvciw2VoDc70Tl2A&s=09Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
V-JEPA: Revisiting Feature Prediction for Learning Visual Representations from Video (Explained) Yannic Kilcher 2024-02-19 | #vjepa #meta #unsupervisedlearning V-JEPA is a method for unsupervised representation learning of video data by using only latent representation prediction as objective function.Weights & Biases course on Structured LLM Outputs: wandb.me/course-yannicOUTLINE:0:00 - Intro1:45 - Predictive Feature Principle8:00 - Weights & Biases course on Structured LLM Outputs9:45 - The original JEPA architecture27:30 - V-JEPA Concept33:15 - V-JEPA Architecture44:30 - Experimental Results46:30 - Qualitative Evaluation via DecodingBlog: ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-joint-embedding-predictive-architecturePaper: ai.meta.com/research/publications/revisiting-feature-prediction-for-learning-visual-representations-from-videoAbstract:This paper explores feature prediction as a stand-alone objective for unsupervised learning from video and introduces V-JEPA, a collection of vision models trained solely using a feature prediction objective, without the use of pretrained image encoders, text, negative examples, reconstruction, or other sources of supervision. The models are trained on 2 million videos collected from public datasets and are evaluated on downstream image and video tasks. Our results show that learning by predicting video features leads to versatile visual representations that perform well on both motion and appearance-based tasks, without adaption of the model’s parameters; e.g., using a frozen backbone, our largest model, a ViT-H/16 trained only on videos, obtains 81.9% on Kinetics-400, 72.2% on Something-Something-v2, and 77.9% on ImageNet1K.Authors: Adrien Bardes Quentin Garrido Xinlei Chen Michael Rabbat Yann LeCun Mido Assran Nicolas Ballas Jean PonceLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
What a day in AI! (Sora, Gemini 1.5, V-JEPA, and lots of news) Yannic Kilcher 2024-02-18 | Your regularly irregular dose of Machine Learning News!W&B Course on LLM Structured Outputs: wandb.me/course-yannicOUTLINE:0:00 - OpenAI Sora3:25 - Gemini 1.5 with 1 Million Tokens context window4:50 - V-JEPA6:50 - Sam Altman raises 7 TRILLION dollars for AI chips9:30 - Sponsor: Weights & Biases course on Structure Output from LLMs11:30 - Bard becomes Gemini13:55 - GOODY-2: The world's most responsible model16:05 - miqu-1-70b leaked from Mistral18:25 - Zuckerberg on Meta's open approach to AI models21:40 - 1X advances robotics23:30 - Questions around Bard's arena leaderboard position27:00 - Various other newsReferences:gist.github.com/yk/65fe3d582a43540a61718b9e4b0706d0(they were too long for this description)Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Lumiere: A Space-Time Diffusion Model for Video Generation (Paper Explained) Yannic Kilcher 2024-02-04 | #lumiere #texttovideoai #google LUMIERE by Google Research tackles globally consistent text-to-video generation by extending the U-Net downsampling concept to the temporal axis of videos.OUTLINE:0:00 - Introduction8:20 - Problems with keyframes16:55 - Space-Time U-Net (STUNet)21:20 - Extending U-Nets to video37:20 - Multidiffusion for SSR prediction fusing44:00 - Stylized generation by swapping weights49:15 - Training & Evaluation53:20 - Societal Impact & ConclusionPaper: arxiv.org/abs/2401.12945Website: lumiere-video.github.ioAbstract:We introduce Lumiere -- a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion -- a pivotal challenge in video synthesis. To this end, we introduce a Space-Time U-Net architecture that generates the entire temporal duration of the video at once, through a single pass in the model. This is in contrast to existing video models which synthesize distant keyframes followed by temporal super-resolution -- an approach that inherently makes global temporal consistency difficult to achieve. By deploying both spatial and (importantly) temporal down- and up-sampling and leveraging a pre-trained text-to-image diffusion model, our model learns to directly generate a full-frame-rate, low-resolution video by processing it in multiple space-time scales. We demonstrate state-of-the-art text-to-video generation results, and show that our design easily facilitates a wide range of content creation tasks and video editing applications, including image-to-video, video inpainting, and stylized generation.Authors: Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, Inbar MosseriLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
AlphaGeometry: Solving olympiad geometry without human demonstrations (Paper Explained) Yannic Kilcher 2024-01-21 | #deepmind #alphageometry #llm AlphaGeometry is a combination of a symbolic solver and a large language model by Google DeepMind that tackles IMO geometry questions without any human-generated trainind data.OUTLINE:0:00 - Introduction1:30 - Problem Statement7:30 - Core Contribution: Synthetic Data Generation9:30 - Sampling Premises13:00 - Symbolic Deduction17:00 - Traceback19:00 - Auxiliary Construction25:20 - Experimental Results32:00 - Problem Representation34:30 - Final CommentsPaper: nature.com/articles/s41586-023-06747-5Abstract:Proving mathematical theorems at the olympiad level represents a notable milestone in human-level automated reasoning1,2,3,4, owing to their reputed difficulty among the world’s best talents in pre-university mathematics. Current machine-learning approaches, however, are not applicable to most mathematical domains owing to the high cost of translating human proofs into machine-verifiable format. The problem is even worse for geometry because of its unique translation challenges1,5, resulting in severe scarcity of training data. We propose AlphaGeometry, a theorem prover for Euclidean plane geometry that sidesteps the need for human demonstrations by synthesizing millions of theorems and proofs across different levels of complexity. AlphaGeometry is a neuro-symbolic system that uses a neural language model, trained from scratch on our large-scale synthetic data, to guide a symbolic deduction engine through infinite branching points in challenging problems. On a test set of 30 latest olympiad-level problems, AlphaGeometry solves 25, outperforming the previous best method that only solves ten problems and approaching the performance of an average International Mathematical Olympiad (IMO) gold medallist. Notably, AlphaGeometry produces human-readable proofs, solves all geometry problems in the IMO 2000 and 2015 under human expert evaluation and discovers a generalized version of a translated IMO theorem in 2004.Authors: Trieu H. Trinh, Yuhuai Wu, Quoc V. Le, He He & Thang Luong Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Mixtral of Experts (Paper Explained) Yannic Kilcher 2024-01-13 | #mixtral #mistral #chatgpt OUTLINE:0:00 - Introduction3:00 - Mixture of Experts6:00 - Classic Transformer Blocks11:15 - Expert Routing17:00 - Sparse Expert Routing22:00 - Expert Parallelism25:00 - Experimental Results31:30 - Routing Analysis33:20 - ConclusionPaper: arxiv.org/abs/2401.04088Abstract:We introduce Mixtral 8x7B, a Sparse Mixture of Experts (SMoE) language model. Mixtral has the same architecture as Mistral 7B, with the difference that each layer is composed of 8 feedforward blocks (i.e. experts). For every token, at each layer, a router network selects two experts to process the current state and combine their outputs. Even though each token only sees two experts, the selected experts can be different at each timestep. As a result, each token has access to 47B parameters, but only uses 13B active parameters during inference. Mixtral was trained with a context size of 32k tokens and it outperforms or matches Llama 2 70B and GPT-3.5 across all evaluated benchmarks. In particular, Mixtral vastly outperforms Llama 2 70B on mathematics, code generation, and multilingual benchmarks. We also provide a model fine-tuned to follow instructions, Mixtral 8x7B - Instruct, that surpasses GPT-3.5 Turbo, Claude-2.1, Gemini Pro, and Llama 2 70B - chat model on human benchmarks. Both the base and instruct models are released under the Apache 2.0 license.Authors: Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El SayedLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Until the Litter End Yannic Kilcher 2024-01-10 | litter.ykilcher.comLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
LLaMA Pro: Progressive LLaMA with Block Expansion (Paper Explained) Yannic Kilcher 2024-01-07 | Note: The H800 is a variant of the H100 for the Chinese marketOUTLINE:0:00 - Introduction5:30 - Adding new blocks to LLaMA15:00 - Block expansion27:40 - Experiments30:40 - ConclusionPaper: arxiv.org/abs/2401.02415Other Paper: https://proceedings.mlr.press/v162/shen22f/shen22f.pdfAbstract:Humans generally acquire new skills without compromising the old; however, the opposite holds for Large Language Models (LLMs), e.g., from LLaMA to CodeLLaMA. To this end, we propose a new post-pretraining method for LLMs with an expansion of Transformer blocks. We tune the expanded blocks using only new corpus, efficiently and effectively improving the model's knowledge without catastrophic forgetting. In this paper, we experiment on the corpus of code and math, yielding LLaMA Pro-8.3B, a versatile foundation model initialized from LLaMA2-7B, excelling in general tasks, programming, and mathematics. LLaMA Pro and its instruction-following counterpart (LLaMA Pro-Instruct) achieve advanced performance among various benchmarks, demonstrating superiority over existing open models in the LLaMA family and the immense potential of reasoning and addressing diverse tasks as an intelligent agent. Our findings provide valuable insights into integrating natural and programming languages, laying a solid foundation for developing advanced language agents that operate effectively in various environments.Authors: Chengyue Wu, Yukang Gan, Yixiao Ge, Zeyu Lu, Jiahao Wang, Ye Feng, Ping Luo, Ying ShanLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
I created an AI-powered Social Network Yannic Kilcher 2024-01-02 | #ai #chatgpt #socialmedia I created a social network that operates entirely in the latent space.Litter (aka Latent Twitter) will pull images and text through multiple modality conversions before it hits the network, so you can communicate just the essence of your message.Website: litter.ykilcher.comCode: github.com/yk/litterOUTLINE:0:00 - Introduction1:10 - How does it work?3:30 - Improving Yann LeCun's post4:20 - Posting images5:05 - Image examples6:40 - Final wordsLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
NeurIPS 2023 Poster Session 4 (Thursday Morning) Yannic Kilcher 2023-12-26 | OUTLINE:0:30 - Activity Grammars for Temporal Action Segmentation8:50 - Diffusion-TTA: Test-time Adaptation of Discriminative Models via Generative Feedback17:05 - On the Role of Noise in the Sample Complexity of Learning Recurrent Neural Networks: Exponential Gaps for Long Sequences21:20 - Sketching Algorithms for Sparse Dictionary Learning: PTAS and Turnstile Streaming27:10 - Equivariant Adaptation of Large Pretrained Models33:10 - Multi-Head Adapter Routing for Cross-Task Generalization39:25 - Geometry-Aware Adaptation for Pretrained Models46:10 - Adversarial Learning for Feature Shift Detection and CorrectionPapers:Title: Activity Grammars for Temporal Action SegmentationLink: arxiv.org/abs/2312.04266Author:Dayoung Gong, Joonseok Lee,Deunsol Jung, Suha Kwak, Minsu Cho--------Title: Diffusion-TTA: Test-time Adaptation of Discriminative Models via Generative FeedbackLink: arxiv.org/abs/2311.16102Author:Mihir Prabhudesai, Tsung-Wei Ke,Alexander C. Li, Deepak Pathak, Katerina Fragkiadaki--------Title: On the Role of Noise in the Sample Complexity of Learning Recurrent Neural Networks: Exponential Gaps for Long SequencesLink: arxiv.org/abs/2305.18423Author:Alireza Fathollah Pour, Hassan Ashtiani--------Title: Sketching Algorithms for Sparse Dictionary Learning: PTAS and Turnstile StreamingLink: arxiv.org/abs/2310.19068Author:Gregory Dexter, Petros Drineas,David P. Woodruff, Taisuke Yasuda--------Title: Equivariant Adaptation of Large Pretrained ModelsLink: arxiv.org/pdf/2310.01647.pdfAuthor:Arnab Kumar Mondal, Siba Smarak Panigrahi,Sékou-Oumar Kaba, Sai Rajeswar, Siamak Ravanbakhsh--------Title: Multi-Head Adapter Routing for Cross-Task GeneralizationLink: arxiv.org/abs/2211.03831Author:Lucas Caccia, Edoardo Ponti, Zhan Su,Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni--------Title: Geometry-Aware Adaptation for Pretrained ModelsLink: arxiv.org/abs/2307.12226Author:Nicholas Roberts, Xintong Li, Dyah Adila,Sonia Cromp, Tzu-Heng Huang, Jitian Zhao, Frederic Sala--------Title: Adversarial Learning for Feature Shift Detection and CorrectionLink: arxiv.org/abs/2312.04546Author: Miriam Barrabes, Daniel Mas Montserrat,Margarita Geleta, Xavier Giro-i-Nieto, Alexander G. IoannidisLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Art @ NeurIPS 2023 Yannic Kilcher 2023-12-25 | I stumbled across the art exhibits at NeurIPSLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Paper Explained) Yannic Kilcher 2023-12-24 | #mamba #s4 #ssm OUTLINE:0:00 - Introduction0:45 - Transformers vs RNNs vs S46:10 - What are state space models?12:30 - Selective State Space Models17:55 - The Mamba architecture22:20 - The SSM layer and forward propagation31:15 - Utilizing GPU memory hierarchy34:05 - Efficient computation via prefix sums / parallel scans36:01 - Experimental results and comments38:00 - A brief look at the codePaper: arxiv.org/abs/2312.00752Abstract:Foundation models, now powering most of the exciting applications in deep learning, are almost universally based on the Transformer architecture and its core attention module. Many subquadratic-time architectures such as linear attention, gated convolution and recurrent models, and structured state space models (SSMs) have been developed to address Transformers' computational inefficiency on long sequences, but they have not performed as well as attention on important modalities such as language. We identify that a key weakness of such models is their inability to perform content-based reasoning, and make several improvements. First, simply letting the SSM parameters be functions of the input addresses their weakness with discrete modalities, allowing the model to selectively propagate or forget information along the sequence length dimension depending on the current token. Second, even though this change prevents the use of efficient convolutions, we design a hardware-aware parallel algorithm in recurrent mode. We integrate these selective SSMs into a simplified end-to-end neural network architecture without attention or even MLP blocks (Mamba). Mamba enjoys fast inference (5× higher throughput than Transformers) and linear scaling in sequence length, and its performance improves on real data up to million-length sequences. As a general sequence model backbone, Mamba achieves state-of-the-art performance across several modalities such as language, audio, and genomics. On language modeling, our Mamba-3B model outperforms Transformers of the same size and matches Transformers twice its size, both in pretraining and downstream evaluation.Authors: Albert Gu, Tri DaoLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Another Hit Piece on Open-Source AI Yannic Kilcher 2023-12-23 | Stanford researchers find problematic content in LAION-5B.Link: https://purl.stanford.edu/kh752sm9123Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
NeurIPS 2023 Poster Session 3 (Wednesday Evening) Yannic Kilcher 2023-12-18 | OUTLINE:0:00 - Intro0:20 - Graph of Circuits with GNN for Exploring the Optimal Design Space15:20 - Empowering Collaborative Filtering with Principled Adversarial Contrastive Loss22:11 - Cluster-aware Semi-supervised Learning: Relational Knowledge Distillation Provably Learns Clustering28:30 - Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception?Papers:Graph of Circuits with GNN for Exploring the Optimal Design Space openreview.net/pdf?id=VNjJAWjuEUEmpowering Collaborative Filtering with Principled Adversarial Contrastive Loss arxiv.org/abs/2310.18700Cluster-aware Semi-supervised Learning: Relational Knowledge Distillation Provably Learns Clustering arxiv.org/abs/2307.11030Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception? arxiv.org/abs/2309.13038v2Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
NeurIPS 2023 Poster Session 2 (Wednesday Morning) Yannic Kilcher 2023-12-16 | Papers:0:30 - Bifurcations and loss jumps in RNN training arxiv.org/abs/2310.175618:40 - LEDITS++: Limitless Image Editing using Text-to-Image Models arxiv.org/abs/2311.1671113:50 - Lexinvariant Language Models arxiv.org/abs/2305.1634919:15 - Transformers learn to implement preconditioned gradient descent for in-context learning arxiv.org/abs/2306.0029723:25 - Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation arxiv.org/abs/2305.0156930:40 - GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization arxiv.org/abs/2309.1602037:00 - Hardware Resilience Properties of Text-Guided Image Classifiers arxiv.org/abs/2311.14062Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
NeurIPS 2023 Vendor Hall Yannic Kilcher 2023-12-14 | Some impressions from the vendor hall of NeurIPS 2023Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
NeurIPS 2023 Poster Session 1 (Tuesday Evening) Yannic Kilcher 2023-12-14 | Papers:CamoPatch: An Evolutionary Strategy for Generating Camoflauged Adversarial Patches (openreview.net/forum?id=B94G0MXWQX)FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning (arxiv.org/abs/2309.14062)Causes and Effects of Unanticipated Numerical Deviations in Neural Network Inference Frameworks (openreview.net/forum?id=6zyFgr1b8Q)Conservative State Value Estimation for Offline Reinforcement Learning (arxiv.org/abs/2302.06884)Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
NeurIPS Live Stream Vendor Hall Yannic Kilcher 2023-12-11 | Links:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Did Google fake their Gemini Video? Yannic Kilcher 2023-12-11 | #gemini #gpt4 #chatgpt Google DeepMind released a model called Gemini and alongside, released a marketing video, which displays the model in a very "advantageous" way.https://deepmind.google/technologies/gemini/#introduction storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdfdevelopers.googleblog.com/2023/12/how-its-made-gemini-multimodal-prompting.htmlLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Text Embeddings Reveal (Almost) As Much As Text Yannic Kilcher 2023-12-09 | This paper outlines how, under certain circumstances, text embeddings can be used to reconstruct the original embedded text.OUTLINE:0:00 - Intro6:50 - Vec2Text: Iterative Embedding Inversion12:20 - How to train this?21:20 - Experimental results26:10 - How can we prevent this?31:20 - Some thoughts on sequence lengthsPaper: arxiv.org/abs/2310.06816Abstract:How much private information do text embeddings reveal about the original text? We investigate the problem of embedding \textit{inversion}, reconstructing the full text represented in dense text embeddings. We frame the problem as controlled generation: generating text that, when reembedded, is close to a fixed point in latent space. We find that although a naïve model conditioned on the embedding performs poorly, a multi-step method that iteratively corrects and re-embeds text is able to recover 92% of 32-token text inputs exactly. We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes. Our code is available on GithubAuthors: John X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov, Alexander M. RushLinks:Homepage: ykilcher.comMerch: ykilcher.com/merchYouTube: youtube.com/c/yannickilcherTwitter: twitter.com/ykilcherDiscord: ykilcher.com/discordLinkedIn: linkedin.com/in/ykilcherIf you want to support me, the best thing to do is to share out the content :)If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):SubscribeStar: subscribestar.com/yannickilcherPatreon: patreon.com/yannickilcherBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cqEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9mMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
