Uploaded February 2025 | Updated September 2026, 9 hours ago
January 30, 2025 - Here is Tülu 3 405B 🐫 our open-source post-training model that surpasses the performance of DeepSeek-V3! The last member of the Tülu 3 family demonstrates that our recipe, which includes Reinforcement Learning from Verifiable Rewards (RVLR) scales to 405B - with performance on par with GPT-4o, and surpassing prior open-weight post-trained models of the same size including Llama 3.1.
Learn more about Tülu 3 405B on the Ai2 blog: allenai.org/blog/tulu-3-405B
Chat with Tülu 3 405B on the Ai2 Playground: playground.allenai.org/?model=tulu3-405b
Look back at the Tülu 3 recipe: allenai.org/blog/tulu-3-technical
Check out models on Hugging Face: huggingface.co/collections/allenai/tulu-3-models-673b8e0dc3512e30e7dc54f5
Questions? Ask on Discord: discord.gg/NE5xPufNwu
January 30, 2025 - Here is Tülu 3 405B 🐫 our open-source post-training model that surpasses the performance of DeepSeek-V3! The last member of the Tülu 3 family demonstrates that our recipe, which includes Reinforcement Learning from Verifiable Rewards (RVLR) scales to 405B - with performance on par with GPT-4o, and surpassing prior open-weight post-trained models of the same size including Llama 3.1.
Learn more about Tülu 3 405B on the Ai2 blog: allenai.org/blog/tulu-3-405B
Chat with Tülu 3 405B on the Ai2 Playground: playground.allenai.org/?model=tulu3-405b
Look back at the Tülu 3 recipe: allenai.org/blog/tulu-3-technical
Check out models on Hugging Face: huggingface.co/collections/allenai/tulu-3-models-673b8e0dc3512e30e7dc54f5
Questions? Ask on Discord: discord.gg/NE5xPufNwu
![Explaining Answers with Entailment Trees
Explaining Answers with Entailment Trees
Bhavana Dalvi
https://allenai.org/team/bhavanad
Our goal, in the context of open-domain textual question-answering (QA), is to explain answers by showing the line of reasoning from what is known to the answer, rather than simply showing a fragment of textual evidence (a rationale). If this could be done, new opportunities for understanding and debugging the systems reasoning become possible. Our approach is to generate explanations in the form of entailment trees, namely a tree of multipremise entailment steps from facts that are known, through intermediate conclusions, to the hypothesis of interest (namely the question + answer). To train a model with this skill, we created ENTAILMENTBANK, the first dataset to contain multistep entailment trees. Given a hypothesis (question + answer), we define three increasingly difficult explanation tasks: generate a valid entailment tree given (a) all relevant sentences (b) all relevant and some irrelevant sentences, or (c) a corpus. We show that a strong language model can partially solve these tasks, in particular when the relevant sentences are included in the input (e.g., 35% of trees for (a) are perfect), and with indications of generalization to other domains. This work is significant as it provides a new type of dataset (multistep entailments) and baselines, offering a new avenue for the community to generate richer, more systematic explanations. [ Paper published in EMNLP 2021, https://arxiv.org/abs/2104.08661 ] Explaining Answers with Entailment Trees](https://i.ytimg.com/vi/QPSZQYA1RmA/mqdefault.jpg)






![Enhancing Reasoning in Smaller Models through Self-Training
Abstract: Smaller language models can develop robust reasoning capabilities through pre-training, fine-tuning, or knowledge distillation from large language models (LLMs). However, unlike LLMs that employ a diverse array of reasoning strategies, smaller models typically rely on a single dominant approach. This limitation restricts their effectiveness in handling different multi-step reasoning tasks, which require a wide range of strategies in order to solve them. To address this challenge, self-training leverages the model’s own generated data, enabling smaller models to autonomously learn and adapt their reasoning strategies for improved performance across diverse tasks.
I will talk about a self-guided iterative distillation framework (SIKeD [1]), which combines multi-strategy outputs from LLMs with self-generated data from the smaller model to identify the most effective strategy for a given task in an on-policy manner.
Later, I will talk about how this self-training approach can be extended to improve refinement in models, where a model can learn to iteratively refine its output, eventually learning to pick the right strategy in its first attempt (SMART [2]).
[1] https://arxiv.org/abs/2410.18574
[2] https://arxiv.org/abs/2410.16128
Bio: Kumar Shridhar is a final-year Ph.D. candidate at ETH Zürich, Switzerland, under the supervision of Prof. Mrinmaya Sachan from ETH and Dr. Nicholas Monath from Google DeepMind. Prior to his doctoral studies, he spent summers interning at FAIR, Microsoft Research, and Alexa AI, and improving conversational AI at different startups.
His research focuses on advancing the reasoning capabilities of large language models (LLMs) and developing efficient distillation methods to impart these skills to smaller models. Moreover he is also working model alignment, autonomous agents, and model refinement. He is also a member of Swiss AI initiative, where the team is training foundational models across various domains. Enhancing Reasoning in Smaller Models through Self-Training](https://i.ytimg.com/vi/SS59gCT2KKs/mqdefault.jpg)


