Recycling finetuned models to pretrain: on loss spaces, fusing and evolving pretraining @allenai
Recycling finetuned models to pretrain: on loss spaces, fusing and evolving pretraining @allenai
Uploaded March 2023 | Updated September 2026, 1 day ago
This work will discuss the recent advancement in recycling finetuned model. Harnessing the data and computation invested in one or more models to collaboratively improve the pre-trained model they originated form. The work will also touch on our initial understandings of how and why fusing several models by weight averaging works. - Leshem Choshen/IBM Research
Recycling finetuned models to pretrain: on loss spaces, fusing and evolving pretrainingTraining Human-AI TeamsWhy Natural Language is the Right Vehicle for Complex ReasoningMolmo 2 | Video TrackingReframing Instructional Prompts to GPTk’s LanguageOLMoTrace | Connecting a language model’s response back to its training dataInteractive Reading and Authoring with IDEs for Ideas | AI2Grounding Foundation Models for Embodied IntelligenceToward Intelligent Writing Support Beyond Completing SentencesEnabling Scientific Research with Language AgentsDREAM: Improving Situational QA by First Elaborating the SituationObjective Mismatch in Reinforcement Learning from Human Feedback
Ai2 |

Recycling finetuned models to pretrain: on loss spaces, fusing and evolving pretraining

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER