ARO: A new lens on matrix optimization for LLMs @MicrosoftResearch
ARO: A new lens on matrix optimization for LLMs  @MicrosoftResearch
Uploaded March 2026 | Updated November 2026, 6 hours ago
We present Adaptively Rotated Optimization (ARO), a matrix optimizer that speeds up LLM training by applying updates in a rotated, geometry-aware coordinate system. Guided by new insights on global structures on LLM loss landscapes, ARO treats rotation as a unifying principle for sample efficiency, and proposed a new update policy that is applicable to all model weight matrices. In large scale controlled experiments, ARO consistently outperforms AdamW and orthogonalization-based method, maintaining its gains as models and training budgets scale.

Paper: arxiv.org/abs/2602.09006

This session aired on March 3, 2026, at Microsoft Research Forum, Season 2 Episode 3.

Register for the series to learn about future episodes: events.microsoft.com/flow/ms/researchforum/register/page/microsoftresearchforumreg/?OCID=msr_researchforum_YTDescription
Explore all previous episodes: https://aka.ms/researchforumYTplaylist
ARO: A new lens on matrix optimization for LLMsLanguage & Voice AI for Africa: From Data to Deployment and ImpactPromptionsMagenticLite: A full-stack agentic experience powered by Small ModelsWhat is Density Functional Theory (DFT)Abstracts: Zero-shot models in single-cell biology with Alex LuDeep Reinforcement Learning in Supply Chain OptimizationsWhat’s Your Story: Jacki O’NeillIdeas: More AI-resilient biosecurity with the Paraphrase ProjectuGAP: A Unified Geospatial AI Platform for African Geographic DataVoluMe: Authentic 3D Video Calls from Live Gaussian Splat PredictionWhat AIs impact on individuals means for the health workforce and industry
Microsoft Research |

ARO: A new lens on matrix optimization for LLMs

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER