Uploaded October 2025 | Updated September 2026, 1 hour ago
Lance Legel presents DeepEarth, a deep learning architecture for simulation of the planet. DeepEarth learns continuous joint probability distributions of multimodal data around (x, y, z, t) coordinates of sparse observations, by minimizing the error from reconstruction of masked data. Pre-trained self-supervised foundation models first extract latent embeddings of observations from various modalities and datasets.
These embeddings are then further projected into a learnable spacetime embedding. Diverse scientific domains can be unified by DeepEarth through their modality and spacetime embeddings. Bidirectional cross-attention layers between modality and spacetime embeddings learn conditional distributions in latent space that allow DeepEarth to minimize the KL-Divergence of joint probability distributions in real vs. simulated data.
This allows DeepEarth to answer statistical questions in a Bayesian fashion, e.g., "At a given location and time, what is the joint probability distribution of possible values for variables α and β?" and "For given α and β values, what is the probability distribution over space and time?" Ecological models for predicting pollination (DeepEarth🦋) and fire (DeepEarth🔥) are now in development with promising preliminary results. To follow the latest progress, download code and data, and contribute to development, please visit github.com/legel/deepearth.
Bio: Lance Legel received his bachelor’s degree in Physics from the University of Florida and a Master of Science degree from the University of Colorado with a thesis on deep neural networks. He was a NASA planetary physicist, founded 3co.ai and Ecodash.ai, and leads the development of DeepEarth, a planetary-scale self-supervised foundation model for geospatial and ecological intelligence. For more background, please see: linkedin.com/in/legel/.
Lance Legel presents DeepEarth, a deep learning architecture for simulation of the planet. DeepEarth learns continuous joint probability distributions of multimodal data around (x, y, z, t) coordinates of sparse observations, by minimizing the error from reconstruction of masked data. Pre-trained self-supervised foundation models first extract latent embeddings of observations from various modalities and datasets.
These embeddings are then further projected into a learnable spacetime embedding. Diverse scientific domains can be unified by DeepEarth through their modality and spacetime embeddings. Bidirectional cross-attention layers between modality and spacetime embeddings learn conditional distributions in latent space that allow DeepEarth to minimize the KL-Divergence of joint probability distributions in real vs. simulated data.
This allows DeepEarth to answer statistical questions in a Bayesian fashion, e.g., "At a given location and time, what is the joint probability distribution of possible values for variables α and β?" and "For given α and β values, what is the probability distribution over space and time?" Ecological models for predicting pollination (DeepEarth🦋) and fire (DeepEarth🔥) are now in development with promising preliminary results. To follow the latest progress, download code and data, and contribute to development, please visit github.com/legel/deepearth.
Bio: Lance Legel received his bachelor’s degree in Physics from the University of Florida and a Master of Science degree from the University of Colorado with a thesis on deep neural networks. He was a NASA planetary physicist, founded 3co.ai and Ecodash.ai, and leads the development of DeepEarth, a planetary-scale self-supervised foundation model for geospatial and ecological intelligence. For more background, please see: linkedin.com/in/legel/.
![Enhancing Reasoning in Smaller Models through Self-Training
Abstract: Smaller language models can develop robust reasoning capabilities through pre-training, fine-tuning, or knowledge distillation from large language models (LLMs). However, unlike LLMs that employ a diverse array of reasoning strategies, smaller models typically rely on a single dominant approach. This limitation restricts their effectiveness in handling different multi-step reasoning tasks, which require a wide range of strategies in order to solve them. To address this challenge, self-training leverages the model’s own generated data, enabling smaller models to autonomously learn and adapt their reasoning strategies for improved performance across diverse tasks.
I will talk about a self-guided iterative distillation framework (SIKeD [1]), which combines multi-strategy outputs from LLMs with self-generated data from the smaller model to identify the most effective strategy for a given task in an on-policy manner.
Later, I will talk about how this self-training approach can be extended to improve refinement in models, where a model can learn to iteratively refine its output, eventually learning to pick the right strategy in its first attempt (SMART [2]).
[1] https://arxiv.org/abs/2410.18574
[2] https://arxiv.org/abs/2410.16128
Bio: Kumar Shridhar is a final-year Ph.D. candidate at ETH Zürich, Switzerland, under the supervision of Prof. Mrinmaya Sachan from ETH and Dr. Nicholas Monath from Google DeepMind. Prior to his doctoral studies, he spent summers interning at FAIR, Microsoft Research, and Alexa AI, and improving conversational AI at different startups.
His research focuses on advancing the reasoning capabilities of large language models (LLMs) and developing efficient distillation methods to impart these skills to smaller models. Moreover he is also working model alignment, autonomous agents, and model refinement. He is also a member of Swiss AI initiative, where the team is training foundational models across various domains. Enhancing Reasoning in Smaller Models through Self-Training](https://i.ytimg.com/vi/SS59gCT2KKs/mqdefault.jpg)









