A Conversation with Bill Gates Hosted by Eric Horvitz Microsoft Research 2024-10-21 | A Conversation with Bill Gates Hosted by Eric Horvitz
Look Ma, no markers: holistic performance capture without the hassle Microsoft Research 2024-10-17 | We tackle the problem of highly-accurate, holistic performance capture for the face, body and hands simultaneously. Motion-capture technologies used in film and game production typically focus only on face, body or hand capture independently, involve complex and expensive hardware and a high degree of manual intervention from skilled operators. While machine-learning-based approaches exist to overcome these problems, they usually only support a single camera, often operate on a single part of the body, do not produce precise world-space results, and rarely generalize outside specific contexts. In this work, we introduce the first technique for marker-free, high-quality reconstruction of the complete human body, including eyes and tongue, without requiring any calibration, manual intervention or custom hardware. Our approach produces stable world-space results from arbitrary camera rigs as well as supporting varied capture environments and clothing. We achieve this through a hybrid approach that leverages machine learning models trained exclusively on synthetic data and powerful parametric models of human shape and motion. We evaluate our method on a number of body, face and hand reconstruction benchmarks and demonstrate state-of-the-art results that generalize on diverse datasets.See the project page for more details and dataset download instructions: https://aka.ms/synthmocap
Hairmony: Fairness-aware hairstyle classification Microsoft Research 2024-10-17 | We present a method for prediction of a person’s hairstyle from a single image. Despite growing use cases in user digitization and enrollment for virtual experiences, available methods are limited, particularly in the range of hairstyles they can capture. Human hair is extremely diverse and lacks any universally accepted description or categorization, making this a challenging task. Most current methods rely on parametric models of hair at a strand level. These approaches, while very promising, are not yet able to represent short, frizzy, coily hair and gathered hairstyles. We instead choose a classification approach which can represent the diversity of hairstyles required for a truly robust and inclusive system. Previous classification approaches have been restricted by poorly labeled data that lacks diversity, imposing constraints on the usefulness of any resulting enrollment system. We use only synthetic data to train our models. This allows for explicit control of diversity of hairstyle attributes, hair colors, facial appearance, poses, environments and other parameters. It also produces noise-free ground-truth labels. We introduce a novel hairstyle taxonomy developed in collaboration with a diverse group of domain experts which we use to balance our training data, supervise our model, and directly measure fairness. We annotate our synthetic training data and a real evaluation dataset using this taxonomy and release both to enable comparison of future hairstyle prediction approaches. We employ an architecture based on a pre-trained feature extraction network in order to improve generalization of our method to real data and predict taxonomy attributes as an auxiliary task to improve accuracy. Results show our method to be significantly more robust for challenging hairstyles than recent parametric approaches. Evaluation with taxonomy-based metrics also demonstrates the fairness of our method across diverse hairstyles.For more details and dataset download instructions see the project page: http://aka.ms/hairmony
Data Formulator: Create Rich Visualization with AI iteratively Microsoft Research 2024-10-01 | Data Formulator is a research prototype tool that enables users to iterate with AI to create rich data visualizations. Users define their goals through a blend of user interface interactions and natural language inputs, while the AI automatically generates and adjusts the code to transform data into charts.
Pretrainers Guide to Training Data: Measuring Effects of Age, Domain Coverage, Quality, & Toxicity Microsoft Research 2024-09-27 | Pretraining is the preliminary and fundamental step in developing capable language models (LM). Despite this, pretraining data design is critically under-documented and often guided by empirically unsupported intuitions. To address this, we pretrain 28 1.5B parameter decoder-only models, training on data curated (1) at different times, (2) with varying toxicity and quality filters, and (3) with different domain compositions. First, we quantify the effect of pretraining data age. A temporal shift between evaluation data and pretraining data leads to performance degradation, which is not overcome by finetuning. Second, we explore the effect of quality and toxicity filters, showing a trade-off between performance on standard benchmarks and risk of toxic generations. Our findings indicate there does not exist a one-size-fits-all solution to filtering training data. We also find that the effects of different types of filtering are not predictable from text domain characteristics. Lastly, we empirically validate that the inclusion of heterogeneous data sources, like books and web, is broadly beneficial and warrants greater prioritization. These findings constitute the largest set of experiments to validate, quantify, and expose many undocumented intuitions about text pretraining, which we hope will help support more informed data-centric decisions in LM development.
AI for Business Transformation: Lessons from Healthcare Microsoft Research 2024-09-18 | AI is already helping healthcare organizations drive innovation and efficiency gains. Peter Lee and Vijay Mital explore how Microsoft is helping, and how the lessons from healthcare could apply across many other industries.0:00 – Introduction4:35 – Beginnings of AI transformation 6:22 – Early lessons from healthcare 12:30 – The importance of trust 18:17 – How do we get started? 25:00 – Building rapport with AI 27: 31 – A spectacular rate of progress 30:53 – Is my business ripe for AI? 35:04 – The emergence of multimodal AI 40:30 – Connecting the dots for practical opportunities 41:37 – How is this generation of AI innovation like previous tech? Related: Azure Essentials Show | Essential AI for Business: https://aka.ms/EssentialAIForBusiness
AI for Business Transformation: Multimodal Models Microsoft Research 2024-09-18 | Multimodal models hold the key to unprecedented advances in AI, moving beyond text and numbers to incorporate a much wider spectrum of inputs. AI research leaders Peter Lee and Vijay Mital discuss how researchers are exploring a new generation of models to open broad new opportunities for businesses.0:00 – Introduction7:39 – Different types of AI models 12:46 – The emergence of multimodal AI 17:17 – Large action models 21:42 – AI apps 30:06 – The transformation mindset Related: Azure Essentials Show | Essential AI for Business: https://aka.ms/EssentialAIForBusiness
AI for Business Transformation: The Business of Data Microsoft Research 2024-09-18 | The value of data has never been higher—it has become an essential driver of business decisions and it is critical for deploying AI successfully. AI research leaders Peter Lee and Vijay Mital explore how AI can maximize that value and how business leaders can prepare for the waves of innovation to come and get the most out of their investments in AI.0:00 – Introduction4:59 – A new type of gold rush 10:46 – Data architecture 20:04 – Retrieval augmented generation (RAG) 28:34 – What can generative AI unlock? 33:26 – Protecting privacy 43:22 – Being responsible with data Related: Azure Essentials Show | Essential AI for Business: https://aka.ms/EssentialAIForBusiness
Ludic Design for Accessibility Microsoft Research 2024-09-18 | Technology solutions for accessibility have long been created using a narrow utilitarian lens, especially in the Global South due to multi-dimensional challenges and resource constraints—an emphasis on purely functional outcomes supported by sterile cost-benefit analysis that ignores the fact that people with disability are people first with their own aspirations for leisure and enjoyment in addition to skills and employment. We propose an alternate design methodology called the Ludic Design for Accessibility (LDA) that puts play and playfulness at the center of all assistive technology design and use. For more details visit https://aka.ms/ludicdesign
At the Foothills of an AI Era in Science | Gilbert S. Omenn Grand Challenges Address Microsoft Research 2024-09-16 | Eric Horvitz, Chief Scientific Officer of Microsoft, delivers the 2024 Gilbert S. Omenn Grand Challenges Address, entitled, “At the Foothills of an AI Era in Science.” The lecture was delivered at the AAAS Science & Technology Policy Forum in Washington DC, on July 12, 2024.
Fostering appropriate reliance on AI Microsoft Research 2024-09-03 | Mihaela Vorvoreanu, Director UX Research and Responsible AI Education, Microsoft Aether, discusses reliance on AI. Because of their probabilistic nature, all AI systems will make mistakes. One of the main challenges in human-AI interaction is to foster appropriate reliance on AI, and empower users of AI systems to determine when to accept or not accept an AI system's recommendation. Hear about the work being done at Microsoft to foster appropriate reliance and help people accept AI outputs when they are correct, and reject them when they are wrong.Microsoft Research Forum, September 3, 2024See more at https://aka.ms/ResearchForum-Sep2024
A generative model of biology for in-silico experimentation and discovery Microsoft Research 2024-09-03 | Kevin Yang, Senior Researcher, Microsoft Research New England, shares how deep learning is enabling us to generate novel and useful biomolecules, allowing researchers and practitioners to better understand biology.Microsoft Research Forum, September 3, 2024See more at https://aka.ms/ResearchForum-Sep2024
Project Aurora: The first large-scale foundation model of the atmosphere Microsoft Research 2024-09-03 | Megan Stanley, Senior Researcher, Microsoft Research AI for Science, talks about Aurora, a cutting-edge foundation model that offers a new approach to weather forecasting that could transform our ability to predict and mitigate the impacts of extreme events, air pollution, and the changing climate.Microsoft Research Forum, September 3, 2024See more at https://aka.ms/ResearchForum-Sep2024
Direct Nash Optimization: Teaching language models to self-improve with general preferences Microsoft Research 2024-09-03 | Corby Rosset, Senior Researcher, Microsoft Research AI Frontiers, discusses teaching language models to self-improve using a preference oracle like GPT-4, framing it as a two-player game to find an optimal policy at a Nash equilibrium, and achieving state-of-the-art win rates against GPT-4 Turbo on benchmarks such as Alpaca-Eval and MT-Bench.Microsoft Research Forum, September 3, 2024See more at https://aka.ms/ResearchForum-Sep2024
Analog optical computing for sustainable AI and beyond Microsoft Research 2024-09-03 | Francesca Parmigiani and Jiaqi Chu, researchers at Microsoft Research Cambridge, discuss a new kind of computer – an analog optical computer – that has the potential to accelerate AI inference and hard optimization workloads by 100x, leveraging hardware-software co-design to improve the efficiency and sustainability of real-world applications.Microsoft Research Forum, September 3, 2024See more at https://aka.ms/ResearchForum-Sep2024
Panel Discussion: Beyond Language: The future of multimodal models in healthcare, gaming, and AI Microsoft Research 2024-09-03 | Microsoft researchers discuss the transformative potential and core challenges of multimodal models across various domains, including precision health, game intelligence, and foundation models. Microsoft researchers will share their thoughts on future directions, bridging gaps, and fostering synergies within the field.Katja Hofmann, Senior Principal Researcher, Microsoft ResearchJianwei Yang, Principal Researcher, Microsoft Research RedmondHoifung Poon, General Manager, Microsoft Research Health FuturesJohn Langford (host), Partner Research Manager, Microsoft Research AI FrontiersMicrosoft Research Forum, September 3, 2024See more at https://aka.ms/ResearchForum-Sep2024
Research Forum 4 | Keynote: Phi-3-Vision: A highly capable and small language vision model Microsoft Research 2024-09-03 | Jianfeng Gao, Distinguished Scientist and Vice President in Microsoft Research Redmond, introduces Phi-3-Vision, an advanced and economical open-source multimodal model. As a member of the Phi-3 model family, Phi-3-Vision enhances language models by integrating multi-sensory skills, seamlessly combining language and vision capabilities.Microsoft Research Forum, September 3, 2024See more at https://aka.ms/ResearchForum-Sep2024
CataractBot: An LLM-Powered Experts-in-the-Loop Chatbot for Cataract Patients Microsoft Research 2024-09-03 | Patients need reliable health information, but the digital age overwhelms them with excess and often inaccurate information. Patients trust medical professionals the most, but time constraints constrain this communication. To bridge this gap, we at Microsoft Research Lab India in partnership with the Sankara Eye Hospital in Bangalore, developed CataractBot. CataractBot is an experts-in-the-loop chatbot, powered by GPT-4. Patients can ask cataract surgery questions on WhatsApp, and the bot instantly answers using a curated knowledge base and delivers expert-verified answers asynchronously. CataractBot proved valuable in a real-world deployment used by 500+ patient and attendants. It saves time, has multilingual and multimodal support for diverse literacy levels, and adds a privacy layer between patients and doctors, cultivating trust through expert verification. We have open-sourced our work to guide future designs of expert-mediated LLM bots.Build Your Own expert Bot (BYOeB): github.com/microsoft/byoebResearch Paper: arxiv.org/abs/2402.04620People:Microsoft Research India: Mohit Jain, Pragnya Ramjee, Satvik Golechha, Shreyas KulkarniSankara Eye Hospital: Kaushik Murali, Geeta Fulari, Bhuvan Sachdeva
ML for High-Performance Climate and Earth Virtualization Engines Microsoft Research 2024-08-27 | Speaker: Torsten Hoefler Host: Karin Strauss Machine learning presents a great opportunity for Climate simulation and research. We will discuss some ideas from the Earth Virtualization Engines summit in Berlin and several research results ranging from ensemble prediction and bias correction of simulation output, extreme compression of high-resolution data, and a vision towards affordable km-scale ensemble simulations. We will also discuss programming framework research to improve simulation performance. Specifically, our ensemble spread prediction and bias correction network applied to global data, achieves a relative improvement in ensemble forecast skill (CRPS) of over 14%. Furthermore, we demonstrate that the improvement is larger for extreme weather events on select case studies. We also show that our post-processing can use fewer trajectories to achieve comparable results to the full ensemble. Our ML-based compression method achieves data reduction from 300x to more than 3,000x and outperforms the state-of-the-art compressor SZ3 in terms of weighted RMSE and MAE. It can faithfully preserve important large scale atmosphere structures and does not introduce artifacts. When using the resulting neural network as a 790x compressed data loader to train the WeatherBench forecasting model, its RMSE increases by less than 2%. The three orders of magnitude compression democratizes access to high-resolution climate data and enables numerous new research directions. We will close by discussing ongoing research directions and opportunities for using machine learning for ensemble simulations and combine several machine learning techniques. All those methods will contribute to enabling km-scale global climate simulations.See more at microsoft.com/en-us/research/video/ml-for-high-performance-climate-and-earth-virtualization-engines
Final intern talk: Distilling Self-Supervised-Learning-Based Speech Quality Assessment into Compact Microsoft Research 2024-08-27 | Speaker: Benjamin Stahl Host: Hannes Gamper In this talk, we explore advancements in computational models for speech quality assessment. Self-supervised learning models have emerged as powerful front-ends, outperforming supervised-only models. However, their large size renders them impractical for production tasks. We discuss strategies to distill self-supervised learning-based models into more compact forms using unlabeled data, achieving significant size reduction while maintaining an advantage over supervised-only models.See more at microsoft.com/en-us/research/video/final-intern-talk-distilling-self-supervised-learning-based-speech-quality-assessment-into-compact
Decoding the Human Brain – A Neurosurgeon’s Experience Microsoft Research 2024-08-26 | Pascal O. Zinn, MD, PhD, FAANS Dr. Zinn will discuss his experience leveraging state-of-the-art techniques for decoding the human brain during awake surgeries and complex brain tumor removals. Utilizing electrical brain stimulation, electrocorticography (ECoG) and advanced neuropsychological methods, Dr. Zinn and his team explore intricate conscious functions of the human brain in real-time. The team works in a concerted effort across disciplines to tailor this approach to each individual patient, designing the safest and most efficient surgical paradigms to maximize brain tumor removal while preserving essential functions such as speech, memory, and cognition/consciousness. His talk will focus on the latest advancements in neurosurgery and neuroscience, providing a unique glimpse into the intersection of medical innovation and technology on the man-machine interface in neurosurgery.See more at microsoft.com/en-us/research/video/decoding-the-human-brain-a-neurosurgeons-experience
Mapping the World: Creating a Global and Temporal High-Resolution Building Density Map Microsoft Research 2024-08-09 | This seminar was hosted by Microsoft Research Africa, Nairobi together with the Microsoft AI for Good team in July 2024.Knowing where people live is critical for urban planning, disaster management, and more broadly, for informing public policies. Unfortunately, in many countries, especially in the global south, census data is often outdated. At the same time, existing building and population layers are either very coarse in resolution, limited in coverage areas, or lack a temporal dimension.In response to this need, our study leverages high-resolution PlanetScope satellite imagery and a fusion of extensive building footprint datasets to develop a robust method for estimating the extent of built-up areas globally and over time, from 2017 Q3 to 2023 Q3. Our methodology relies on an adapted version of a U-Net Convolutional Neural Network, pretrained on ImageNet, and augmented with a mixture of experts component to handle various landscapes and generalize better at a global scale. We generate precise building density values for 40 x 40-meter patches globally. Our approach enables precise and automated building density mapping to inform public decision-making at an unprecedented scale.Speaker: Dr. Gilles Quentin HachemeLearn more about Microsoft Research Lab – Africa, Nairobi: microsoft.com/en-us/research/lab/microsoft-research-lab-africa-nairobi/seminars
AgriAdvisor Concept Video Microsoft Research 2024-08-08 | AgriAdvisor is a pilot demo created in collaboration with TomorrowNow.org, Tomorrow.io and Kenya Agriculture and Livestock Research Organisation (KALRO). This AI copilot provides small-scale farmers with real-time, reliable weather and agronomic guidance that can help them improve on their on-farm decision making. AgriAdvisor leverages generative AI to deliver the best available agronomic practices into actionable advice for Kenyan farmers in both English and Swahili.See more at microsoft.com/en-us/research/video/agriadvisor-concept-video
Proactive Resume and Pause of Resources for Microsoft Azure SQL Database Serverless Microsoft Research 2024-07-15 | Demand-driven resource allocation for cloud databases has become a popular research direction. Recent approaches have evolved from reactive policies to proactive decision making. These approaches leverage not only the current resource demand but also the predicted demand to make more informed resource allocation decisions for each database and thus improve the quality of service and reduce the operational costs. We present an infrastructure that enables proactive resource allocation capabilities for millions of serverless Azure SQL databases. Our solution finds near-optimal middle ground between high availability of resources, low operational costs, and low computational overhead of the proactive policy. We describe the design principles we followed and the architectural decisions we made during this cross-team, multi-year journey. Given the size and scope of our solution, we believe that the relational cloud databases in other companies could benefit from the proactive resource allocation capabilities.Project: microsoft.com/en-us/research/project/moneyballPublication: microsoft.com/en-us/research/publication/prorp
Advances in Natural Language Generation for Indian Languages Microsoft Research 2024-07-12 | Speaker: Dr. Raj Dabre Host: Sunayana Sitaram Much of recent progress for natural language generation (NLG) has been in the context of English and, in general, high resource languages, however, Indian languages have yet to see similar paradigm shifts despite their speaking population comprising about a fifth of the world's population. Two major constraints are data and compute, and in this talk, I will touch on both. I will begin with our earliest work on IndicBART, which leveraged monolingual data and helped overcome resource scarcity of Indian languages as measured on the IndicNLG benchmark. I will then highlight three recent works, two focusing on overcoming data scarcity via mass crawling, cleaning and synthetic data creation with the third focusing on compute scarcity via leveraging romanization alongside an existing strong English LLM. This will hopefully lead to discussions which will help push the boundary of language modeling and NLG for Indian languages.See more at microsoft.com/en-us/research/video/advances-in-natural-language-generation-for-indian-languages
Making Sentence Embeddings Robust to User-Generated Content Microsoft Research 2024-06-07 | This seminar was hosted by Microsoft Research Africa, Nairobi together with the Microsoft AI for Good team in May 2024.User-generated content (UGC), e.g. social media posts written in "Internet language", presents a lot of lexical variations and deviates from standard language. As a result, NLP models which were mostly trained on standard texts have been known to perform poorly on UGC, and sentence embedding models like LASER are no exception. In this talk, we focus on the robustness of LASER to UGC data. We evaluate this robustness by LASER’s ability to represent non-standard sentences and their standard counterparts close to each other in the embedding space. Inspired by previous works extending LASER to other languages and modalities, we propose RoLASER, a robust English encoder trained using a teacher-student approach to reduce the distances between the representations of standard and UGC sentences. We also use data augmentation to generate synthetic UGC-like training data. We show that RoLASER significantly improves LASER’s robustness to both natural and artificial UGC data by achieving up to 2× and 11× better alignment scores. We also perform a fine-grained analysis on artificial UGC data and find that our model greatly outperforms LASER on its most challenging UGC phenomena such as keyboard typos and social media abbreviations. Evaluation on downstream tasks shows that RoLASER performs comparably to or better than LASER on standard data, while consistently outperforming it on UGC data.Speaker: Lydia NishimweLearn more about Microsoft Research Lab – Africa, Nairobi: microsoft.com/en-us/research/lab/microsoft-research-lab-africa-nairobi/seminars
Research Forum 3 | Keynote: Building Globally Equitable AI Microsoft Research 2024-06-06 | Jacki O'Neill, Lab Director of Microsoft Research Africa, Nairobi, discusses the importance of creating globally equitable generative AI. She addresses the technical and sociotechnical challenges that must be tackled to positively transform work futures worldwide.Microsoft Research Forum, June 4, 2024See more at https://aka.ms/ResearchForum-Jun2024*edited video - slide correction
AutoGen Update: Complex Tasks and Agents Microsoft Research 2024-06-04 | Adam Fourney discusses the effectiveness of using multiple agents, working together, to complete complex multi-step tasks. He will showcase their capability to outperform previous single-agent solutions on benchmarks like GAIA, utilizing customizable arrangements of agents that collaborate, reason, and utilize tools to achieve complex outcomes.Microsoft Research Forum, June 4, 2024See more at https://aka.ms/ResearchForum-Jun2024*edited to update a slide
MatterGen: A Generative Model for Materials Design Microsoft Research 2024-06-04 | Tian Xie introduces MatterGen, a generative model that creates new inorganic materials based on a broad range of property conditions required by the application, aiming to shift the traditional paradigm of materials design with generative AI.Microsoft Research Forum, June 4, 2024See more at https://aka.ms/ResearchForum-Jun2024
Driving Industry Evolution: Exploring the Impact of Generative AI on Sector Transformation Microsoft Research 2024-06-04 | Jiang Bian discusses how generative AI transforms industries by bridging gaps between AI capabilities and sector needs. He will showcase domain-specific foundation models and versatile AI agents, setting new industry standards.Microsoft Research Forum, June 4, 2024See more at https://aka.ms/ResearchForum-Jun2024
Challenges and Opportunities of Large Multi-Modal Models for Blind and Low Vision Users: CLIP Microsoft Research 2024-06-04 | Insights into the Challenges and Opportunities of Large Multi-Modal Models for Blind and Low Vision Users: A Case Study on CLIPDaniela Massiceti delves into the transformative potential of multimodal models such as CLIP for assistive technologies. Specifically focusing on the blind/low-vision community, the talk explores the current distance from realizing this potential and the advancements needed to bridge this gap.Microsoft Research Forum, June 4, 2024See more at https://aka.ms/ResearchForum-Jun2024
Panel Discussion: Generative AI for Global Impact: Challenges and Opportunities Microsoft Research 2024-06-04 | Microsoft researchers discuss the challenges and opportunities of making AI more inclusive and impactful for everyone—from data that represents a broader range of communities and cultures to novel use cases for AI that are globally relevant.Microsoft Research Forum, June 4, 2024See more at https://aka.ms/ResearchForum-Jun2024
Join us for Research Forum on September 3, 2024 Microsoft Research 2024-05-14 | REGISTER NOW: https://aka.ms/ResearchForum_YTRegAI is transforming the way we all live, work, and think. Join us for Microsoft Research Forum, where you'll learn about the most recent advances and hear bold new ideas from the global research community. See you on September 3, 2024, for Episode 4. Register now. https://aka.ms/ResearchForum_YTReg
Unlocking Real world solutions with AI – Chris Bishop Microsoft Research 2024-05-10 | Chris Bishop reveals how AI is revolutionizing material science with an innovative battery electrolyte material. With the help of MatterGen, an AI system akin to a search engine, researchers can explore novel material options with precision and efficiency. The broad potential of these AI systems spans industries from drug discovery to environmental science.See more at microsoft.com/en-us/research/video/unlocking-real-world-solutions-with-ai-chris-bishop
How will AI transform precision medicine? – Ava Amini Microsoft Research 2024-05-10 | Ava Amini explores the role of AI in precision medicine, with a focus on personalized cancer treatments. By leveraging AI to analyze broad biological data sets, researchers aim to tailor therapies to individual patients based on their specific needs. Innovative methods like EvoDiff offer promising alternatives to the current population-based approaches, and open doors to advance precision oncology.See more at microsoft.com/en-us/research/video/how-will-ai-transform-precision-medicine-ava-amini
AI Case Studies for Natural Science Research with Bonnie Kruft Microsoft Research 2024-05-03 | Generative AI is unlocking new research tools for bold scientific discoveries. We sort through the hype and take a deep dive into some practical examples of groundbreaking research enabled by generative AI such as small molecular inhibitors for treating infectious disease and the discovery of new materials for energy storage. As researchers reduce the discovery time from years to months, how are they ensuring that safe and responsible practices are used to instill public trust in the process?0:00 Introduction0:23 Scientific discovery is the most important use of AI1:23 What Large Language Models bring to science2:06 What makes scientific discovery different?3:40 Prior knowledge6:19 "No-free-lunch theorem"9:16 Generative AI model MatterGen9:55 Drug discovery and deep learning14:12 Large Language Models v other training models15:40 The evolution of generative AI models16:37 How generative AI models can assist scientists19:08 The role of AI in drug development22:35 How Large Language Models can work with science-based models26:00 Looking ahead for AI in scienceMIT Technology Review's EmTech Digital: event.technologyreview.com/emtech-digital-europe-2024/home
AI For All: Embracing Equity for All Microsoft Research 2024-04-29 | This grand seminar was hosted by Microsoft Research Africa, Nairobi together with the Microsoft AI for Good team in April 2024. It was the second in their grand seminars which brought together people from academia, and industry to discuss how we can embrace equity in the AI spaceThe three-hour event included a keynote by Dr. Lavri Labi titled “Addressing Bias and Representation in Generative AI Outputs,” where he examined the profound impact of generative AI on society as Generative AI, driven by large-scale datasets, often reflects the biases inherent in the data it is trained on. This was followed by a spotlight discussion by Stanslaus Mwongela who explained how UlizaLlama, a first-of-its-kind open-access LLM by Jacaranda Health fine-tuned from the Llama 2 open weight model by Meta, marks an innovative step forward in making AI-driven support accessible to Swahili-speaking communities. The session ended with a panel discussion with experts from academia and industry including Dr. Kalika Bali, Dr. Lavri Labi, Millicent Ochieng, Dr. John Wamburu, Maxamed Ahmed, Dr. Gilles Hacheme and moderated by Dr. Girmaw Abebe Tadesse.Speakers: Dr Jacki O'Neill, Dr. Lavri Labi, Stanslaus Mwongela, Dr. Girmaw Abebe Tadesse, Dr. Kalika Bali, Dr. Lavri Labi, Millicent Ochieng, Dr. John Wamburu, Maxamed Ahmed, Dr. Gilles Hacheme and Samuel Maina Learn more about MARI: microsoft.com/en-us/research/group/microsoft-africa-research-institute-mari
Combining Machine Learning and Bayesian networks for Decision Support in Arrythmia Diagnosis Microsoft Research 2024-04-09 | We propose an architecture for a personal health agent (PHA) that combines machine learning and a Bayesian network for detecting and diagnosing arrhythmia based on electrocardiogram (ECG) characteristics. Focusis placed on atrial fibrillation (AF), the commonest type of arrhythmia. Machine learning is used for classifying the ECG signal. The absence of a Pwave in an ECG is the hallmark indication of AF. Four ML models are trained to classify an ECG signal based on the presence or absence of the P-wave: multilayer perceptron (MLP), logistic regression, support vector machine, and random forest. The MLP was the best performing model with an accuracy of 89.61% and an F1 score of 88.68%. A BN representing AF risk factors is developed based on expert knowledge from the literature and evaluated using Pitchforth and Mengersen’s validation framework. The presence or absence of a P-wave as determined by the ML model is input into the BN. In a bid to extend this work, instead of a binary classification of the ECG signal based on the presence or absence of a P-wave, we classify an ECG signal as either atrial fibrillation, other arrhythmia, or no arrhythmia. Four ML models, i.e., gradient boosting, random forest, multilayer perceptron and support vector machine, are compared and evaluated using a dataset of 5,340 records containing 12-lead ECG signals created from the Chapman-Shaoxing database. Among the four models, the gradient boosting model produces the best accuracy of 82.88%. The detected pattern is integrated into a BN that captures expert knowledge about the causes of arrhythmia. The agent has the ability to guide the diagnosis process. It suggests what questions to ask to increase certainty in the presence of arrhythmia, and what arrhythmia causes to follow up. The architecture is evaluated using application use cases.Speaker: Tezira Wanyana, University of Cape Town, Mbarara University of Science and TechnologyLearn more about MARI: microsoft.com/en-us/research/group/microsoft-africa-research-institute-mari
Strategic Subset Selection in Satellite Imagery: Machine Vision Insights Microsoft Research 2024-03-20 | The abundance of the currently available satellite and aerial images contrasts sharply with the scarcity of labels for these images. With data on such a grand scale, labeling everything, even to a small degree, is impractical. This raises an important question: which image patches should we prioritize for labeling? The data-centric machine learning challenge at the Machine Vision for Earth Observation 2023 (opens in new tab) workshop addressed this issue using the DFC2022 (opens in new tab) dataset. In this challenge, participants were provided with a set of image patches for land cover segmentation. Instead of focusing on model training, the participants were tasked with selecting three subsets of the training data (1%, 10%, and 25%) to be labeled. These subsets were then used to train a standard deep learning semantic segmentation model following a fixed routine, and the trained model was evaluated on an annotated test set that was not disclosed to the contestants. This challenge highlighted the importance of selecting the most advantageous samples for the training process, and managing the label noise present in the DFC2022 dataset. We present our winning methods for subset selection in satellite imagery.Speakers: Dr. Akram Zaytar & Dr. Simone Nsutezo Fobi, Microsoft AI for GoodLearn more about MARI: microsoft.com/en-us/research/group/microsoft-africa-research-institute-mari
Generative AI and Plural Governance: Mitigating Challenges and Surfacing Opportunities Microsoft Research 2024-03-11 | Microsoft Research Forum | Episode 2 | March 5, 2024Madeleine Daepp talked about the potential impacts and challenges of generative AI in a year with over 70 major global elections, and AI and Society Fellow Vanessa Gathecha discussed her work on disinformation in Kenya and Sub-Saharan Africa.See more at https://aka.ms/ResearchForum-Mar2024
GigaPath: Foundation Model for Digital Pathology Microsoft Research 2024-03-11 | Microsoft Research Forum | Episode 2 | March 5, 2024Naoto Usuyama proposed GigaPath, a novel approach for training large vision transformers for gigapixel pathology images, utilizing a diverse real-world cancer patient dataset, with the goal of laying a foundation for cancer pathology AI.See more at https://aka.ms/ResearchForum-Mar2024
Getting Modular with Language Models: Building, Reusing a Library of Experts for Task Generalization Microsoft Research 2024-03-11 | Microsoft Research Forum | Episode 2 | March 5, 2024Alessandro Sordoni shared recent efforts on building and re-using large collections of expert language models to improve zero-shot and few-shot generalization to unseen tasks.See more at https://aka.ms/ResearchForum-Mar2024
The Metacognitive Demands and Opportunities of Generative AI Microsoft Research 2024-03-11 | Microsoft Research Forum | Episode 2 | March 5, 2024Lev Tankelevitch explored how metacognition—the psychological capacity to monitor and regulate one's cognitive processes—provides a valuable perspective for comprehending and addressing the usability challenges of generative AI systems around prompting, assessing and relying on outputs, and workflow optimization.See more at https://aka.ms/ResearchForum-Mar2024
Whats new in AutoGen? Microsoft Research 2024-03-11 | Microsoft Research Forum | Episode 2 | March 5, 2024Chi Wang discussed the latest updates on AutoGen – the multi-agent framework for next generation AI applications. This includes milestones achieved, community feedback, new exciting features, and ongoing research and challenges.See more at https://aka.ms/ResearchForum-Mar2024
Research Forum 2 | Panel Discussion: Transforming the Natural Sciences with AI Microsoft Research 2024-03-11 | Microsoft Research Forum | Episode 2 | March 5, 2024Microsoft researchers shared their advancements in the fields of foundations models, drug discovery, material design and machine learning. They will highlight how deep learning is transforming the natural sciences.See more at https://aka.ms/ResearchForum-Mar2024
Research Forum 2 | Keynote: The Revolution in Scientific Discovery Microsoft Research 2024-03-11 | REGISTER NOW for Microsoft Research Forum, an event series that explores recent research advances, bold new ideas, and important discussions in the era of general AI: https://aka.ms/ResearchForumRegYTMicrosoft Research Forum | Episode 2 | March 5, 2024Chris Bishop shared the vision for how AI for science will leverage AI to model and predict natural phenomena, including the exciting real-world progress being made by the team.
Connectivity is a thing, is THE thing Microsoft Research 2024-02-29 | The Internet began switching packets on October 29, 1969. Since then, we’ve been awash in connectivity – over five billion people are already on the Internet, mostly mobile, mostly video. Stories about connectivity are full of mission, platform, and traffic evolutions, surprises, and pathologies. The first Ethernet, at the Xerox Palo Alto Research Center, ran over a shared coaxial cable at 2.94Mbps ─ 10,000 times faster than the network it replaced. Why coax and not Wi-Fi in 1973? I’ll tell some of these stories starting when I helped bring up TELNET at MIT on the Arpanet in 1970. At Xerox PARC we helped birth the personal computer by inventing Ethernet for the Internet. My 1979 Internet startup 3Com Corporation used the Silicon Valley ecosystem to help grow the Internet into the billions, and in the mid-1980s to move Silicon Valley from Boston to Palo Alto. Speaker: Robert Metcalfe, Turing Laureate, Emeritus Professor at UT AustinHost: Venkat Padmanabhan, Deputy Managing Director, Microsoft Research India
Meet the 2024 Microsoft Research AI & Society fellows Microsoft Research 2024-01-31 | These esteemed fellows will join researchers at Microsoft to collaborate from all over the world across 13 high impact research challenges. Microsoft recognizes the value of bridging academic, industry, policy, and regulatory worlds and seeks to ignite interdisciplinary collaboration that drives real-world impact. Learn more about the fellows and the research challenges on our program page: microsoft.com/en-us/research/academic-program/ai-society-fellows/fellows
Research Forum: Closing Remarks and Announcements Microsoft Research 2024-01-31 | Microsoft Research Forum - January 2024Ashley Llorens, VP and Distinguished Scientist at Microsoft Research presents closing remarks and announcements at Microsoft Research Forum.See more at https://aka.ms/ResearchForum-Jan2024
Kahani: Visual Storytelling through Culturally Nuanced Images Microsoft Research 2024-01-31 | Microsoft Research Forum, January 30, 2024Sameer Segal, Principal Research Software Development Engineer at Microsoft Research India, discusses Kahani, a research prototype that allows the user to create visually stunning and culturally nuanced images just by describing them in their local languages.See more at https://aka.ms/ResearchForum-Jan2024
Generative AI meets Structural Biology: Equilibrium Distribution Prediction Microsoft Research 2024-01-31 | Microsoft Research Forum, January 30, 2024Shuxin Zheng, Principal Researcher at Microsoft Research AI4Science presents how his team uses generative AI to solve a long-standing challenge in structural biology and molecular science—predicting equilibrium distribution for molecular systems.See more at https://aka.ms/ResearchForum-Jan2024
