ML4H: Danielle Bitterman, Interrogating the quality of a language model’s clinical knowledge @broadinstitute
ML4H: Danielle Bitterman, Interrogating the quality of a language model’s clinical knowledge  @broadinstitute
Uploaded January 2026 | Updated November 2026, 3 days ago
In this video, Danielle Bitterman (BWH) presents 'Interrogating the quality of a language model’s clinical knowledge'.

Presentation Date: 11/06/24

Abstract: There is incredible enthusiasm about the potential of large language models in healthcare, but evaluating their clinical capabilities remains a barrier to translating them into clinic. In particular, the extent to which performance on widely used knowledge benchmarks translates into safe and useful clinical applications in unknown. In this talk I will highlight my research focused on understanding and anticipating where language models are likely to exhibit unfavorable or unsafe behavior, including risks of bias. I will discuss how information and model’s pretraining data can help us anticipate risks, and discuss standardized approaches to robust model evaluation.

Bio: Danielle Bitterman, M.D. is an Assistant Professor at Harvard Medical School, whose research is dedicated to developing and implementing AI advances for safe, equitable cancer care. She is a physician-scientist in the Artificial Intelligence in Medicine Program at Mass General Brigham, and a radiation oncologist in the Department of Radiation Oncology at Brigham and Women’s Hospital/Dana-Farber Cancer. She completed her post-doctoral fellowship in NLP at the Computational Health Informatics Program and Boston Children’s Hospital. Dr. Bitterman’s lab uses NLP to transform the medical records into systems that actively data-driven care of patients with cancer. Her interests include language model medical knowledge grounding and bias assessments, automated information extraction from the electronic health records, and translational studies of AI in the clinic. Dr. Bitterman’s research has been published in high-impact journals and conference proceedings, including Nature Medicine, Lancet Digital Health, the Journal of Clinical Oncology, NeurIPS, and EMNLP. Her research is funded by the National Cancer Institute, the American Cancer Society, and the American Society for Radiation Oncology.

For more information, visit: broadinstitute.org

Copyright Broad Institute, 2025. All rights reserved.
ML4H: Danielle Bitterman, Interrogating the quality of a language model’s clinical knowledgeML4H: Pierre Elias: Developing, Validating & Deploying an AI Model to Exceed My Clinical ExpertiseTodd Golub on the future of biomedicinegVasc - Full VideoCLEAR 2026: Keynote, Computational Aspects of Causal InferenceMPG Primer: Integrating multimodal data to understand disease heterogeneity (2026)gVasc - Short VersionMachine Learning in Drug Discovery: Ava AminiThe role of intrinsically disordered regions in directing transcription factor binding vast genomes11th Annual Rare Disease Day Event: Ajamete KaykasHow AI and machine learning will transform biomedical research: Caroline UhlerAnna Greka: Core Institute Member #broadinstitute #raredisease #science #genetics
Broad Institute |

ML4H: Danielle Bitterman, Interrogating the quality of a language model’s clinical knowledge

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER