Uploaded August 2024 | Updated September 2026, 1 day ago
Abstract: In this talk I will share observations about the role of language in visualization and in other modes of visual expression. I will pose questions such as: how do we decide what to express via language vs via visuals? How do we choose what kind of text to use when creating visualizations, and does that choice matter? Does anyone prefer text over visuals, under what circumstances, and why? Why is visualization ineffective for expressing text content? And what are the ramifications for this combination given the success of Generative AI at creating and analyzing multimodal data?
Bio: Dr. Marti Hearst is a Professor in the School of Information and the Computer Science Division at UC Berkeley. Her research encompasses user interfaces with a focus on search, information visualization with a focus on text, computational linguistics, and educational technology. She is the author of Search User Interfaces, the first academic book on that topic. She is a former President of the Association for Computational Linguistics, a member of the CHI Academy and the SIGIR Academy, an ACM Fellow, an ACL Fellow, and has received four Excellence in Teaching Awards from the students of UC Berkeley. She received her PhD, MS, and BA degrees in Computer Science from UC Berkeley and was a member of the research staff at Xerox PARC.
Abstract: In this talk I will share observations about the role of language in visualization and in other modes of visual expression. I will pose questions such as: how do we decide what to express via language vs via visuals? How do we choose what kind of text to use when creating visualizations, and does that choice matter? Does anyone prefer text over visuals, under what circumstances, and why? Why is visualization ineffective for expressing text content? And what are the ramifications for this combination given the success of Generative AI at creating and analyzing multimodal data?
Bio: Dr. Marti Hearst is a Professor in the School of Information and the Computer Science Division at UC Berkeley. Her research encompasses user interfaces with a focus on search, information visualization with a focus on text, computational linguistics, and educational technology. She is the author of Search User Interfaces, the first academic book on that topic. She is a former President of the Association for Computational Linguistics, a member of the CHI Academy and the SIGIR Academy, an ACM Fellow, an ACL Fellow, and has received four Excellence in Teaching Awards from the students of UC Berkeley. She received her PhD, MS, and BA degrees in Computer Science from UC Berkeley and was a member of the research staff at Xerox PARC.







![Transformers as Soft Reasoners over Language | AI2
Beginning with McCarthys Advice Taker (1959), AI has pursued the goal of providing a system with explicit, general knowledge and having the system reason over that knowledge. However, expressing the knowledge in a formal (logical or probabilistic) representation has been a major obstacle to this research. This paper investigates a modern approach to this problem where the facts and rules are provided as natural language sentences, thus bypassing a formal representation. We train transformers to reason (or emulate reasoning) over these sentences using synthetically generated data. We provide the first empirical demonstration that this kind of soft reasoning over language is learnable and can achieve high (99%) accuracy, and in a way that generalizes to test data requiring substantially deeper chaining than seen during training (95%+ scores). We also demonstrate that the models transfer well to two hand-authored rulebases, and to rulebases paraphrased into more natural language. These findings are significant as it suggests a new role for transformers, namely as limited ``soft theorem provers operating over explicit theories in language. This in turn suggests new possibilities for explainability, correctability, and counterfactual reasoning in question-answering.
[IJCAI20 paper at https://www.ijcai.org/proceedings/2020/537] Transformers as Soft Reasoners over Language | AI2](https://i.ytimg.com/vi/P5KS0qj1eqc/mqdefault.jpg)


