[ASL] Toward Total Scene Understanding for Autonomous Driving—Drago Anguelov (Waymo) Paul G. Allen School 2026-09-23 | [ASL] Toward Total Scene Understanding for Autonomous Driving—Drago Anguelov (Waymo)
Allen School Distinguished Seminar in Optimization and Data: Philippe Rigollet (May 22, 2023) Paul G. Allen School 2023-05-24 | Title: Statistical applications of Wasserstein gradient flowsSpeaker: Philippe Rigollet (MIT)Date: Monday, May 22, 2023Abstract: Otto calculus is a fundamental toolbox in mathematical optimal transport, imparting the Wasserstein space of probability measures with a Riemmanian structure. In particular, one can compute the Riemannian gradient of a functional over this space and, in turn, optimize it using Wasserstein gradient flows. The necessary background to define and compute Wasserstein gradient flows will be presented in the first part of the talk before moving to statistical applications such as variational inference and maximum likelihood estimation in Gaussian mixture models.Bio: Philippe Rigollet is a Professor of Mathematics at MIT. He received his Ph.D. in mathematics from the University of Paris VI. in 2006. His work is at the intersection of statistics, machine learning, and optimization, focusing primarily on the design and analysis of statistical methods for high-dimensional problems. Rigollet’s recent research focuses on statistical optimal transport and its applications to geometric data analysis and sampling. Please advertise it through your department's mailing list, and send it to anyone who may be interested.This video is closed captioned.
Allen School Distinguished Lecture: Pat Hanrahan (May 16, 2023) Paul G. Allen School 2023-05-18 | Title: Shading Languages and the Emergence of Programmable Graphics SystemsSpeaker: Pat Hanrahan (Stanford University)Date: Tuesday, May 16, 2023Abstract: A major challenge in using computer graphics for movies and games is to create a rendering system that can create realistic pictures of a virtual world. The system must handle the variety and complexity of the shapes, materials, and lighting that combine to create what we see every day. The images must also be free of artifacts, emulate cameras to create depth of field and motion blur, and compose seamlessly with photographs of live action.Pixar's RenderMan was created for this purpose and has been widely used in feature film production. A key innovation in the system is to use a shading language to procedurally describe appearance. Shading languages were subsequently extended to run in real-time on graphics processing units (GPUs), and now shading languages are widely used in game engines. The final step was the realization that the GPU is a data-parallel computer, and that the shading language could be extended into a general-purpose data-parallel programming language. This enabled a wide variety of applications in high performance computing, such as physical simulation and machine learning, to be run on GPUs. Nowadays, GPUs are the fastest computers in the world. This talk will review the history of shading languages and GPUs, and discuss the broader implications for computing.Bio: Pat Hanrahan is the Canon Professor of Computer Science and Electrical Engineering in the Computer Graphics Laboratory at Stanford University. His research focuses on rendering algorithms, graphics systems, and visualization.Hanrahan received a Ph.D. in biophysics from the University of Wisconsin-Madison in 1985. As a founding employee at Pixar Animation Studios in the 1980s, Hanrahan led the design of the RenderMan Interface Specification and the RenderMan Shading Language. In 1989, he joined the faculty of Princeton University. In 1995, he moved to Stanford University. More recently, Hanrahan served as a co-founder and CTO of Tableau Software. He has received three Academy Awards for Science and Technology, the SIGGRAPH Computer Graphics Achievement Award, the SIGGRAPH Stephen A. Coons Award, and the IEEE Visualization Career Award. He is a member of the National Academy of Engineering and the American Academy of Arts and Sciences. In 2019, he received the ACM A. M. Turing Award.This video is closed captioned.
Real-time target sound extraction using attention Paul G. Allen School 2023-05-03 | Real-time target sound extraction, ICASSP 2023Bandhav Veluri, University of WashingtonWe present the first neural network model to achieve real-time and streaming target sound extraction. To accomplish this, we propose Waveformer, an encoder-decoder architecture with a stack of dilated causal convolution layers as the encoder, and a transformer decoder layer as the decoder. This hybrid architecture uses dilated causal convolutions for processing large receptive fields in a computationally efficient manner while also leveraging the generalization performance of transformer-based architectures.We provide code, dataset, and audio samples: https://waveformer.cs.washington.edu/.This video is closed captioned.
Winter 2023 Robotics Colloquium: Luca Carlone (MIT) Paul G. Allen School 2023-04-28 | This talk was held on April 21, 2023.Title: Next-Generation Robot Perception: Hierarchical Representations, Certifiable Algorithms, and Self-Supervised LearningAbstract: Spatial perception —the robot’s ability to sense and understand the surrounding environment— is a key enabler for robot navigation, manipulation, and human-robot interaction. Recent advances in perception algorithms and systems have enabled robots to create large-scale geometric maps of unknown environments and detect objects of interest. Despite these advances, a large gap still separates robot and human perception: Humans are able to quickly form a holistic representation of the scene that encompasses both geometric and semantic aspects, are robust to a broad range of perceptual conditions, and are able to learn without low-level supervision. This talk discusses recent efforts to bridge these gaps. First, we show that scalable metric-semantic scene understanding requires hierarchical representations; these hierarchical representations, or 3D scene graphs, are key to efficient storage and inference, and enable real-time perception algorithms. Second, we discuss progress in the design of certifiable algorithms for robust estimation; in particular we discuss the notion of "estimation contracts", which provide first-of-a-kind performance guarantees for estimation problems arising in robot perception. Finally, we observe that certification and self-supervision are twin challenges, and the design of certifiable perception algorithms enables a natural self-supervised learning scheme; we apply this insight to 3D object pose estimation and present self-supervised algorithms that perform on par with state-of-the-art, fully supervised methods, while not requiring manual 3D annotations.Biography: Luca Carlone is the Leonardo Career Development Associate Professor in the Department of Aeronautics and Astronautics at the Massachusetts Institute of Technology, and a Principal Investigator in the Laboratory for Information & Decision Systems (LIDS). He received his PhD from the Polytechnic University of Turin in 2012. He joined LIDS as a postdoctoral associate (2015) and later as a Research Scientist (2016), after spending two years as a postdoctoral fellow at the Georgia Institute of Technology (2013-2015). His research interests include nonlinear estimation, numerical and distributed optimization, and probabilistic inference, applied to sensing, perception, and decision-making in single and multi-robot systems. His work includes seminal results on certifiably correct algorithms for localization and mapping, as well as approaches for visual-inertial navigation and distributed mapping. He is a recipient of the Best Student Paper Award at IROS 2021, the Best Paper Award in Robot Vision at ICRA 2020, a 2020 Honorable Mention from the IEEE Robotics and Automation Letters, a Track Best Paper award at the 2021 IEEE Aerospace Conference, the 2017 Transactions on Robotics King-Sun Fu Memorial Best Paper Award, the Best Paper Award at WAFR 2016, the Best Student Paper Award at the 2018 Symposium on VLSI Circuits, and he was best paper finalist at RSS 2015, RSS 2021, and WACV 2023. He is also a recipient of the AIAA Aeronautics and Astronautics Advising Award (2022), the NSF CAREER Award (2021), the RSS Early Career Award (2020), the Google Daydream (2019), the Amazon Research Award (2020, 2022), and the MIT AeroAstro Vickie Kerrebrock Faculty Award (2020). He is an IEEE senior member and an AIAA associate fellow. At MIT, he teaches “Robotics: Science and Systems,” the introduction to robotics for MIT undergraduates, and he created the graduate-level course “Visual Navigation for Autonomous Vehicles”, which covers mathematical foundations and fast C++ implementations of spatial perception algorithms for drones and autonomous vehicles.
GlucoScreen: A Smartphone-based Readerless Glucose Test Strip for Prediabetes Screening Paul G. Allen School 2023-03-30 | Researchers at the University of Washington’s Paul G. Allen School of Computer Science & Engineering and UW Medicine have developed GlucoScreen, a prototype system that leverages the capacitive touch sensing capabilities of any smartphone to measure blood glucose levels without the need for a separate reader. As with conventional glucose testing, the user applies a drop of blood to a test strip, where it reacts with enzymes on the strip. But instead of requiring a glucometer to process the results, GlucoScreen incorporates inexpensive circuitry that transmits the test data from the strip to a smartphone for processing via simulated taps on the phone's touch screen. The strip's operation is powered entirely by the phone's flash; no batteries or USB connection are required. This novel approach would make prediabetes screening less costly and more accessible — enabling one-time screening on a large scale — and could potentially be expanded to enable smartphone-based screening of other health conditions in the future.This video is closed captioned.
Allen School Distinguished Lecture: Peter Lee (Microsoft Research & Incubations) Paul G. Allen School 2023-03-30 | Title: The Emergence of General AI for MedicineAbstract:Large language models, such as GPT-4 and later more powerful ones, have emerged as powerful new tools for information work, particularly when coupled with chat interfaces as in ChatGPT. These systems are demonstrating impressive capabilities across many domains, and they have the potential to improve health-care delivery and accelerate medical science. In this talk, we will present the results of our intensive year-long study exploring the benefits and risks of applying these systems to medicine. Our findings indicate that these systems may be the most significant technological advance in health care and medicine to date, despite receiving no specialized training in the field. We will showcase examples of how general AI can be used in health care and medicine, and then discuss the implications for the future as these systems continue to evolve, becoming increasingly more intelligent and capable.Bio:Dr. Peter Lee is Corporate Vice President, Research and Incubations, at Microsoft. He leads Microsoft Research across its nine laboratories around the world. He also oversees several incubation teams for new research-powered lines of business, the largest of which today is Microsoft's growing healthcare and life sciences effort. Dr. Lee has extensive experience in managing fundamental research to commercial impact in a range of areas, spanning artificial intelligence, to quantum computing, to biotechnology, and more. Before joining Microsoft in 2010, he was at DARPA, where he established a new technology office that created operational capabilities in machine learning, data science, and computational social science. From 1987 to 2005 he was a Professor at Carnegie Mellon University, and from 2005 to 2008 the Head of the university’s computer science department. Today, in addition to his management responsibilities, Dr. Lee speaks and writes widely on technology trends and policies. He is a member of the National Academy of Medicine. He serves on the Boards of Directors of the Allen Institute for Artificial Intelligence, the Brotman Baty Institute for Precision Medicine, and the Kaiser Permanente Bernard J. Tyson School of Medicine. In public service, Dr. Lee was a commissioner on President Obama's Commission on Enhancing National Cybersecurity and led several studies for both PCAST and the National Academies on the impact of federal research investments on economic growth. He has testified before both the US House Science and Technology Committee and the US Senate Commerce Committee.This Distinguished Lecture was held on March 28, 2023.This video is closed captioned.
I Am CSE: Jiafei Duan Paul G. Allen School 2023-03-14 | Allen School graduate student Jiafei Duan of the Robotics State Estimation Lab and of Ranjay Krishna's lab discusses his research interests into developing a scalable and intuitive approach to collect large-scale data to be used to train a robot.This video is closed captioned.
Distinguished Seminar in Optimization & Data: Éva Tardos (Cornell) Paul G. Allen School 2023-01-30 | Talk title: Stability and Learning in Strategic Queueing SystemsAbstract: Over the last two decades we have developed good understanding of how to quantify the impact of strategic user behavior on outcomes in many games (including traffic routing and online auctions) and showed that the resulting bounds extend to repeated games assuming players use a form of no-regret learning to adapt to the environment. Unfortunately, these results do not apply when outcomes in one round affect the game in the future, as is the case in many applications. In this talk, we study this phenomenon in the context of a game modeling queuing systems: routers compete for servers, where packets that do not get served need to be resent, resulting in a system where the number of packets at each round depends on the success of the routers in the previous rounds. In joint work with Jason Gaitonde, we analyze the resulting highly dependent random process. We find that if the capacity of the servers is high enough to allow a centralized and knowledgeable scheduler to get all packets served even with double the packet arrival rate, then despite selfish behavior of the queues, the expected number of packets in the queues will remain bounded throughout time, assuming older packets have priority. Further, if queues are more patient in evaluating their outcomes , maximizing their long-run success rate, stability can be ensured with just 1.58 times extra capacity, strictly better than what is possible assuming the no-regret property.Bio: Éva Tardos is a Jacob Gould Schurman Professor of Computer Science, currently chair of the Department of Computer Science for a second term after being chair 2006-2010. She was Interim Dean for Computing and Information Sciences 2012-2013 and more recently was Associate Dean for Diversity & Inclusion at Cornell University. She received her BA and PhD from Eötvös University in Budapest. She joined the faculty at Cornell in 1989. Tardos's research interest is algorithms and interface of algorithms and incentives. She is most known for her work on network-flow algorithms and quantifying the efficiency of selfish routing. She has been elected to the National Academy of Engineering, the National Academy of Sciences, the American Philosophical Society, the American Academy of Arts and Sciences, and to the Hungarian Academy of Sciences. She is the recipient of a number of fellowships and awards including the Packard Fellowship, the Gödel Prize, Dantzig Prize, Fulkerson Prize, ETACS prize, and the IEEE von Neumann Medal. She co-wrote the widely used textbook Algorithms Design. She has been editor-in-Chief of the Journal of the ACM and of the SIAM Journal of Computing, and was editor of several other journals, and was program committee member and chair for several ACM and IEEE conferences in her area.
Distinguished Lecture: Telle Whitney (Jan 19, 2023) Paul G. Allen School 2023-01-25 | Title: Technology Innovation and Inclusion: the five Cs – an entrepreneurial journey toward an inclusive futureSpeaker: Telle Whitney (Co-Founder, GHC and NCWIT)Date: Thursday, January 19, 2023Abstract:Technology is rapidly changing our future, but technology creation isn't always welcoming to everyone. Dr. Whitney, a Computer Scientist and world-recognized expert on women and technology, will talk about her journey in technology and the importance of risk-taking at critical junctures. She will speak on how the five Cs - Courage, Confidence, Communication, Curiosity, and Creativity – helped her navigate a constantly evolving landscape. She will discuss how her entrepreneurial journey took a left turn from her career as a technology leader when she became the founding CEO of the Anita Borg Institute and scaled the organization and the Grace Hopper Celebration to have global reach and impact. She will observe and comment on practices adopted by organizations to create inclusive cultures.Bio:Telle Whitney is a senior executive leader, an entrepreneur, and a recognized advocate and expert on women and technology. She has over 20 years of leadership experience and was named one of Fast Company's Most Influential Women in Technology. She is a frequent speaker on the topic of Women and Technology. Telle has been called "a pioneer for the promotion of women technologists" and "one of the most inspirational leaders I have ever known." Telle co-founded the Grace Hopper Celebration of Women in Computing Conference in 1994 and served as CEO of the non-profit Anita Borg Institute from 2002 to September 2017. She transformed the Institute into a recognized world leader for women and technology. She has won numerous awards, including the ACM distinguished service award, an honorary degree from CMU, and is an honorary member of IEEE. She serves on multiple for-profit and non-profit boards. She is also the co-founder of the National Center for Women and Information Technology (NCWIT). She was elected to the National Academy of Engineering in 2022. Telle holds a Ph.D. and M.S. in Computer Science from the California Institute of Technology and a BS in Computer Science from the University of Utah.This video is closed captioned.
Distinguished Lecture: Amin Vahdat (Jan 12, 2023) Paul G. Allen School 2023-01-25 | Title: Reinventing Computing: How necessity will transform next-generation infrastructureSpeaker: Amin Vahdat (Google)Date: Thursday, January 12, 2023Abstract:The rise of scale-out, general-purpose, cluster computing coupled with sustained exponential growth in underlying hardware performance fueled planetary-scale services and connectivity that would have seemed unimaginable 20 years ago. However, the underlying trends fueling this transformation are ending just as we face unprecedented demand in machine learning, data processing, and sovereignty. In this talk, we will review both the history of this transformation and argue that the current computing landscape necessitates novel thinking and invention to fuel the next 1000x growth in capacity and capability, all while supporting a fundamentally different security and reliability posture to support modern societal infrastructure. In particular, we discuss how emerging approaches in i) computing specialization, ii) system rather than component optimization, and iii) new hardware organization, data hierarchies, and programming models are shaping the architectural paradigms for the next epoch of computing.Bio:Amin Vahdat is a Fellow and Vice President of Engineering at Google, where his team is responsible for engineering and product management for Compute (Borg/Cluster Scheduling, and Operating Systems), Platforms (TPUs, Accelerators, Servers, Storage, and Networking), Network Infrastructure (Datacenter, Campus, RPC, and End Host network software), and Google Cloud's Compute, Storage, and Network Products (including Google Compute Engine, Cloud Networking, and Google Cloud Storage), and the Systems Research Group. Until 2019, he was the Area Technical Lead for Networking at Google, responsible for Google's Technical Infrastructure roadmap in collaboration with peers in Compute, Storage, and Hardware. Vahdat is active in the Computer Science systems and networking research communities, with his work being recognized by eight best paper awards and five test of time awards.In the past, he was the SAIC Professor of Computer Science and Engineering at UC San Diego and the Director of UCSD's Center for Networked Systems. Vahdat received his PhD from UC Berkeley in Computer Science, is an ACM Fellow and a past recipient of the NSF CAREER award, UC Berkeley Distinguished EECS Alumni Award, the Alfred P. Sloan Fellowship, the SIGCOMM Networking Systems Award, and the Duke University David and Janet Vaughn Teaching Award. Most recently, Amin was awarded the SIGCOMM lifetime achievement award for his contributions to data center and wide area networks.This video is closed captioned.
Distinguished Lecture: Jenny Lay-Flurrie (Nov 11, 2022) Paul G. Allen School 2023-01-25 | Title: Accessibility at MicrosoftSpeaker: Jenny Lay-Flurrie (Chief Accessibility Officer, Microsoft)Date: Thursday, November 10, 2022, 3:30 pmAbstract:Jenny Lay-Flurrie, Chief Accessibility Officer at Microsoft, discusses Microsoft's journey to bridge the disability divide, the gap in societal inclusion for people with disabilities. Jenny will cover a brief history of accessibility at Microsoft, the evolution of the accessibility industry, inclusive design, and AI and innovation.Bio:Jenny Lay-Flurrie is chief accessibility officer at Microsoft, leading the company's efforts to drive great products, services and websites that empower people and organizations to achieve more. Her team is at the forefront of creating positive experiences that apply technology to make a difference in the world and the lives of individuals, from how we hire and support people with disabilities in employment to innovative technology that aims to revolutionize what’s possible for people with disabilities.With the help of her team and broad community within Microsoft, Lay-Flurrie leads many initiatives to empower people with disabilities both inside and outside of Microsoft. She founded the Disability Employee Resource Group at Microsoft and chaired it for 10 years. She created the Disability Answer Desk, which provides specialist customer support to people with disabilities (over 1M calls handled to date), hosts the annual Microsoft Ability Summit, which focuses on empowering 2000+ attendees with the inclusive and innovative thinking necessary to enable people around the world. Instrumental in projects such as Autism Hiring Program, Soundscape and the Microsoft Ability Hackathon, which has supported over 500 hackathon teams building technology for people with disabilities. Lay-Flurrie was recognized as nominated as a technology ground breaker by CEO Satya Nadella in Wired Magazine, and is a contributor to the book, "The Ability Hacks," which shares behind-the-scenes stories of the hackers who pioneered two innovative hack-turned-solutions used by people with disabilities worldwide.Outside of Microsoft, Jenny is on the board of Gallaudet University and Team Gleason. She was recognized as a 'Disability Employment Champion of Change' by the White House in October 2014 and as one of Fast Companies' Most Creative people in business in 2017.Twitter: @jennylayfluffy. Microsoft Bio Story (news.microsoft.com/stories/people/jenny-lay-flurrie.html)This Distinguished Lecture is co-hosted by the Allen School and CREATE.This video is close captioned.
The Power of Gradients in Inverse Dynamics Problems (Tao Du, MIT) Paul G. Allen School 2022-12-16 | Abstract: Traditionally, inverse dynamics refers to the problem of reconstructing the forces in a dynamic system from its kinematic motion and has wide applications in robotics, biomechanics, and computer graphics. This talk considers a more broadened definition of inverse dynamics that infers various computational design parameters of rigid-body, deformable-body, and fluidic dynamic systems. To solve this challenging problem, we develop a series of computational tools that unleash the full power of analytical gradients from a physics simulator in many non-traditional ways. First, we demonstrate the usage of gradients in exploring the shape and controller design space of rigid and soft robots. Next, we discuss transferring these computational designs to hardware and show the power of gradients in constructing digital twins of two such real-world robots: a rigid-body quadrotor and a deformable-body underwater robot. We end this talk by envisioning future opportunities for physics simulation gradients in computational fabrication, robotics, and machine learning.Biography: Tao Du is a Postdoctoral Associate at MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), working with Professor Wojciech Matusik and Professor Daniela Rus. His research aims to combine physics simulation, machine learning, and numerical optimization techniques to solve real-world inverse dynamics problems. His representative works include building differentiable simulation platforms for graphics and robotics research, developing computational design pipelines for real-world robots, and understanding the simulation-to-reality gap of dynamic systems. His work has been published in top-tier graphics, learning, and robotics journals and conferences and has been featured by major technical media outlets. Before continuing at MIT as a Postdoctoral Associate, Tao Du obtained his Ph.D. in Computer Science from MIT in 2021 and his Master's in Computer Science from Stanford in 2015.
Allen School Colloquium: Sara Beery (CalTech) Paul G. Allen School 2022-12-15 | **This video will be closed captioned.**Tuesday, April 5, 2022, 3:30 pmAbstractWe require a real-time, modular earth observation system that unites efforts across research groups in order to provide the vital information necessary for global-scale impact in sustainability and conservation in the face of climate change. The development of such systems requires collaborative, interdisciplinary approaches that translate diverse sources of raw information into accessible scientific insight. For example, we need to monitor species in real time and in greater detail to quickly understand which conservation efforts are most effective and take corrective action. Current ecological monitoring systems generate data far faster than researchers can analyze it, making scaling up impossible without automated data processing. However, ecological data collected in the field presents a number of challenges that current methods, like deep learning, are not designed to tackle. These include strong spatiotemporal correlations, imperfect data quality, fine-grained categories, and long-tailed distributions. My work seeks to overcome these challenges, and includes methods which can learn from imperfect data, systematic frameworks for measuring and overcoming performance drops due to domain shift, and the deployment of efficient human-AI systems that have made significant real-world conservation impact. My future research agenda will expand upon the strong foundation built by my past and current research. It will seek to make effective use of all available modalities of data, incorporate expert knowledge systematically, and ensure these systems are equitable and ethical – all fundamental and unresolved challenges for CV&ML.BioSara Beery is a final-year PhD Candidate in Computing and Mathematical Sciences at Caltech, advised by Pietro Perona. She has always loved the natural world and has seen a growing need for technology-based approaches to conservation and sustainability challenges. Her research focuses on building computer vision methods that enable efficient, accessible, and equitable global-scale biodiversity monitoring. She was honored to be awarded both the PIMCO Data Science Fellowship and the Amazon AI4Science Fellowship, which recognize senior graduate students that have had a remarkable impact in machine learning and data science, and in their application to fields beyond computer science. Her work is funded in part by an NSF Graduate Research Fellowship and the Caltech Resnick Sustainability Institute. She seeks to break down knowledge barriers between fields: she founded the successful AI for Conservation slack community (with over 650 members), and she is the founding director of the Caltech Summer School on Computer Vision Methods for Ecology. She works closely with Microsoft AI for Earth, Google Research, and Wildlife Insights where she helps turn her research into usable tools for the ecological community. Sara's experiences as a professional ballerina, a nontraditional student, and a queer woman have taught her the value of unique and diverse perspectives, both inside and outside of the research community. She is passionate about increasing diversity and inclusion in STEM through mentorship, teaching, and outreach.
Allen School Colloquium: Xinyun Chen (UC Berkeley) Paul G. Allen School 2022-12-15 | **This video will be closed captioned.**Thursday, March 31, 2022, 3:30 pmAbstractWith the advancement of modern technologies, programming becomes ubiquitous not only among professional software developers, but also for general computer users. However, gaining programming expertise is time-consuming and challenging. Therefore, program synthesis has many applications, where the computer automatically synthesizes programs from specifications such as natural language descriptions and input-output examples. In this talk, I will present my work on learning-based program synthesis, where I have developed deep learning techniques for various program synthesis problems. Despite the remarkable success of deep neural networks for many domains, including natural language processing and computer vision, existing deep neural networks are still insufficient for handling challenging symbolic reasoning and generalization problems.My learning-based program synthesis research lies in two folds: (1) learning to synthesize programs from potentially ambiguous and complex specifications; and (2) neural-symbolic learning for language understanding. I will first talk about program synthesis applications, where my work demonstrates the applicability of learning-based program synthesizers for production usage. I will then present my work on neural-symbolic frameworks that integrate symbolic components into neural networks, which achieve better reasoning and generalization capabilities. In closing, I will discuss the challenges and opportunities of further improving the complexity and generalizability of learning-based program synthesis for future work.BioXinyun Chen is a Ph.D. candidate at UC Berkeley, working with Prof. Dawn Song. Her research lies at the intersection of deep learning, programming languages, and security. Her recent research focuses on learning-based program synthesis and adversarial machine learning. She received the Facebook Fellowship in 2020, and Rising Stars in Machine Learning in 2021. Her work SpreadsheetCoder for spreadsheet formula prediction was integrated into Google Sheets, and she was part of the AlphaCode team when she interned at DeepMind.
Allen School Colloquium: Natasha Jaques (Google + UC Berkeley) Paul G. Allen School 2022-12-15 | Monday, April 4, 2022, 3:30 pmAbstractSocial learning helps humans and animals rapidly adapt to new circumstances, coordinate with others, and drives the emergence of complex learned behaviors. What if it could do the same for AI? This talk describes how Social Reinforcement Learning in multi-agent and human-AI interactions can address fundamental issues in AI such as learning and generalization, while improving social abilities like coordination. I propose a unified method for improving coordination and communication based on causal social influence. I then demonstrate that multi-agent training can be a useful tool for improving learning and generalization. I present PAIRED, in which an adversary learns to construct training environments to maximize regret between a pair of learners, leading to the generation of a complex curriculum of environments. Agents trained with PAIRED generalize more than 20x better to unknown test environments. Ultimately, the goal of my research is to create intelligent agents that can assist humans with everyday tasks; this means leveraging social learning to interact effectively with humans. I show that learning from human social and affective cues scales more effectively than learning from manual feedback. However, it depends on accurate recognition of such cues. Therefore I discuss how to dramatically enhance the accuracy of affect detection models using personalized multi-task learning to account for inter-individual variability. Together, this work argues that Social RL is a valuable approach for developing more general, sophisticated, and cooperative AI, which is ultimately better able to serve human needs.BioNatasha Jaques holds a joint position as a Senior Research Scientist at Google Brain and Visiting Postdoctoral Scholar at UC Berkeley. Her research focuses on Social Reinforcement Learning in multi-agent and human-AI interactions. Natasha completed her PhD at MIT, where her thesis received the Outstanding PhD Dissertation Award from the Association for the Advancement of Affective Computing. Her work has also received Best Demo at NeurIPS, an honourable mention for Best Paper at ICML, Best of Collection in the IEEE Transactions on Affective Computing, and Best Paper at the NeurIPS workshops on ML for Healthcare and Cooperative AI. She has interned at DeepMind, Google Brain, and was an OpenAI Scholars mentor. Her work has been featured in Science Magazine, Quartz, IEEE Spectrum, MIT Technology Review, Boston Magazine, and on CBC radio. Natasha earned her Masters degree from the University of British Columbia, and undergraduate degrees in Computer Science and Psychology from the University of Regina.
Allen School Colloquium: Azalia Mirhoseini (Google) Paul G. Allen School 2022-12-15 | Wednesday, March 30, 2022, 3:30 pmAbstractOver the past decade, advances in both deep learning and computer systems have prepared AI to revolutionize our world. To fuel the next generation of AI breakthroughs, my research goal is to learn to optimize the design and evolution of computational systems. In this talk, I will describe my work on 1) learning-based methods for systems design automation, and 2) systems-inspired neural architecture design for scaling large machine learning models. First, I will discuss a deep reinforcement learning method to perform hardware mapping and model parallelism for large neural networks. Our algorithm learns the implicit tradeoffs between memory, computation, and communication of the underlying computing platform and finds optimized and transferable mapping strategies. This work inspired a new and ongoing trend of policy gradient methods for solving combinatorial optimization in computer systems. Next, I will describe a generalizable deep reinforcement learning method for chip floorplanning, which is a long pole in the overall chip design process. In under 6 hours, our method can generate floorplans that are superhuman or comparable on modern chips, whereas existing baselines require human experts in the loop and can take several weeks. This method was recently published in Nature and is used in production to generate chip layouts for next-generation AI accelerators and Pixel phones at Google. I will then discuss a framework on HW/SW co-design for deep learning accelerators. Currently, improving Perf/TCO by 2x per accelerator generation is considered a great success. The proposed learning-based method automatically searches over a combinatorially large space of datapath, software schedule, and compiler optimization passes to generate accelerators that are up to 6x better in Perf/TCO than existing baselines. Finally, I will discuss our work on the sparsely gated mixture of experts, a conditional neural network architecture that allows scalable training of models with 100B+ parameters on datasets with 100B+ examples. This architecture uses an intelligent gating mechanism that routes input examples to a subset of the modules (“experts”) within the larger model. It set a new state of the art in machine translation and language modeling and enabled 2-3x faster runtime than top-performing baselines. I will conclude my talk by discussing how we can move towards a future where computers learn to evolve and optimize themselves.BioAzalia Mirhoseini is a Staff Research Scientist and Team Lead at Google Brain. She is the co-founder of the Machine Learning for Systems Team, a larger research effort focused on developing advanced learning-based methods to design next generation computer systems and hardware. Azalia has published more than 40 peer-reviewed papers at scientific venues such as Nature, ICML, ICLR, NeurIPS, UAI, ASPLOS, SIGMETRICS, DAC, DATE, and ICCAD. She has a Ph.D. in Electrical and Computer Engineering from Rice University. She has received a number of awards, including the MIT Technology Review 35 under 35 award, the Best Ph.D. Thesis Award at Rice and a Gold Medal in the National Math Olympiad in Iran. Her work has been covered in various media outlets including CNBC, ABC News, MIT Technology Review and IEEE Spectrum.
Allen School Colloquium: Gilbert Bernstein (UC Berkeley + MIT) Paul G. Allen School 2022-12-15 | Tuesday, March 29, 2022, 3:30 pmAbstractComputer Graphics and Visual Computing problems challenge us with a near inexhaustible demand for more resolution and scale in order to simulate the climate, reconstruct 3d environments, train neural networks, and produce games & movies. Building such applications requires integrating a wide range of disciplinary expertise (e.g. physics, numerical methods, geometry and custom hardware) into a single system. However, abstraction barriers are regularly discarded in the name of higher-performance, leading to code that must be written and maintained by super-experts—programmers who simultaneously possess deep knowledge of all relevant disciplines. Programming languages, especially Domain Specific Languages (DSLs) are perhaps the most promising approach to recovering a separation of concerns in such high-performance systems.In this talk, I will first describe my work on DSLs to enable parallel portability of physical simulation and optimization programs, drawing on ideas from databases and automatic differentiation to structure these problem domains. Then I will discuss more recent work on “horizontal DSLs” designed to address concerns that cut across multiple application domains, such as automatic differentiation of discontinuous functions, and programming language support for maximizing the utilization of new hardware accelerators.BioGilbert Bernstein is a Postdoctoral Scholar at the University of California, Berkeley and MIT CSAIL, working with Professor Jonathan Ragan-Kelley. His research lies in Computer Graphics and Programming Languages, especially the design of high-performance domain specific languages for numeric computing applications such as physical simulation, optimization and inverse problems. His work spans the gamut from user interfaces, to differentiable programming, parallel-portability, and new hardware design languages. His work has been published at SIGGRAPH, POPL, PLDI, & OOPSLA, as well as being incorporated into products at Adobe, Autodesk, and Disney. He holds a Ph.D. in Computer Science from Stanford University, where he was advised by Pat Hanrahan.
Allen School Colloquium: Yang Song (Stanford) Paul G. Allen School 2022-12-15 | Monday, March 28, 2022, 3:30 pmAbstractGenerating data with complex patterns, such as images, audio, and molecular structures, requires fitting very flexible statistical models to the data distribution. Even in the age of deep neural networks, building such models is difficult because they typically require an intractable normalization procedure to represent a probability distribution. To address this challenge, I propose to model the vector field of gradients of the data distribution (known as the score function), which does not require normalization and therefore can take full advantage of the flexibility of deep neural networks. I will show how to (1) estimate the score function from data with flexible deep neural networks and efficient statistical methods, (2) generate new data using stochastic differential equations and Markov chain Monte Carlo, and even (3) evaluate probability values accurately as in a traditional statistical model. The resulting method, called score-based generative modeling, achieves record-breaking performance in applications including image synthesis, text-to-speech generation, time series prediction, and point cloud generation, challenging the long-time dominance of generative adversarial networks (GANs) on many of these tasks. Furthermore, unlike GANs, score-based generative models are suitable for Bayesian reasoning tasks such as solving ill-posed inverse problems, and I have demonstrated their superior performance on sparse-view computed tomography and accelerated magnetic resonance imaging. Finally, I will discuss my future research plan on improving the controllability and generalization of generative models, as well as their broader impacts on machine learning, science & engineering, and society.BioYang Song is a final year Ph.D. student at Stanford University. His research interest is in deep generative models and their applications to inverse problem solving and AI safety. His first-author papers have been recognized with an Outstanding Paper Award at ICLR-2021, and an oral presentation at NeurIPS-2019. He is a recipient of the Apple PhD Fellowship in AI/ML, and the J.P. Morgan PhD Fellowship.
Allen School Colloquium: Ryan Marcus (MIT) Paul G. Allen School 2022-12-15 | Thursday, March 17, 2022, 3:30 pmAbstractDatabase management systems (DBMSes) depend on query optimizers to transform a user's declarative query into an efficient execution plan. Query optimizers are critical because a bad query plan can be orders of magnitude slower than the optimal plan. Modern query optimizers are complex and expensive to maintain, as they integrate a wide range of hand-tuned heuristics and manually-engineered cost models which must be updated for every new capability added to the DBMS. I will present two recent approaches to query optimization that leverage deep reinforcement learning to simultaneously improve query performance and decrease maintenance burden. The first approach, Neo (VLDB 19), combines tree convolution neural networks with a novel value iteration technique to fully replace a traditional query optimizer, yielding as much as 2x improvements after just 36 hours of training on stable workloads. The second approach, Bao (SIGMOD 21), targets dynamic workloads, and learns to "steer" an existing query optimizer by training an agent via a contextual multi-armed bandit framework. More broadly, both Neo and Bao highlight the huge potential impact of applying machine learning to systems problems, giving us a glimpse of what a fully learned system could do, as well as highlighting several potential pitfalls along the way.BioRyan Marcus is a postdoc at MIT, where he researches learned systems. Ryan focuses on the potential of machine learning to underpin the next generation of data management systems, especially query optimization, data storage, and indexing. Before MIT, Ryan received his PhD from Brandeis University, where he studied machine learning techniques for automating cloud data management systems. Ryan is also a scientist at Intel Labs, an avid World of Warcraft player, and generally amenable to every kind of snack you could imagine.
Allen School Colloquium: Divya Mahajan (Microsoft) Paul G. Allen School 2022-12-15 | Monday, March 14, 2022, 3:30 pmAbstractAdvances in Artificial Intelligence (AI) and Machine Learning (ML) are beginning to revolutionize medicine, manufacturing, commerce, transportation, and other key aspects of our lives. However, such transformative effects are predicated on providing high-performance compute capabilities to enable these learning algorithms. Domain specific accelerators are an efficient and performant means to meet the compute requirements of these large-scale AI/ML. As the new age data-centers become heterogeneous with these emerging domain specific hardware, we must rethink both the architecture and the corresponding system stack. In this talk, I will provide an overview of my contributions to design, deploy, and utilize accelerators for a wide class of AI/ML applications. I will first discuss pioneering works TABLA and DaNA, which are comprehensive full-stack solutions for machine learning accelerators that integrate with data management systems. These solutions expose a high-level programming interface to programmers that have limited knowledge about hardware design, nevertheless, can benefit from performance and efficiency gains through acceleration. Then, I will describe FAE, a novel framework that leverages statistical properties of data to best utilize the heterogeneous compute and memory resources for recommender model training. Finally, I will conclude with my future research vision towards devising architectures and systems for sustainable massive-scale distributed AI/ML by exploring the challenges which arise from the cross-pollination of different components in the data processing pipeline.BioDivya Mahajan is a Senior Researcher in the Cloud Accelerated Systems & Technologies group at Microsoft. She leads the research, design, and deployment of communication primitives for massive-scale distributed deep learning. She obtained her PhD in Computer Science from Georgia Institute of Technology. She obtained her Masters from The University of Texas Austin, Texas and Bachelors from Indian Institute of Technology Ropar. Her research interests lie in designing novel architectures and building robust systems to address the needs of new and emerging applications. She is passionate about continuing innovative research to have a broad impact on computing and society in general. Divya is the recipient of National Council for Women and Information Technology Collegiate Award, President of India Gold Medal at IIT, and has been a Finalist in the Qualcomm Innovation Fellowships. Her work has been recognized with the College of Computing Dissertation Award, HPCA Distinguished Paper Award, and has appeared in top architecture, database, systems, and machine learning venues like ISCA, MICRO, HPCA, ASPLOS, VLDB, NeurIPS and high impact journals like IEEE Micro. microsoft.com/en-us/research/people/divyam
Allen School Colloquium: Ram Alagappan (VMware Research) Paul G. Allen School 2022-12-15 | Thursday, March 10, 2022, 3:30 pmAbstractDistributed storage systems form the core of modern cloud services. Like many systems software, these systems are built using layering: designers layer distributed protocols (e.g., Paxos, 2PC) upon local storage stacks. Such layering abstracts details about the local storage stack to the layers above, easing development. I will show that such black-box layering, unfortunately, masks vital information, resulting in poor reliability. I will then demonstrate that reliability can be significantly improved by co-designing these layers.In the first half of the talk, I will show how local storage-layer faults in one node can lead to global data loss, corruption, and unavailability in many widely used systems. I then present CTRL, a new foundation that uses the co-design approach to avoid such problems, improving reliability. I implement CTRL in two practical systems and show that CTRL greatly improves resiliency to storage faults while incurring little performance overhead.BioRam Alagappan is a postdoctoral researcher at the VMware Research Group. He earned his Ph.D., working with Professors Andrea Arpaci-Dusseau and Remzi Arpaci-Dusseau at the University of Wisconsin - Madison. His work has been published at top systems venues and has won three best paper awards (FAST 17, 18, and 20). His dissertation also won an honorable mention for the UW CS Best Dissertation. His open-source frameworks have had a practical impact: these tools have exposed more than 80 severe vulnerabilities across 20 widely used systems. Ideas from his work have been adopted by a financial database to make it resilient
Allen School Colloquium: Aishwarya Ganesan (VMware Research) Paul G. Allen School 2022-12-15 | Tuesday, March 8, 2022, 3:30 pmAbstractComputer systems underpin every modern application that we interact with today. When designing systems, one must often tradeoff strong guarantees for performance or vice-versa. The same tradeoff exists in distributed storage systems as well; designers must often choose consistency or performance. In this talk, I will show how we can build distributed storage systems that provide strong consistency yet also perform well. My key insight to achieving this goal is to defer enforcing consistency until state is externally visible. Based on this insight, I design two novel distributed storage systems. First, I present Skyros, a new replication protocol that exploits storage-interface properties to defer expensive coordination. Skyros realizes that many update interfaces are nil-externalizing: they do not expose system state immediately. By taking advantage of nil-externality, Skyros offers significantly lower latencies than traditional replication protocols while still providing strong consistency. Second, I present consistency-aware durability (CAD), a new durability primitive that enables stronger consistency. CAD shifts the point of durability from writes to reads. By delaying writes, CAD enables high performance; however, by ensuring durability before serving reads, CAD enables the construction of stronger consistency models.Bio:Aishwarya Ganesan is a postdoctoral researcher at VMware Research. She completed her PhD from the University of Wisconsin - Madison in Computer Sciences in 2020, advised by Andrea Arpaci-Dusseau and Remzi Arpaci-Dusseau. She is broadly interested in distributed systems and storage systems. Her work has been recognized with best-paper awards at FAST 20 and FAST 18 and a best paper award nomination at FAST 17. She was selected for the Rising Stars in EECS workshop and a recipient of Facebook 2019 PhD Fellowship. She also received the graduate student instructor award for teaching graduate-level distributed systems at UW Madison.
Allen School Colloquium: Benedikt Bünz (Stanford) Paul G. Allen School 2022-12-15 | Monday, March 7, 2022, 3:30 pmAbstractBlockchains are an exciting area of research that touches on many areas of Computer Science and beyond. This technology has the potential to enable a fast, cheap, and private financial system based on distributed consensus and cryptography, instead of trusted parties. Despite this potential, the reality still shows severe limitations of blockchains: (i) transactions can cost hundreds of dollars and take minutes to confirm, (ii) some blockchains offer little privacy, and (iii) proof-of-work consensus consumes too much energy. In this talk, I will discuss powerful techniques that follow a prover paradigm and can mitigate these limitations. The first technique, called Bulletproofs, is a general-purpose zero-knowledge proof system that is specifically designed to enable confidential blockchain transactions. Bulletproofs requires minimal trust assumptions and gives the shortest zero-knowledge proofs without a trusted setup. The system is widely deployed and powers tens of thousands of private blockchain transactions per day. The second technique, called inner pairing products, is a way to aggregate many zero knowledge proofs into a single short proof. This can significantly reduce on-chain data, leading to a significant increase in transactions per second that the chain can process. The third technique is a new concept called a verifiable delay function (VDF) that is vital for permission-less and eco-friendly consensus. VDFs are already deployed in Filecoin and Chia, and are planned for the upcoming upgrade to Ethereum.BioBenedikt Bünz is a PhD candidate at Stanford University, a member of Dan Boneh’s applied cryptography lab, and a recipient of the Microsoft Research Fellowship at the Simons Institute. His work on the science of Blockchains uses tools from applied cryptography, distributed systems, and algorithmic game theory. His research focuses on building new proof protocols for improving the privacy, scalability, and ecological impact of blockchains. Several of his research results have had a significant industry impact. His work on Bulletproofs, secures tens of thousands of private transactions on Blockchains like Monero or Signal’s Mobilecoin. His seminal work on Verifiable Delay Functions (VDFs) sparked the VDF Alliance, a multi-million dollar initiative composed of academic, non-profit, and corporate collaborators.
Allen School Colloquium: Alex Lombardi (MIT) Paul G. Allen School 2022-12-15 | Monday, February 28, 2022, 3:30 pmAbstractOne of computer science's greatest insights has been in understanding the power and versatility of *proofs*, which were revolutionized in the 1980s to utilize *interaction* as well as other resources such as randomization and computational hardness. Today, they form the backbone of both theoretical and practical cryptography and are simultaneously the source of deep connections to areas such as complexity theory, game theory, and quantum computation.In this talk, I will introduce general-purpose tools, techniques, and abstractions for two key aspects of cryptographic proof systems that have been poorly understood for decades:1) Can we remove interaction from interactive proofs? Already in the 1980s, Fiat and Shamir proposed a heuristic *but unproven* methodology for removing interaction from interactive proofs, which is now ubiquitous and essential for practical applications. However, it remained open for over 30 years to prove the security of this transformation in essentially any setting of interest.My work on the Fiat-Shamir transform has led to resolutions to many long-standing open problems, including (i) building non-interactive zero knowledge proof systems based on lattice cryptography, (ii) establishing the existence of highly efficient and succinct non-interactive proof systems, and (iii) demonstrating that foundational protocols from the 80s fail to compose in parallel.2) Are classical interactive protocols secure against quantum computers? At its heart, the problem of analyzing and ruling out quantum attacks on cryptographic protocols is the issue of “rewinding.” The inability to rewind a quantum attack stems from the no-cloning theorem, a fundamental property of quantum information. As a result, very few classical protocols were known to be secure against quantum attacks.In a recent work, I showed how to overcome these difficulties and settle many foundational questions on post-quantum cryptographic proof systems. Our main technique is showing how to efficiently extract certain pieces of (classical) information from a quantum attacker without disturbing its internal state.BioAlex Lombardi is a graduate student at MIT advised by Vinod Vaikuntanathan. He is broadly interested in cryptography and theoretical computer science with a focus on cryptographic proof systems and post-quantum security.
Allen School Colloquium: Andrea Wei Coladangelo (UC Berkeley) Paul G. Allen School 2022-12-15 | Thursday, February 24, 2022, 3:30 pmAbstractQuantum computers will reshape the landscape of cryptography. On the one hand, they threaten the security of most modern cryptosystems. On the other, they offer fundamentally new ways to realize tasks that were never before thought to be possible. In this talk, I will explore the interplay between quantum computation and cryptography, and the many exciting questions at this intersection. I will describe examples that leverage quantum computers to protect against coercion in online elections, and to prevent piracy of software.BioAndrea Coladangelo is a postdoctoral fellow at the Simons Institute for the Theory of Computing, working with Umesh Vazirani. He obtained his PhD from Caltech under the supervision of Thomas Vidick. In 2020, Andrea co-founded qBraid, a startup that makes learning and developing quantum algorithms more accessible. Andrea is the recipient of several awards, including the best student paper prize at QIP 2019, and the Bhansali Family Doctoral prize in Computer Science for his work on quantum correlations and entanglement.
Allen School Colloquium: Pang Wei Koh (Stanford University) Paul G. Allen School 2022-12-12 | Tuesday, February 22, 2022, 3:30 pmAbstractMachine learning systems are widely deployed today, but they are unreliable. They can fail – and with catastrophic consequences – on subpopulations of the data, such as particular demographic groups, or when deployed in different environments from what they were trained on. In this talk, I will describe our work towards building reliable machine learning systems that are robust to these failures. First, I will show how we can use influence functions to understand the predictions and failures of existing models through the lens of their training data. Second, I will discuss the use of distributionally robust optimization to train models that perform well across all subpopulations. Third, I will describe WILDS – a benchmark of in-the-wild distribution shifts spanning applications such as pathology, conservation, remote sensing, and drug discovery – and show how current state-of-the-art methods, which perform well on synthetic distribution shifts, still fail to be robust on these real-world shifts. Finally, I will describe our work on building more reliable COVID-19 models, using anonymized cellphone mobility data, to inform public health policy; this is a challenging application as the underlying environment is often changing and there is substantial heterogeneity across demographic subpopulations.BioPang Wei Koh is a PhD student at Stanford, advised by Percy Liang. He studies the theory and practice of building reliable machine learning systems. His research has been published in Nature and Cell, featured in media outlets such as The New York Times and The Washington Post, and recognized by best paper awards at ICML and KDD, a Meta Research PhD fellowship, and the Kennedy Prize for best honors thesis at Stanford. Prior to his PhD, he was the 3rd employee and Director of Partnerships at Coursera.
Allen School Colloquium: Matthew Golub (Stanford) Paul G. Allen School 2022-12-12 | Tuesday, February 15, 2022, 3:30 pmAbstractBehavior and cognition are driven by the coordinated activity of populations of neurons in the brain. A major challenge in systems neuroscience is to infer the computational principles underlying the activity of the neural populations. What are the algorithms implemented by these neural populations? How can we design experiments and analyses with hypotheses about computation in mind? My research group will develop the theory, modeling, and machine learning techniques needed to realize this vision of “reverse engineering” computation in the brain. Progress in this research could lead to new insights into treating neurological injuries and disorders, new paradigms for optimizing our behavior and cognition, and new approaches to generating artificial intelligence.In this talk, I will present lines of previous, ongoing, and proposed research that highlight the potential of this vision. First, I will present a line of brain-computer interface experiments and modeling that revealed principles guiding neural populations as they reorganize during learning. Here, dimensionality reduction and convex optimization provided insight into the constraints faced by neural populations. Second, I will present a framework of network modeling for identifying the computations performed by a population of recorded neurons. Here, we trained sequential variational autoencoders to learn nonlinear dynamical systems (NLDS) capable of generating observed single-trial neural population activity. We then developed techniques for identifying the computations performed through the dynamics of the NLDS. Finally, I will propose future directions for leveraging these tools and identified principles of neural computation toward i) accelerating the brain’s ability to learn, ii) optimizing optogenetic control of neural population dynamics, and iii) understanding the interplay between attention and decision making in the brain.BioMatthew Golub is a Postdoctoral Fellow in Electrical Engineering at Stanford University. His research interests are at the intersection of machine learning, neuroengineering, and basic systems neuroscience. His current projects focus on interpreting nonlinear dynamical systems models of neural population activity underlying decision making processes in the brain. This work has been recognized by a Pathway to Independence Award from the National Institutes of Health. Previously, Matthew received his PhD in Electrical and Computer Engineering (ECE) from Carnegie Mellon University, where he developed brain-computer interfaces as a scientific paradigm for investigating the neural bases of learning and feedback motor control. His thesis was recognized by the ECE Department’s Best Thesis Award.
Allen School Colloquium: Daniel Grier (University of Waterloo) Paul G. Allen School 2022-12-12 | Monday, February 14, 2022, 3:30 pmAbstractQuantum computing is at an exciting moment in its history, with some high-profile experimental successes in building programmable quantum devices. That said, these quantum devices (at least in the near term) will be restricted in several ways, raising questions about their power relative to classical computers. In this talk, I will present three results which give us a better understanding of these near-term quantum devices, revealing key features which make them superior to their classical counterparts.First, I will show that constant-depth quantum circuits can solve problems that cannot be solved by any constant-depth classical circuit consisting of AND, OR, NOT, and PARITY gates---giving the largest-known unconditional separation between natural classes of quantum and classical circuits. Second, I will show that these quantum circuits can nevertheless be simulated quickly by classical algorithms that have no depth restriction, emphasizing the role that depth plays in provable quantum advantage. Finally, I will address some of the experimental challenges in implementing linear optical quantum computers, and I will prove that they outperform classical computers using standard conjectures but in more practical experimental regimes.BioDaniel is a postdoctoral researcher at the Institute for Quantum Computing at the University of Waterloo. He received his PhD in Computer Science at MIT, where he was advised by Scott Aaronson and was supported by an NSF Graduate Research Fellowship. His research lies at the intersection of complexity theory and quantum computing, with a particular focus on the power of near-term quantum computing devices.
Emeritus Lecture: Oren Etzioni (Oct 2022) Paul G. Allen School 2022-11-09 | What I learned: stories and lessons from an academic lifetimeThursday, October 27, 2022, 3:30 pmAbstractIn high school, I aspired to be a standup comic, but I walked into the wrong bar and ended up studying computer science. Over the last 40 years, I had some wild rides and contributed to the Allen School's illustrious history. I plan to share some amusing highlights with you as parables whose morals are broadly applicable.BioDr. Oren Etzioni is the founding CEO of the Allen Institute for AI (AI2). He now serves as Technical Director of the AI2 Incubator. He is Professor Emeritus, University of Washington as of October 2020 and a Venture Partner at the Madrona Venture Group. His awards include AAAI Fellow, Seattle’s Geek of the Year, and several Test of Time and Best Paper awards. He has founded several companies including Farecast (acquired by Microsoft). He has written over 200 technical papers, as well as commentary on AI for The New York Times, Wired, and Nature. He helped to pioneer meta-search, online comparison shopping, machine reading, and Open Information Extraction.This talk was given on October 27, 2022.This video is close captioned.
Emeritus Lecture: Pedro Domingos (Oct 2022) Paul G. Allen School 2022-11-08 | Machine Learning: The Last 20 Years and the NextPedro Domingos (Emeritus - University of Washington)AbstractThe last 20 years were eventful ones in machine learning, and the Allen School played a big part. In this talk I will briefly look at seven fields we helped start or grow: massive-scale learning, adversarial learning, influence maximization in social networks, machine learning for data integration, statistical relational learning, symmetry-based learning, and deep learning. In each case I will tell the story of how it came about, summarize the main results, and lay out today's challenges and research frontiers. I will also touch on how we helped develop machine learning education and popularize it into the technological, economic and cultural force it is today, and speculate on where it might be headed.BioPedro Domingos is Professor Emeritus of Computer Science & Engineering at the University of Washington's Allen School, and the author of The Master Algorithm, the worldwide bestseller introducing machine learning to a broad audience. He is a winner of the SIGKDD Innovation Award and the IJCAI John McCarthy Award, two of the highest honors in data science and AI, and a Fellow of AAAS and AAAI. His papers and systems have won ten awards at major AI conferences. He co-founded the International Machine Learning Society in 2001.This lecture was originally given on October 20, 2022.This video is close captioned.
PLSE Research Group (October 2022) Paul G. Allen School 2022-11-08 | Various Presenters (Allen School)Originally presented on Thursday, October 6, 2022This video is close captioned.Speaker: Oliver FlattTitle: Small Proofs from Congruence ClosureSatisfiability Modulo Theory (SMT) solvers and equality saturation engines must generate proof certificates from e-graph-based congruence closure procedures to enable verification and conflict clause generation. Smaller proof certificates speed up these activities. Though the problem of generating proofs of minimal size is known to be NP-complete, existing proof minimization algorithms for congruence closure generate unnecessarily large proofs and introduce asymptotic overhead over the core congruence closure procedure. In this paper, we introduce an O(n^5) time algorithm which generates optimal proofs under a new relaxed "proof tree size" metric that directly bounds proof size. We then relax this approach further to a practical O(n \log(n)) greedy algorithm which generates small proofs with no asymptotic overhead. We implemented our techniques in the egg equality saturation toolkit, yielding the first certifying equality saturation engine. We show that our greedy approach in egg quickly generates substantially smaller proofs than the state-of-the-art Z3 SMT solver on a corpus of 3760 benchmarks.Speaker: Gus SmithTitle: Synthesizing FPGA ISA Implementations with LakeroadCompiling hardware designs to FPGAs -- flexible hardware platforms composed of configurable hardware units -- has long been a frustrating task. Legacy compilers developed in the early days of FPGAs have not kept up with the increasing complexity of modern FPGAs. To address this, FPGA compilers are becoming more and more like software compilers, in which a program (i.e. a hardware design) is gradually lowered through increasingly fine-grained levels of abstraction, with the goal of eventually converting the entire program into a list of instructions in a low-level instruction set architecture (ISA). Currently, state-of-the-art FPGA compilers implement their ISAs by hand for each FPGA backend they would like to target. In this work, we present a tool, Lakeroad, which uses program synthesis to implement an ISA given just a high-level description of the FPGA architecture. We demonstrate how Lakeroad is able to implement a rich ISA across a diverse set of FPGAs.Speaker: Sirui LuTitle: Grisette: Symbolic Compilation as a Monadic LibraryThe development of constraint solvers has enabled automated reasoning about programs, shifting the engineering burden to implementing symbolic compilation tools that translate programs into efficient constraints. It is desirable to have reusable symbolic compilation tools to reduce this engineering burden, and such reusable tools should be both efficient and user-friendly. We describe a new formulation of a reusable symbolic compiler architecture, Grisette. Internally, Grisette is based on a novel algorithm that efficiently merges and normalizes program states, improving symbolic compilation speed and reducing constraint size. For the user, it provides a functional, statically-typed, and monadic interface, allowing the user to easily implement high-performance symbolic compilers by configuring our system with domain knowledge.Speaker: Dan CascavalTitle: Referencing CAD ProgramsIn order to write programs that generate geometry and represent spaces of possible Computer-Aided Design (CAD) models, a programmer must refer to geometric elements that might appear or disappear depending on model inputs. We give a domain specific language for defining such CAD references that is safe, unambiguous, and does not duplicate code.
ARDW: An Augmented Reality Workbench for Printed Circuit Board Debugging Paul G. Allen School 2022-11-03 | Debugging printed circuit boards (PCBs) can be a time-consuming process, requiring frequent context switching between PCB design files (schematic and layout) and the physical PCB. To assist electrical engineers in debugging PCBs, we present ARDW, an augmented reality workbench consisting of a monitor interface featuring PCB design files, a projector-augmented workspace for PCBs, tracked test probes for selection and measurement, and a connected test instrument. The system supports common debugging workflows for augmented visualization on the physical PCB as well as augmented interaction with the tracked probes. We quantitatively and qualitatively evaluate the system with ten electrical engineers from industry and academia, finding that ARDW speeds up board navigation and provides engineers with greater confidence in debugging. We discuss practical design considerations and paths for improvement to future systems.
Towards $1 Robots (Ankur Mehta, UCLA) Paul G. Allen School 2022-10-17 | Abstract: Robots are pretty great -- they can make some hard tasks easy, some dangerous tasks safe, or some unthinkable tasks possible. And they're just plain fun to boot. But how many robots have you interacted with recently? And where do you think that puts you compared to the rest of the world's people?In contrast to computation, automating physical interactions continues to be limited in scope and breadth. I'd like to change that. But in particular, I'd like to do so in a way that's accessible to everyone, everywhere. In our lab, we work to lower barriers to robotics design, creation, and operation through material and mechanism design, computational tools, and mathematical analysis.We hope that with our efforts, everyone will be soon able to enjoy the benefits of robotics to work, to learn, and to play.Biography: Prof. Ankur Mehta is an assistant professor of Electrical and Computer Engineering at UCLA, and directs the Laboratory for Embedded Machines and Ubiquitous Robots (LEMUR). Pushing towards his visions of a future filled with robots, his research interests involve printable robotics, rapid design and fabrication, control systems, and multi-agent networks. He has received the NSF CAREER award and a Samueli fellowship, and has received best paper awards in the IEEE Robotics & Automation Magazine and the International Conference on Intelligent Robots and Systems (IROS).
Constraints and Planning for Forceful Robotic Manipulation (Rachel Holladay, MIT) Paul G. Allen School 2022-10-17 | Abstract: Enabling robots to perform multi-stage forceful manipulation tasks, such as twisting a nut on a bolt or pulling a nail with a hammer claw, requires enabling reasoning over interlocking force and motion constraints over discrete and continuous choices. I categorize forceful manipulation as tasks where exerting substantial forces is necessary to complete the task. While all actions with contact involve forces, I focus on tasks where generating and transmitting forces is a limiting factor of the task that must be reasoned over and planned for. I'll first formalize constraints for forceful manipulation tasks where the goal is to exert force, often through a tool, on an object or the environment. These constraints define a task and motion planning problem that we solve to search for both the sequences of discrete actions, or strategy, and the continuous parameters of those actions.Biography: Rachel Holladay is an EECS PhD Student at MIT, where she is a member of the LIS (Learning and Intelligent Systems) Group and the MCube Lab (Manipulation and Mechanisms at MIT). She is interested in developing algorithms for dexterous and composable robotic manipulation and planning. In particular, her doctoral research focuses on enabling robots to complete multi-step manipulation tasks that require reasoning over both force and motion constraints. She received her Bachelor's degree in Computer Science and Robotics from Carnegie Mellon.
Bringing underwater messaging to smartphones Paul G. Allen School 2022-08-29 | Researchers in the University of Washington's Paul G. Allen School of Computer Science & Engineering introduce AquaApp, the first software-only solution for underwater communication and networking on smartphones. AquaApp is an acoustic system that leverages the device's built-in speaker and microphone to send and receive messages in a variety of underwater conditions without any special hardware required. The open-source code is available to download for Android devices at https://underwatermessaging.cs.washington.edu.This video is closed captioned.
IFDS 2022, Jamie Morgenstern (University of Washington) Paul G. Allen School 2022-08-23 | Talk Title: Learning from Multiple Data Sources
IFDS 2022, Ludwig Schmidt (University of Washington) Paul G. Allen School 2022-08-23 | Talk Title: A Data-Centric View on Robustness
IFDS 2022, Samory Kpotufe (Columbia University) Paul G. Allen School 2022-08-23 | Talk Title: Tracking Significant Changes in Bandit
IFDS 2022, Yao Xie (Georgia Tech) Paul G. Allen School 2022-08-23 | Talk Title: Distributionally Robust Hypothesis Tests
Adaptive Attention: Bringing Active Vision into the Camera (Sanjeev Koppal, University of Florida) Paul G. Allen School 2022-07-13 | Abstract: Most cameras today capture images without considering scene content. In contrast, animal eyes have fast mechanical movements that control how the scene is imaged in detail by the fovea, where visual acuity is highest. The prevalence of active vision during biological imaging, and the wide variety of it, makes it very clear that this is an effective visual design strategy. In this talk, I cover our recent work on creating *both* new camera designs and novel vision algorithms to enable adaptive and selective active vision and imaging inside cameras and sensors.Biography: Sanjeev Koppal is an Associate Professor at the University of Florida’s Electrical and Computer Engineering Department. He also holds a UF Term Professor Award for 2021-24. Sanjeev is the Director of the FOCUS Lab at UF. Prior to joining UF, he was a researcher at the Texas Instruments Imaging R&D lab. Sanjeev obtained his Masters and Ph.D. degrees from the Robotics Institute at Carnegie Mellon University. After CMU, he was a postdoctoral research associate in the School of Engineering and Applied Sciences at Harvard University. He received his B.S. degree from the University of Southern California in 2003 as a Trustee Scholar. He is a co-author on best student paper awards for ECCV 2016 and NEMS 2018, and work from his FOCUS lab was a CVPR 2019 best-paper finalist. Sanjeev won an NSF CAREER award in 2020 and is an IEEE Senior Member. His interests span computer vision, computational photography and optics, novel cameras and sensors, 3D reconstruction, physics-based vision, and active illumination.
Northwest Robotics Symposium Session 3 Paul G. Allen School 2022-07-13 | 0:00 Intro0:14 "Blind bipedal stair traversal via sim-to-real reinforcement learning"Jonah Siekmann, Kevin Green, John Warila, Alan Fern, Jonathan Hurst10:55 "Sim-to-Real Learning of Footstep-Constrained Bipedal Dynamic Walking"Helei Duan, Ashish Malik, Jeremy Dao, Aseem Saxena, Kevin Green, Jonah Siekmann, Alan Fern, Jonathan Hurst21:03 "Single-stage Keypoint-based Category-level Object Pose Estimation from an RGB Image"Yunzhi Lin, Jonathan Tremblay, Stephen Tyree, Patricio A. Vela, Stan Birchfield31:56 "Visual Object Recognition in Indoor Environments Using Topologically Persistent Features"Ekta Samani, Xingjian Yang, Ashis Banerjee42:11 "Benchmarking a Robot Hand's Ability to Translate Objects Using Two Fingers"John Morrow, Joshua Campbell, Ravi Balasubramanian, Cindy Grimm53:05 "A Device for Rapid, Automated Trimming of Insect-Sized Flying Robots"Daksh Dhingra, Yogesh Chukewad, Sawyer B. Fuller1:04:38 "Comparing the Perception of Vibrotactile Feedback across Frequency and Body Location"Ryan Quick, Anisha Bontula, Naomi T. Fitter
Northwest Robotics Symposium Session 2 Paul G. Allen School 2022-07-13 | 0:00 Intro0:15 "A Multimodal and Hybrid Framework for Human Navigational Intent Inference"Zhitian Zhang, Jimin Rhim, Angelica Lim, Mo Chen8:57 "Robotic Information Gathering using Semantic Language Instructions"Ian C. Rankin, Seth McCammon, Geoffrey A. Hollinger19:18 "Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers"Arthur Bucker, Luis Figueredo, Sami Haddadin, Ashish Kapoor, Shuang Ma, Rogerio Bonatti29:54 "SORNet: Spatial Object-Centric Representations for Sequential Manipulation"Wentao Yuan, Chris Paxton, Karthik Desingh, Dieter Fox42:05 "Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition"Jonah Siekmann, Yesh Godse, Alan Fern, Jonathan Hurst53:09 "Sim-to-Real Learning for Bipedal Locomotion Under Unsensed Dynamic Loads"Jeremy Dao, Kevin Green, Helei Duan, Alan Fern, Jonathan Hurst1:03:57 "Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions"Wenhao Luo, Wen Sun, Ashish Kapoor1:15:05 "Representation Learning for Event-based Visuomotor Policies"Sai Vemprala, Sami Mian, Ashish Kapoor1:26:03 "LBGP: Learning Based Goal Planning for Autonomous Following in Front"Payam Nikdel, Richard Vaughan, Mo Chen
Northwest Robotics Symposium Session 1 Paul G. Allen School 2022-07-13 | 0:00 Intro7:16 "Comedians in cafes getting data: Comparing performances of a robotic stand-up comedian in the wild" John Vilk, Naomi T. Fitter17:42 "Design of an assistive robot for infant mobility interventions"Ashwin Vinoo, Layne Case, Gabriela R. Zott, Joseline Raja Vora, Ameer Helmi, Samuel W. Logan, Naomi T. Fitter27:50 "Leveraging Post Hoc Context for Faster Learning in Bandit Settings with Applications in Robot-Assisted Feeding" Ethan K. Gordon, Sumegh Roychowdhury, Tapomayukh Bhattacharjee, Kevin Jamieson, Siddhartha S. Srinivasa37:37 "Human Perceptions of a Curious Robot that Performs Off-Task Actions"Nick Walker, Kevin Weatherwax, Julian Allchin, Leila Takayama, Maya Cakmak47:55 "Modeling Human Helpfulness with Individual and Contextual Factors for Robot Planning"Amal Nanavati, Christoforos Mavrogiannis, Kevin Weatherwax, Leila Takayama, Maya Cakmak, Siddhartha S. Srinivasa57:06 "Iterative Repair of Social Robot Programs from Implicit User Feedback via Bayesian Inference" Michael Jae-Yoon Chung, Maya Cakmak1:07:29 "Bringing WALL-E out of the Silver Screen: Understanding How Transformative Robot Sound Affects Human Perception" Brian J. Zhang, Nick Stargu, Samuel Brimhall, Lilian Chan, Jason Fick, Naomi T. Fitter1:18:20 "Motivating Physical Activity via Competitive Human-Robot Interaction"Boling Yang, Golnaz Habibi, Patrick Lancaster, Byron Boots, Joshua Smith1:28:37 "A Shared Autonomy Surface Disinfection System Using a Mobile Manipulator Robot"Alan G. Sanchez, William B. Smart
Interactive Imitation Learning: Planning Alongside Humans (Sanjiban Choudhury, Cornell) Paul G. Allen School 2022-07-11 | Abstract: Advances in machine learning have fueled progress towards deploying real-world robots from assembly lines to self-driving. However, if robots are to truly work alongside humans in the wild, they need to solve fundamental challenges that go beyond collecting large-scale datasets. Robots must continually improve and learn online to adapt to individual human preferences. How do we design robots that both understand and learn from natural human interactions?Biography: Sanjiban Choudhury is a Research Scientist at Aurora Innovation and soon-to-be Assistant Professor at Cornell University. His research goal is to enable robots to work seamlessly alongside human partners in the wild. To this end, his work focuses on imitation learning, decision making and human-robot interaction. He obtained his Ph.D. in Robotics from Carnegie Mellon University and was a Postdoctoral fellow at the University of Washington. His research has received best paper awards at ICAPS 2019, finalist for IJRR 2018, and AHS 2014, and winner of the 2018 Howard Hughes award. He is a Siebel Scholar, class of 2013.
ClearBuds: Wireless Binaural Earbuds for Learning-based Speech Enhancement Paul G. Allen School 2022-07-01 | We present ClearBuds, the first end-to-end hardware and software system that utilizes a neural network to enhance speech streamed from two wireless earbuds. Real-time speech enhancement for wireless earbuds requires high-quality sound separation and background cancellation, operating in real-time and on a mobile phone. ClearBuds bridges state-of-the-art deep learning for blind audio source separation and in-ear mobile systems by making two key technical contributions: 1) a new wireless earbud design capable of operating as a synchronized, binaural microphone array, and 2) a lightweight dual-channel speech enhancement neural network that runs on a mobile device. Our neural network has a novel cascaded architecture that combines a time-domain conventional neural network with a spectrogram-based frequency masking neural network to reduce the artifacts in the audio output. Results show that our wireless earbuds achieve a synchronization error less than 64 𝜇s and our network has a runtime of 21.4 ms on an accompanying mobile phone.
ClearBuds: Wireless Binaural Earbuds for Learning-based Speech Enhancement (Short) Paul G. Allen School 2022-07-01 | We present ClearBuds, the first end-to-end hardware and software system that utilizes a neural network to enhance speech streamed from two wireless earbuds. Real-time speech enhancement for wireless earbuds requires high-quality sound separation and background cancellation, operating in real-time and on a mobile phone. ClearBuds bridges state-of-the-art deep learning for blind audio source separation and in-ear mobile systems by making two key technical contributions: 1) a new wireless earbud design capable of operating as a synchronized, binaural microphone array, and 2) a lightweight dual-channel speech enhancement neural network that runs on a mobile device. Our neural network has a novel cascaded architecture that combines a time-domain conventional neural network with a spectrogram-based frequency masking neural network to reduce the artifacts in the audio output. Results show that our wireless earbuds achieve a synchronization error less than 64 𝜇s and our network has a runtime of 21.4 ms on an accompanying mobile phone.Contact: clearbuds@cs.washington.eduWebsite and paper: clearbuds.cs.washington.edu
Dynamic Robot Manipulation: Learned Optimization, Deformable Materials, and the Cloud Paul G. Allen School 2022-06-17 | Speaker: Jeffrey Ichnowski, UC BerkeleyAbstract: Robots in unstructured environments manipulate objects slowly and intermittently, relying on bursts of computation for planning. This is in stark contrast to humans who routinely use fast dynamic motions to manipulate and move objects or vault power cords over chairs when vacuuming. Dynamic motions can speed task completion, manipulate objects out of reach, and increase reliability, but they require: (1) integrating grasp planning, motion planning, and time-parameterization, (2) lifting quasi-static assumptions, and (3) intermittent access to powerful computing. I will describe how integrating grasp analysis into motion planning can speed up motions, how integrating deep-learning can speed up computation, and how integrating inertial and learned constraints can lift quasi-static assumptions to allow high-speed manipulation. I will also describe how cloud computing can provide on-demand access to immense computing to speed up motion planning and a new cloud-robotics framework that makes it easy.Speaker bio: Jeffrey Ichnowski is a post-doctoral researcher in the RISE lab and AUTOLAB at the University of California at Berkeley. He researches algorithms and systems for high-speed motion, task, and grasp planning for robots, using cloud-based high-performance computing, optimization, and deep learning. Jeff has a Ph.D. in computational robotics from the University of North Carolina at Chapel Hill. Before returning to academia, he founded startups and was an engineering director and the principal architect at SuccessFactors, one of the world’s largest cloud-based software-as-a-service companies
Performing tympanometry using smartphones Paul G. Allen School 2022-06-16 | Middle ear disorders are one of the most common causes of preventable hearing loss. While the developing world bears a disproportionate burden of these disorders, it often lacks access to diagnostic tools to support early detection and treatment. Tympanometry is a key test to measure middle ear function, but remains available only on expensive test equipment that is bulky and inconvenient to transport and requires a wall outlet for power. We present a low-cost smartphone-based tympanometry system that works by changing the air pressure of the ear canal and measuring eardrum mobility. Our system, for which the software and hardware are openly available, has the potential to make screening of middle ear disorders more accessible in low-resource settings.
Allen School Distinguished Lecture: David Patterson (UC Berkeley + Google) Paul G. Allen School 2022-05-09 | Title: A Decade of Machine Learning Accelerators: Lessons Learned and Carbon FootprintDate: April 26, 2022Abstract: The Success of dep neural networks (DNNs) from Machine Learning (ML) has inspired domain specific architectures (DSAs) for them. ML has two phases: training, which constructs accurate models, and interference, which serves those models. Google's first generation DSA offered 50x improvement over conventional architectures for inference in 2015. Google next built the first production DSA supercomputer for the much harder problem of training. Subsequent generations greatly improved performance of both phases. We start with ten lessons learned, such as DNNs grow rapidly; workloads quickly evolve with DNN advances; the bottleneck is memory, not floating-point units; and semiconductor technology advances unequally. The rapid growth of DNNs rightfully raised concerns about their carbon footprint. The second part of the talk identifies the 4Ms (Model, Machine, Mechanism, Map) that, if optimized, can reduce ML training energy by up to 100x and carbon emissions up to 1000x. By improving the 4Ms, ML held steady at less than 15% of Google's total energy use despite it consuming approximately 75% of its floating point operations. Given the importance of climate change, ML papers should include emissions explicitly to foster competition on more than just model quality. External estimates have been off 100x-100,000x, so publishing emissions also ensures accurate accounting, which helps pinpoint the biggest challenges for climate change. With continuing focus on the 4Ms, we can realize the amazing potential of ML to positively impact many fields in a sustainable way.Bio: David Patterson is a UC Berkeley professor emeritus, a Google distinguished engineer, RIOS Laboratory Director, and the RISC-V International Vice-Chair. He received BA, MS, and PhD degrees from UCLA. His Berkeley projects on Reduced Instruction Set Computers (RISC), Redundant Array of Inexpensive Disks (RAID), and Network of Workstation (NOW) helped lead to multi-billion-dollar industries. The best known of his seven books is Computer Architecture: A Quantitative Approach. He and his co-author John Hennessy shared the 2017 ACM A.M Turing Award, the 2021 BBVA Foundation Frontiers of Knowledge Award, and the 2022 NAE Charles Stark Draper Prize for Engineering.This video is closed captioned.
Allen School Distinguished Lecture: Christos Papadimitriou (Columbia) Paul G. Allen School 2022-05-09 | Title: How does the brain beget the mind?Date: Thursday, April 21, 2022Abstract:There is no doubt that cognition and intelligence are the results of neural activity -- but how, exactly? How do molecules, neurons, and synapses give rise to reasoning, language, plans, stories, art, math? Despite dazzling progress in experimental neuroscience, as well as in cognitive science, we do not seem to be making progress on the overarching question. As Richard Axel recently put it in an interview: "We don't have a logic for the transformation of neuronal activity to thought and action. I view discerning [this] logic as the most important future direction of neuroscience." What kind of formal system would qualify as this "logic"?I will introduce a computational system whose basic data structure is the assembly of neurons -- assemblies are large populations of neurons known to represent concepts, words, ideas, episodes, etc. The Assembly Calculus is biologically plausible in the sense that Its primitives are behaviors of assemblies observed in, or suggested by, experiments, and can be provably (through both mathematical proof and simulations in biologically realistic platforms) "compiled down" to the activity of neurons and synapses. Experiments show that this programming framework can simulate -- exclusively through the spiking of neurons -- high-level cognitive functions, such as parsing natural language and planning in the blocks world. I believe that this formalism is well-positioned to help in bridging the gap between the brain and the mind.Bio:Christos Papadimitriou is the Donovan Family Professor of Computer Science at Columbia University. Before joining Columbia in 2017, he was a professor at UC Berkeley for the previous 22 years, and before that he taught at Harvard, MIT, NTU Athens, Stanford, and UCSD. He has written five textbooks and many articles on algorithms and complexity, and their applications to optimization, databases, control, AI, robotics, economics and game theory, the Internet, evolution, and the brain. He holds a PhD from Princeton (1976), and eight honorary doctorates, including from ETH, University of Athens, EPFL, and Univ. de Paris Dauphine. He is a member of the National Academy of Sciences of the US, the American Academy of Arts and Sciences, and the National Academy of Engineering, and he has received the Knuth prize, the Goedel prize, the Babbage award, the von Neumann medal, as well as the 2018 Harvey Prize by Technion. He is the author of three novels: Turing, Logicomix and his latest Independence. This video is closed captioned.
![[ASL] Toward Total Scene Understanding for Autonomous Driving—Drago Anguelov (Waymo)](https://i.ytimg.com/vi/pK5ChzMsfE0/hqdefault.jpg)