Amii
Keen Technologies Research Directions: John Carmack, Upper Bound 2025
updated
Abstract:
There have been many methods proposed to generate reading comprehension questions. However, few allow specific skills to be targeted and there is still a gap in quality between automatically generated and manually created questions. To narrow this gap, we present SoftSkillQG, a new soft-prompt based language model for generating skill targeted reading comprehension questions that does not require manual effort to target new skills. In this presentation, we explore the strengths of SoftSkillQG and how its weaknesses can be addressed.
Presenter Bio:
Spencer von der Ohe is a Computing Science master’s student at the University of Alberta. He is interested in natural language processing and how it can be applied to real world problems. Spencer is supervised by Dr. Alona Fyshe.
Abstract:
In the context of video games, Downloadable Content (DLC) refers to additional content or features that can be incorporated into a game after its initial release. In the fields of Automated Game Design (AGD) and Procedural Content Generation (PCG), various approaches have been explored to augment game content. However, no work has attempted to automatically generate DLCs. This talk will cover our concept of how AGD and PCG can be employed to develop compelling DLCs. We’ll specifically focus on extending the existing level structure through level inpainting and generating additional mechanics through a reinforcement learning-based approach.
Presenter Bio:
Johor Jara Gonzalez is a PhD student at the University of Alberta and Alberta Machine Intelligence Institute (Amii). His research interests are focused on computational creativity, and more precisely DLC Generation and Mechanics in Video Games. He has presented at AAAI’s Artificial Intelligence and Interactive Digital Entertainment conference and reviewed for IEEE's Transactions on Games.
This is an excerpt from Bowling’s full interview from the Approximately Correct Podcast
See the full interview here: youtu.be/V-R2B0yElzA
✨Listen to Approximately Correct✨
Spotify: open.spotify.com/show/3GwbDft0b4xfatxmW8kW5c
Apple Podcasts: podcasts.apple.com/us/podcast/teaser-introducing-approximately-correct/id1725553442?i=1000641479209
✨Social Media✨
Instagram: instagram.com/amiithinks
Tiktok: tiktok.com/@amiithinks
Twitter: twitter.com/AmiiThinks
LinkedIn: linkedin.com/company/amii
✨Listen to Approximately Correct✨
Spotify: open.spotify.com/show/3GwbDft0b4xfatxmW8kW5c
Apple Podcasts: podcasts.apple.com/us/podcast/teaser-introducing-approximately-correct/id1725553442?i=1000641479209
✨Social Media✨
Instagram: instagram.com/amiithinks
Tiktok: tiktok.com/@amiithinks
Twitter: twitter.com/AmiiThinks
LinkedIn: linkedin.com/company/amii
This is an excerpt from Bowling’s full interview from the Approximately Correct Podcast
See the full interview here: youtu.be/V-R2B0yElzA
✨Listen to Approximately Correct✨
Spotify: open.spotify.com/show/3GwbDft0b4xfatxmW8kW5c
Apple Podcasts: podcasts.apple.com/us/podcast/teaser-introducing-approximately-correct/id1725553442?i=1000641479209
✨Social Media✨
Instagram: instagram.com/amiithinks
Tiktok: tiktok.com/@amiithinks
Twitter: twitter.com/AmiiThinks
LinkedIn: linkedin.com/company/amii
✨Listen to Approximately Correct✨
Spotify: open.spotify.com/show/3GwbDft0b4xfatxmW8kW5c
Apple Podcasts: podcasts.apple.com/us/podcast/teaser-introducing-approximately-correct/id1725553442?i=1000641479209
✨Social Media✨
Instagram: instagram.com/amiithinks
Tiktok: tiktok.com/@amiithinks
Twitter: twitter.com/AmiiThinks
LinkedIn: linkedin.com/company/amii
Abstract:
Data-driven teleoperation research investigates the application of machine learning models for mapping interactions between control interfaces (e.g., a joystick) and robotic arms to simplify human-robotic interactions. There is a real-world utility for such AI systems in assistive robotics, where people living with a disability rely on robots to complete daily tasks (e.g., opening doors, brushing teeth, etc.). Existing mappings involve cycling between controlling subsets of a robot’s pose. These can be tedious because they can require switching the control mode anywhere from 30 to 60 times to accomplish daily tasks. Data-driven mappings simplify teleoperation by mapping a user's inputs directly to desired robot motions, eliminating the need to switch the control mode. This presentation covers previous works investigating data aspects of such systems, including data collection procedures (supervised vs unsupervised) and choice of data modalities (robotic proprioceptive information, images, text, and belief). The latter half of the presentation discusses our own work explicitly enforcing teleoperation properties and briefly includes incorporating such models with shared autonomy systems. Overall, attendees can expect to learn how to build data-driven teleoperation systems and an introduction to existing research in this application of machine learning.
Presenter Bio:
Michael Przystupa is a 5th year PhD student at the University of Alberta working at the intersection of machine learning and robotics. His research has spanned the applications of deep learning in robotics topics including visual servoing, motor primitives and teleoperation. He is co-supervised by Dr. Martin Jagersand and Dr. Matthew Taylor.
Abstract:
In this seminar presented by the Alberta Machine Intelligence Institute (Amii), Subhojeet Pramanik, University of Alberta, explains that Recurrent Linear Transformer addresses limitations in the transformer architecture by proposing a recurrent alternative to the self-attention mechanism. Overcoming issues of context-dependent inference and high computational costs, the approach demonstrates longer context and reduced computational complexity over vanilla self-attention mechanism. Evaluating performance in pixel-based reinforcement learning environments, the approach outperforms the state-of-the-art GTrXL by at least 40% faster FPS and more than 50% lesser memory usage. Notably, our approach achieves over 37% improvement in performance on harder tasks.
Presenter Bio:
Subhojeet Pramanik works in deep learning and reinforcement learning. His expertise lies in building scalable algorithms, specifically in designing efficient transformer architectures. He completed his MSc at the University of Alberta under the supervision of Adam White and Marlos C. Machado. During his masters, he worked in representation learning in the partially observable reinforcement learning setting. He was also an intern at the Huawei Research Lab in Edmonton working on applying reinforcement learning techniques to deep learning operator fusion. Previously, Subhojeet worked at IBM Cloud for two years as an ML Engineer in topics such as deep learning and NLP.
Abstract:
This talk develops the idea that reinforcement learning can solve the problems of sports analytics. I focus on ranking player performance. We measure the value of a player’s actions by how much they increase their team’s chance of success (i.e., scoring the next goal). I describe deep reinforcement learning techniques for building success probability models from sports data. The resulting action values and player rankings are illustrated with data from the National Hockey League comprising over 3M events.
Presenter Bio:
Dr. Oliver Schulte is a Professor in the School of Computing Science at Simon Fraser University, Vancouver, Canada. He received his Ph.D. from Carnegie Mellon University in 1997. His current research focuses on machine learning for structured data, such as sports events, networks, and relational databases. He has given several invited talks on sports analytics and spent two years at Sportlogiq, a sports analytics company. He has published papers in leading AI and machine learning venues on a variety of topics, including sports analytics, learning Bayesian networks, game theory, and scientific discovery. While he has won some nice awards, his biggest claim to fame may be a draw against chess world champion Gary Kasparov.
Abstract:
It is widely believed that model-based reinforcement learning can improve sample efficiency by synthesizing imagined experience which generalizes beyond the data. However, learned value functions also generalize. Why should we expect model generalization to be inherently better? In this talk, I address this question with a simple theoretical result. I also present extensive empirical results which demonstrate that the intuition behind the theory leads to significant practical benefits in environments with underlying structure that allows learned models to generalize.
Presenter Bio:
Kenny Young is a PhD student at the University of Alberta, supervised by Richard Sutton. He has worked on a variety of topics in and around reinforcement learning, including developing the popular MinAtar testbed. Most recently he is interested in model-based reinforcement learning as a means to build agents that are capable of learning, planning and thinking about their world.
Abstract:
In recent years, language models (LMs) have emerged as transformative tools in artificial intelligence. But they are not good independent solvers for complex reasoning tasks. This talk will introduce a new approach, which incorporates LMs within a larger framework for complex reasoning. Here, LMs propose solutions or logical pathways, which are then analyzed and utilized by the framework. This talk will show two challenging reasoning problems effectively addressed using this paradigm.
Presenter Bio:
Dr. Hongyuan Mei is currently a Research Assistant Professor at Toyota Technological Institute at Chicago (TTIC). He obtained his PhD from the Department of Computer Science at Johns Hopkins University (JHU), where he was advised by Jason Eisner. Hongyuan's research spans machine learning and natural language processing. Currently, he is most interested in harnessing and improving the reasoning capabilities of large language models to solve challenging problems such as event prediction. His research has been supported by a Bloomberg Data Science PhD Fellowship, the 2020 JHU Jelinek Memorial Award, and research gifts from Adobe and Ant Group. His technical innovations have been integrated into real-world products such as Alipay, the world’s largest mobile digital payment platform, which serves more than one billion users. His research has been covered by Fortune Magazine and Tech At Bloomberg.
Abstract:
This talk will present several challenges and opportunities within the area of team sports analytics and key research areas within multiagent systems (MAS). We specifically consider invasion games, where players invade the opposing team’s territory and can interact anywhere on a playing surface (ice hockey or soccer). We discuss how MAS is well-equipped to study invasion games and will benefit both MAS and sports analytics fields. We highlight topics along two axes: short-term strategy (coaching) and long-term team planning (management).
Presenter Bio:
Dr. David Radke is a Senior Research Scientist with the Chicago Blackhawks of the National Hockey League (NHL). He holds a PhD from the University of Waterloo where he was advised by Dr. Kate Larson and Dr. Tim Brecht. His research areas include artificial intelligence (AI) and ice hockey analytics, specifically focusing on the areas of multiagent systems and reinforcement learning. He has published several papers at several top AI conferences, including IJCAI and AAMAS. He has also published papers at LINHAC, a hockey analytics conference, where he received a best paper award in 2022 for his work with NHL tracking data. Dr. Radke's graduate work was supported by the Vector Institute computing resources and NSERC, Ontario Graduate Scholarship, and Type 1 Cheriton awards.
Abstract:
Our research focuses on leveraging expressive deep learning tools to model and comprehend human actions, involving diverse areas such as motion synthesis from action categories (action2motion) or textual descriptions (text2motion), image-based character animation (motion2video), generative motion stylization, and reciprocal generation of motions and texts (motion2text-2-motion). To approach those goals, we introduce two extensive multimodal human action datasets: HumanAct12, with 1,191 motion clips and 90,099 frames annotated for 12 coarse-grained and 34 fine-grained action classes, and HumanML3D, comprising 44,970 textual descriptions and 14,616 motions. Technically, our investigation incorporates advanced deep generative models, such as variational autoencoders, GPT, and masked generative models, resulting in state-of-the-art performance in motion analysis.
Presenter Bio:
Chuan Guo is a final-year Ph.D. student in the ECE department at the University of Alberta. His research interests mainly revolve around 3D human motion synthesis, stylization, understanding, and character animation. He has a series of works published on top-tier computer vision and machine learning venues, including CVPR, ECCV, IJCV, ICCV, ICLR. Webpage: ericguo5513.github.io
Her research focuses on development and evaluation of technical privacy solutions; including the use of privacy mechanisms for data analysis. She is currently focused on human-centred privacy-preserving machine learning.
Check out her conversation with Adam White, Amii’s Director of Scientific Operations, where she talks about the importance of protecting privacy in machine learning, the shifting legal landscape of privacy, and the problem of the "privacy paradox."
This time, Revan MacQueen rose to the challenge - here, he gives a breakdown for a general audience of "Guarantees for Self-Play in Multiplayer Games via Polymatrix Decomposability," being presented at this year's Conference on Neural Information Processing Systems.
See the other plain language explanations of NeurIPS papers: youtube.com/playlist?list=PLKlhhkvvU8-aPSAsjvTKWFX4hNgaP8fGU&si=jmlRmxQWQhiYzYPi
See the technical versions of these papers here: youtube.com/playlist?list=PLKlhhkvvU8-a5FWzAtMDk4FH1hofg1bzW&si=vniZf5mG5tUNDykb
This time, Chris Solinas rose to the challenge - here, he gives a breakdown for technical audiences of "History Filtering in Imperfect Information Games: Algorithms and Complexity," being presented at this year's Conference on Neural Information Processing Systems.
See the other technical explanations of NeurIPS papers: youtube.com/playlist?list=PLKlhhkvvU8-aPSAsjvTKWFX4hNgaP8fGU&si=jmlRmxQWQhiYzYPi
See the plain language versions of these papers here: youtube.com/playlist?list=PLKlhhkvvU8-a5FWzAtMDk4FH1hofg1bzW&si=vniZf5mG5tUNDykb
This time, Zichen Zhang rose to the challenge - here, he gives a breakdown for technical audiences of "Managing Temporal Resolution in Continuous Value Estimation:A Fundamental Trade-off," being presented at this year's Conference on Neural Information Processing Systems.
See the other technical explanations of NeurIPS papers: youtube.com/playlist?list=PLKlhhkvvU8-aPSAsjvTKWFX4hNgaP8fGU&si=jmlRmxQWQhiYzYPi
See the plain language versions of these papers here: youtube.com/playlist?list=PLKlhhkvvU8-a5FWzAtMDk4FH1hofg1bzW&si=vniZf5mG5tUNDykb
This time, Zichen Zhang rose to the challenge - here, he gives a breakdown for general audiences of "Managing Temporal Resolution in Continuous Value Estimation:A Fundamental Trade-off," being presented at this year's Conference on Neural Information Processing Systems.
See the other plain language explanations of NeurIPS papers: youtube.com/playlist?list=PLKlhhkvvU8-aPSAsjvTKWFX4hNgaP8fGU&si=jmlRmxQWQhiYzYPi
See the technical versions of these papers here: youtube.com/playlist?list=PLKlhhkvvU8-a5FWzAtMDk4FH1hofg1bzW&si=vniZf5mG5tUNDykb
This time, Chris Solinas rose to the challenge - here, he gives a breakdown for general audiences of "History Filtering in Imperfect Information Games: Algorithms and Complexity," being presented at this year's Conference on Neural Information Processing Systems.
See the other plain language explanations of NeurIPS papers: youtube.com/playlist?list=PLKlhhkvvU8-aPSAsjvTKWFX4hNgaP8fGU&si=jmlRmxQWQhiYzYPi
See the technical versions of these papers here: youtube.com/playlist?list=PLKlhhkvvU8-a5FWzAtMDk4FH1hofg1bzW&si=vniZf5mG5tUNDykb
This time, Revan MacQueen rose to the challenge - here, he gives a breakdown for a technical audience of "Guarantees for Self-Play in Multiplayer Games via Polymatrix Decomposability," being presented at this year's Conference on Neural Information Processing Systems.
See the other technical explanations of NeurIPS papers: youtube.com/playlist?list=PLKlhhkvvU8-aPSAsjvTKWFX4hNgaP8fGU&si=jmlRmxQWQhiYzYPi
See the plain language versions of these papers here: youtube.com/playlist?list=PLKlhhkvvU8-a5FWzAtMDk4FH1hofg1bzW&si=vniZf5mG5tUNDykb
Rockwell's research includes AI ethics, textual visualization and analysis and computing in the humanities
Check out his conversation with Alona Fyshe, Amii Fellow and Canada CIFAR Chair, where they discuss Rockwell's thoughts on dialogues with AI, computer ethics and the need for more AI literacy when it comes to large language models.
Abstract:
Recent advancement of imaging sensors and deep learning techniques has opened door to many interesting applications for visual analysis of human motions. In this talk, I will discuss our research efforts toward addressing the related tasks of 3-D human motion syntheses, pose and shape estimation from images and videos, visual action quality assessment. Looking forward, our results could be applied to everyday life scenarios such as natural user interface, AR/VR, robotics, and gaming, among others.
Presenter Bio:
Dr. Li Cheng is a professor at the Department of Electrical and Computer Engineering, University of Alberta. He is associate editor of IEEE Trans. Multimedia. Prior to joining University of Alberta, He worked at A*STAR, Singapore, TTI-Chicago, USA, and NICTA, Australia. His current research interests are mainly on human motion analysis, mobile and robot vision, and multimedia data analytics. More details can be found at http://www.ece.ualberta.ca/~lcheng5/.
Abstract:
Emphatic algorithms are an under-explored approach to off-policy reinforcement learning that emphasize or de-emphasize learning updates in a way that preserves many of the benefits of on-policy algorithms. However, existing methods for estimating how much to emphasize each update suffer from either extreme variance or extreme bias that can both lead to failure of the learning process in different situations. In this talk I discuss how these disparate algorithms can be combined to balance their strengths and weaknesses and successfully learn in situations where one or both of the constituent algorithms fails.
Presenter Bio:
Eric Graves is a PhD candidate at the University of Alberta, advised by Martha White and Richard Sutton. His research has focused on off-policy learning and policy gradient methods with the goal of improving the reliability, practicality, and range of applications of reinforcement learning algorithms. Prior to graduate school, Eric earned a BSc in Computing Science from the University of Alberta and worked in industry for several years as a software and video game developer.
Abstract:
Reward specification is a notoriously difficult problem in reinforcement learning, requiring extensive expert supervision to design robust reward functions. Imitation learning (IL) methods attempt to circumvent these problems by utilizing expert demonstrations instead of using an extrinsic reward function but typically require a large number of in-domain expert demonstrations. Inspired by advances in the field of Video-and-Language Models (VLMs), we present RoboCLIP, an online imitation learning method that uses a single demonstration (overcoming the large data requirement) in the form of a video demonstration or a textual description of the task to generate rewards without manual reward function design. Additionally, RoboCLIP can also utilize out-of-domain demonstrations, like videos of humans solving the task for reward generation, circumventing the need to have the same demonstration and deployment domains. RoboCLIP utilizes pretrained VLMs without any finetuning for reward generation. Reinforcement learning agents trained with RoboCLIP rewards demonstrate 2-3 times higher zero-shot performance than competing imitation learning methods on downstream robot manipulation tasks, doing so using only one video/text demonstration.
Presenter Bio:
Sumedh A Sontakke is a PhD student at the University of Southern California (working with Prof Laurent Itti and Prof Erdem Biyik) and a research intern at Microsoft Research working on robotics from human feedback. He works on building RL agents that can utilize human demonstrations for efficient policy learning. He also works on formulations of curiosity-based reward functions to enable the self-supervised discovery of complex behaviors. In the past, he has spent time at Google Deepmind, Bell Labs, Adobe Research and the Max Planck Institute for Intelligent Systems.
Abstract:
The input data in continual learning (CL) is non-iid, making it fundamentally different from traditional supervised learning. New optimizers, loss functions, and training strategies are developed for CL. Yet the interactions between CL and neural network architectures remain to be under-explored. We investigate the impact of activation functions in CL, revealing that both sparse gradients and representations are pivotal. Then we introduce elephant activation functions which produce both sparse values and gradients, significantly reducing catastrophic forgetting of neural networks.
Presenter Bio:
Qingfeng Lan is a PhD student at the University of Alberta, supervised by Rupam Mahmood. He is interested in designing simple and efficient algorithms, supported by sound theories and verified by rigorous experiments. In particular, his research focuses on continual reinforcement learning.
Abstract:
The development of tools for cancer imaging is dependent on the availability of public imaging datasets, many of which do not include annotations or derived features. We have generated AI-derived annotations for lung cancer imaging collections within the National Cancer Institute Imaging Data Commons (IDC), where the largest collection contains over 26,000 patients. Our work demonstrates how one can harmonize their data and make it publicly available along with code, to promote reproducibility and transparency of AI workflows.
Presenter Bio:
Dr. Deepa Krishnaswamy is a postdoctoral research fellow at Brigham and Women’s Hospital, working with Dr. Andrey Fedorov in the Surgical Planning Laboratory. She completed her PhD at the University of Alberta under the supervision of Dr. Kumaradevan Punithakumar and Dr. Michelle Noga, in the field of registration and segmentation of the left ventricle in ultrasound volumes. Over the past two years of her postdoc, she has focused on developing publicly available, and reproducible methods, for enhancing lung cancer imaging collections in NCI’s Imaging Data Commons. Additionally, she has developed and contributed to a 3DSlicer plugin for multi-parametric MRI annotation of the prostate. Her other interests include prostate data curation, deep learning approaches for MRI segmentation, and open-source tools for visualization.
Abstract:
In this seminar presented by the Alberta Machine Intelligence Institute (Amii) and Technology Alberta, Ali Salma, CMO of Lawtiq.com, shows that in the traditional investment landscape, investors face challenges in finding the right opportunities because of the role of intermediaries and the limitations of human judgment. This talk details the AI Advantage at Lawtiq.com, including an introduction to Lawtiq.com and its mission, how AI is integrated into Lawtiq.com's platform, and the benefits of AI-driven investment matching. These benefits include efficiency, accuracy, and personalization.
Presenter Bio:
Ali Salman is a problem solver, growth strategist, and fractional CMO who has worked with Remax, WFG, Subway, Manpower, YMCA, The Brick, Coca-Cola, Columbia University, Maxwell, Mucho Burrito, KIA, Honda, Samsung and other Fortune 500 and INC 5000 companies. Currently he heads Lawtiq.com
Abstract:
Learning from experience and continual adaptation to changing environments is crucial for intelligent robots to solve an open-ended sequence of increasingly complex tasks. In this talk, I’ll outline some oft-ignored, practical challenges of continual learning on real-world robots and address two such issues: (i) How to specify reinforcement learning tasks?, and (ii) How to set up a real-time learning agent? Our findings helped us produce the first demonstration of pixel-based control by real-time model-free learning on four different kinds of real robots from scratch in just a few hours.
Presenter Bio:
Gautham Vasan is a PhD student at the University of Alberta, advised by Dr. Rupam Mahmood. He is interested in building machines with human-like intelligence. His research focuses on policy gradient methods, real-time learning architectures and temporal abstraction in reinforcement learning. At Kindred Systems Inc, he worked on developing deep reinforcement learning techniques for an automated put wall robot, known as SORT, which efficiently identifies apparel items, picks them, places them, and sorts them into complete end-customer orders. His website: gauthamvasan.github.io
Li's research focuses on computer vision, data analytics and health informatics. Her research outcomes have contributed to projects involving computational medical imaging, cranial implant design, genomes for COVID-19, and security in deep learning.
Check out her conversation with Adam White, Amii’s Director of Scientific Operations, where she talks about her groundbreaking work in visual anomaly detection, how to use AI for healthcare while maintaining patient privacy, and her thoughts on the future of AI models in medicine.
Abstract:
Many medical researchers want a tool that “does what a top medical clinician does, but does it better”. This presentation explores this goal. This requires first defining what “better” means, leading to the idea of outcomes that are “objective” and then to ones that are actionable, with a meaningful evaluation measure. We will discuss some of the subtle issues in this exploration – what does “objective” mean, the role of the (perhaps personalized) evaluation function, multi-step actions, counterfactual issues, distributional evaluations, etc. Collectively, this analysis argues we should learn models whose outcome labels are objective and actionable, as that will lead to tools that are useful and cost-effective.
Presenter Bio:
Russ Greiner worked in both academic and industrial research before settling at the University of Alberta, where he is now a Professor in Computing Science and the founding Scientific Director of the Alberta Machine Intelligence Institute. He has been Program/Conference Chair for various major conferences, and has served on the editorial boards of a number of other journals. He was elected a Fellow of the AAAI, has been awarded a McCalla Professorship and a Killam Annual Professorship; and in 2021, received the CAIAC Lifetime Achievement Award and became a CIFAR AI Chair. In 2022, the Telus World of Science museum honored him with a panel, and he received the (UofA) Precision Health Innovator Award, then in 2023, he received the CS-Can | Info-Can Lifetime Achievement Award. For his mentoring, he received a 2020 FGSR Great Supervisor Award, then in 2023, the Killam Award for Excellence in Mentoring. He has published over 300 refereed papers, most in the areas of machine learning and recently medical informatics, including 5 that have been awarded Best Paper prizes. The main foci of his current work are (1) bio- and medical- informatics; (2) survival prediction; and (3) formal foundations of learnability.
Carmack and Sutton are deeply focused on developing a genuine AI prototype by 2030, including establishing, advancing, and documenting AGI signs of life.
The announcement is followed by an engaging fireside chat with John Carmack and Amii Fellow and Canada CIFAR AI Chair Michael Bowling. Watch the conversation as they discuss innovation in games, open source, virtual reality, rockets and John's vision for the future of AI.
0:00 Keen Technologies Partnership Announcement
16:35 Fireside Chat with John Carmack and Michael Bowling
Abstract:
Reinforcement Learning from Human Feedback (RLHF) has emerged as the central alignment technique used to finetune state-of-the-art AI systems such as GPT-4, Claude, Bard, and Llama-2. Given RLHF's status as the default industry alignment technique, there is a need to carefully study how we got here and what challenges persist in today's state of the art. We review open challenges and fundamental limitations with RLHF with a focus on applications in large language models. Technical progress in some respects is tractable, and this should be seen as a cause for concerted work and optimism. However, other problems with RLHF cannot fully be solved and instead must be avoided or compensated for with non-RLHF approaches.
Presenter Bio:
Stephen "Casp" Casper is a Ph.D student at MIT in Computer Science (EECS) in the Algorithmic Alignment Group advised by Dylan Hadfield-Menell. His research focuses include interpreting, diagnosing, debugging, and auditing deep learning systems.
Abstract:
In this presentation, I argue for a novel theory of lexical semantics. I begin by presenting a theoretical model for wordnets, a class of resources commonly used in lexical semantics. This is followed by a novel analysis of semantic tasks themselves, wherein I propose a first-of-its-kind taxonomy of semantic tasks. For both parts of the presentation, I demonstrate, via experimental evidence, that such theoretical developments are an important contribution to contemporary semantics research.
Presenter Bio:
Bradley Hauer is a PhD candidate at the University of Alberta. He has published more than 30 papers in refereed venues, earning the NAACL 2013 Best Student Paper Award, and a nomination for the IEEE ICSC 2019 Best Paper Award. His work covers a variety of subjects in natural language processing, including multilingual semantics, classical decipherment, and cognate identification. His present research is motivated by a strong interest in theoretical models and explainable methods.
Abstract:
The world provides an abundance of information, but often the majority of possible inputs to an RL agent are irrelevant or even distracting to its current task. We cover two approaches aimed at enhancing performance in challenging environments.
First, the Automatic Noise Filtering (ANF) method is introduced, which learns to focus its efficient sparse connectivity on task-relevant features, outperforming dense algorithms with significantly fewer weights. Second, we highlight MaDi, a method that masks distracting visuals in image-based reinforcement learning, improving focus and generalization. We will conclude with possible extensions of these works, providing an opportunity for discussion and potential collaborations.
Presenter Bio:
Bram Grooten is a PhD student at the TU Eindhoven in the Netherlands, supervised by Decebal Mocanu and Mykola Pechenizkiy. Currently he is visiting the IRL Lab of Matthew Taylor at the University of Alberta until December. His research focuses on Sparse Training, Reinforcement Learning, Generalization, and Autonomous Driving.
Abstract:
When players engage with parts of a video game they do not enjoy, one proposed solution is to adapt the game to the player’s preference. In industry, this adaptation is called “AI directors”, but these are uncommon, domain-specific, and rule-based. In this talk, I will present Nightingale, a reinforcement learning-based AI director developed for the industry game. My team evaluated the effectiveness of this AI director, but found inconclusive evidence. I will present the results and their implications for future AI directors.
Presenter Bio:
Kristen Yu is a PhD Candidate at the University of Alberta supervised by Nathan Sturtevant and co-supervised by Matthew Guzdial. Her research focuses on applying AI techniques to video game design to create a better player experience.
Abstract:
Learning from experience means remaining adaptive and responsive to errors over time. However, gradient-based deep learning can fail dramatically in the continual, online setting. In this work we address this shortcoming by combining two meta-learning methods: the purely online Partition Tree Weighting (PTW) mixture-of-experts algorithm, and a novel variant of the Model-Agnostic Meta-Learning (MAML) initialization-learning procedure. We demonstrate our approach, Replay-MAML PTW, in a piecewise stationary classification task. In this continual, online setting, Replay-MAML PTW matches and even outperforms an augmented learner that is allowed to pre-train offline and is given explicit notification when the task changes. Replay-MAML PTW thus provides a base learner with the benefits of offline training, explicit task sampling, and boundary notification, all for a O(log2 (t)) increase in computation and memory. This makes deep learning more viable for fully online, task-agnostic continual learning, which is at the heart of general intelligence.
Presenter Bio:
Anna Koop recently started as a Research Engineer at Google DeepMind in Montreal, having just about completed her PhD on experiential learning for AI, supervised by Michael Bowling at the University of Alberta. She's been a teacher, programmer, consultant, manager and student in almost every permutation, including Managing Director of the Applied Machine Learning Team at Amii. She is fascinated by learning, can't stay away from research, and is passionately curious about nearly everything.
Set to launch in January 2024, the course is the cornerstone activity for an in-development AI certification — one of the first of its kind to be offered by a Canadian university.
Abstract:
When formulating a problem as a reinforcement learning (RL) task, one of the crucial steps is determining the reward function. Ideally, we seek reward functions that accurately capture the intended problem, are convenient to specify, and facilitate the agent with learning. Many real-world robot learning problems, such as pick-and-place or arriving at a destination, can be seen as a problem of reaching a goal state as soon as possible. These problems can be naturally formulated as episodic RL tasks with termination upon reaching the goal state. For such tasks, a -1 reward every time step until termination is easy to specify and aligns well with our intention. However, such sparse-reward formulations are avoided in practice, as they are thought to be difficult and uninformative for learning. Hence, RL practitioners often construct rewards that reflect the task designer's domain knowledge and preferred behaviours. These handcrafted rewards go beyond specifying what task to solve and guide the agent on how to solve the task. First, I'll elucidate how such guiding rewards can often bias the learned control policy in a potentially sub-optimal way. Then I'll demonstrate the superiority of control policies learned using sparse rewards over guided rewards. Contrary to popular belief, I'll show that it is possible to have robust, reliable learning from scratch on challenging vision-based robotics tasks using only sparse rewards in just a few hours.
Abstract:
At Nanoprecise, we are on a mission to save 2-3% of global carbon emissions by providing right maintenance insight at the right time with fully automated IoT & AI powered condition monitoring of motor driven equipment such as pumps, compressors etc. In this session, we will talk about how we are achieving the same for the asset intensive industries such as Oil & Gas, Manufacturing, Pharma, Cement, Mining etc. We will talk about how Nanoprecise's AI approach, which is physics + AI enabled is giving us an edge over traditional black box AI models such as neural network applied on random data.
Presenter Bio:
Sunil Vedula, CEO and Engineer - is an avid reader and technology commercialization enthusiast. He attained his MBA from the University of Alberta, specializing in Finance and Natural Resource, Energy & Environment related topics. He presented various papers on commercialization potential of alternative technologies such as Hydrogen Fuel Cell, Geothermal Energy to name a few. He interned at a major Asian investment banking firm IDFC Ltd. He attained his Mechanical Engineering from Delhi College of Engineering. He has more than 7 years of experience as a Mechanical Engineer specializing in material stress & structural analysis.
Abstract:
Off-policy policy evaluation has been a critical and challenging problem in reinforcement learning, and Temporal-Difference (TD) learning is one of the most important approaches for addressing it. There has been significant interest in searching for consistent off-policy TD algorithms that are guaranteed to find the on-policy TD fixed point. Notably, Full Importance-Sampling TD is the only existing consistent off-policy TD method under general linear function approximation but, unfortunately, has a high variance and is scarcely practical. This notorious high variance issue motivates the introduction of Emphatic TD, which tames down the variance but has a biased fixed point. Inspired by these two methods, we propose a new consistent algorithm called Average Emphatic TD (AETD) with a transient bias, which strikes a balance between bias and variance. Further, we unify AETD with several existing algorithms and obtain a new family of consistent algorithms called Consistent Emphatic TD (CETD), which can control a smooth bias-variance trade-off by varying the speed at which the transient bias fades. Through theoretical analysis and experiments on a didactic example, we validate the consistency of CETD. Moreover, we show that CETD converges faster to the lowest error in a complex task with a high variance.
Presenter Bio:
Jiamin is an M.Sc. student at the University of Alberta working with Rupam Mahmood. His research interests lie in artificial intelligence, especially reinforcement learning. Currently, he is focusing on off-policy policy evaluation.
Abstract:
This talk presents the approach to grounding concepts to vision using text descriptions that focus on visual attributes. The research includes two studies. The first explores visual word sense disambiguation, a task that involves selecting the image that best represents the contextual meaning of a word. The second study focuses on producing visual descriptions for arbitrary, concrete concepts for use in downstream tasks such as zero-shot image classification and generation. Both studies show that conditioning a pre-trained large language model (LLM) with lexico-semantic knowledge empirically improves visual descriptions, thus confirming the utility of LLM-produced visual descriptions in grounding lexical concepts to the visual domain.
Presenter Bio:
Mike Ogezi is completing his MSc at the University of Alberta, focusing on the intersection of natural language processing and computer vision (supervisor: Dr Greg Kondrak). He is particularly interested in enhancing large language models with visual knowledge. Before his MSc, he was a software engineer working on optimal order matching for cryptocurrency trading. In his downtime, he enjoys playing board and video games.
Abstract:
Experience replay, which stores transitions in a replay memory for repeated use, plays an important role of improving sample efficiency in reinforcement learning. Existing techniques such as reweighted sampling, episodic learning and reverse sweep update process the information in the replay memory to make experience replay more efficient. In this work, we further exploit the information in the replay memory by treating it as an empirical Replay Memory MDP (RM-MDP). By solving it with dynamic programming, we learn a conservative value estimate that only considers transitions observed in the replay memory. Both value and policy regularizers based on this conservative estimate are developed and integrated with model-free learning algorithms. We design the memory density metric to measure the quality of RM-MDP. Our empirical studies quantitatively find a strong correlation between performance improvement and memory density. Our method combines Conservative Estimation with Experience Replay (CEER), improving sample efficiency by a large margin, especially when the memory density is high. Even when the memory density is low, such a conservative estimate can still help to avoid suicidal actions and thereby improve performance.
Presenter Bio:
Hongming Zhang is a Ph.D. student at the University of Alberta working with Martin Müller. He is interested in deep reinforcement learning (DRL) and planning/search techniques. Currently he is particularly focused on enhancing the sample efficiency and learning performance of DRL algorithms through the integration of planning and search.
Abstract:
A default assumption in reinforcement learning (RL) and optimal control is that observations arrive at discrete time points on a fixed clock cycle. Yet, many applications involve continuous-time systems where the time discretization, in principle, can be managed. The impact of time discretization on RL methods has not been fully characterized in existing theory, but a more detailed analysis of its effect could reveal opportunities for improving data-efficiency.
Presenter Bio:
Zichen (Vincent) Zhang is a Ph.D. student in Computing Science at University of Alberta, working with Prof. Dale Schuurmans and Prof. Martin Jagersand. He is interested in machine learning and its applications on robotics perception and control.
Abstract:
Machine teaching deals with the problem of encoding a hypothesis in a small number of labeled examples, called a teaching set. Thus a machine learner can infer the target hypothesis from a small amount of high-quality data, as opposed to a large amount of random data. However, the interaction between teacher and learner is constrained by formal rules, so as to prevent unfair collusion. One central goal is to establish that the same structural properties that allow for learning from a certain number of random examples also allow for (proportionally smaller) collusion-free teaching sets. This presentation introduces the first teaching model achieving this goal, using a new approach to modelling collusion-freeness. (Joint work with Farnam Mansouri, Hans Simon, Adish Singla.)
Presenter Bio:
Dr. Sandra Zilles is a Professor of Computer Science at the University of Regina, where she holds a Tier 1 Canada Research Chair in Computational Learning Theory. She is also a Canada CIFAR AI Chair at the Alberta Machine Intelligence Institute (Amii). Her research concerns mostly theoretical foundations of AI and machine learning, and has been recognized with multiple awards. In 2017, she was inducted into the College of New Scholars, Artists and Scientists of the Royal Society of Canada. Sandra has served on the editorial boards for the Journal of Computer and System Sciences and for the IEEE Transactions on Pattern Analysis and Machine Intelligence. On the Board of Directors for the Pacific Institute for the Mathematical Sciences (PIMS) and on the Board of Directors for Innovation Saskatchewan, she provided guidance on policies for research, innovation, and education in computer science and mathematics.
Abstract:
Industrial environments, such as manufacturing and warehousing, are quickly becoming complex. There are various manual and repetitive tasks which, if not automated, will drastically impact the company's operational efficiency. At Aro, we have developed a novel technology of AI-based multitasking robots. These robots are autonomous and, unlike competitors, can perform more than one task. They heavily rely on various aspects of AI to make real-time decisions.
Presenter Bio:
Dr. Faheem Khan has a PhD in Electrical Engineering and an MBA in strategy. He has been involved with technology start ups for more than ten years. Before starting Aro, he founded Fourien, which develops micro sensors and analytical instruments for the diagnostics industry. Currently he is the CEO at both Aro and Fourien.
Abstract:
The end-to-end learning ability of self-driving vehicles has achieved significant milestones over the last decade owing to rapid advances in deep learning and computer vision algorithms. However, as autonomous driving technology is a safety-critical application of artificial intelligence (AI), road accidents and established regulatory principles necessitate the need for the explainability of intelligent action choices for self-driving vehicles. To facilitate interpretability of decision-making in autonomous driving, we present a Visual Question Answering (VQA) framework, which explains self-driving actions with question-answering-based causal reasoning. To do so, we first collect driving videos in a simulation environment using reinforcement learning (RL) and extract consecutive frames from this log data uniformly for five selected action categories. Further, we manually annotate the extracted frames using question-answer pairs as justifications for the actions chosen in each scenario. Finally, we evaluate the correctness of the VQA-predicted answers for actions on unseen driving scenes. The empirical results suggest that the VQA mechanism can provide support to interpret real-time decisions of autonomous vehicles and help enhance overall driving safety.
Presenter Bio:
Shahin Atakishiyev received a BSc Computer Engineering degree from Qafqaz University, Azerbaijan, in June 2015 and an MSc Computer Engineering degree with a specialization in Software Engineering and Intelligent Systems from the University of Alberta, Canada, in January 2018. Currently, he is a PhD student in the Department of Computing Science at the University of Alberta under the supervision of Prof. Randy Goebel. Shahin's research interests include Safe, Ethical, and Explainable Artificial Intelligence and its applications to real-world problems.
Recognizing dialog acts of users is an essential component in building successful conversational agents. In this work, we propose a dialog act (DA) classifier for two of our open domain conversational agents. For this, we curated a high-quality, multi-domain dataset with ∼24k user utterances labelled into 8 suitable DAs. Our fine-tuned BERT-based model outperforms the baseline SVM classifier by achieving state-of-the-art accuracy on the proposed dataset. Moreover, it generalizes well on unseen data. To address the issue of data scarcity when training DA classifiers, we implemented different data augmentation techniques and compared their performance. Our extensive experiments show that, in a simulated low data regime with only 10 examples per label, methods as simple as synonym replacement can double the size of the existing training data and boost accuracy of our DA classifier by ∼8%. Lastly, we demonstrate how our proposed classifier and augmentation techniques can be adapted to effectively detect dialog acts in languages other than English.
Presenter Bio:
Maliha is a recent Masters graduate from the University of Alberta in the field of Natural Language Processing. She has extensively worked with Large Language Models for detecting user dialog acts in chatbots. Moreover, to tackle the scarcity of labelled training data, she has successfully looked into data augmentation techniques that artificially generate new data to improve model performance. When she is not busy scrolling through LinkedIn looking for a new full-time position as an ML Developer, you can find her singing at a karaoke bar or playing at a board games café. Currently, she is building data pipelines and doing prompt engineering as an Associate ML Developer at AltaML.
Across the research, business, political and societal spectrum, there are different ideas, excitement, and concerns about the future of AI. Academics and research community members are encouraged to voice their individual point-of-view at Upper Bound without restraint. Amii is committed to providing venues to discuss the future of AI in all its forms.
Across the research, business, political and societal spectrum, there are different ideas, excitement, and concerns about the future of AI. Academics and research community members are encouraged to voice their individual point-of-view at Upper Bound without restraint. Amii is committed to providing venues to discuss the future of AI in all its forms.
As a leader in the field of machine learning, Rich has co-authored the seminal textbook Reinforcement Learning: An Introduction, co-founded DeepMind Alberta in 2017 and has had his publications cited more than 93,000 times. He’s also been featured in multiple publications such as the Economist, the New York Times and the Wall Street Journal. No matter your level of AI expertise, this keynote will leave you intrigued and inspired.
Across the research, business, political and societal spectrum, there are different ideas, excitement, and concerns about the future of AI. Academics and research community members are encouraged to voice their individual point-of-view at Upper Bound without restraint. Amii is committed to providing venues to discuss the future of AI in all its forms.


