Uploaded October 2024 | Updated September 2026, 1 week ago
MLST is sponsored by Tufa Labs:
Are you interested in working on ARC and cutting-edge AI research with the MindsAI team (current ARC winners)?
Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more.
Future plans: Expanding to complex environments like Warcraft 2 and Starcraft 2.
Interested? Apply for an ML research position: benjamin@tufa.ai
Professor Alexander Ororbia from the Rochester Institute of Technology takes Tim Scarfe through the case for bio-inspired AI. The central idea is mortal computation: you cannot divorce the software from the hardware that runs it. The brain manages remarkable things on a few watts because its computations are entangled with its physical substrate. GPT-class models, running on von Neumann architectures designed for immortal computation -- where software and hardware are deliberately decoupled -- pay a staggering energy penalty for that separation.
Ororbia explains the building blocks: Markov blankets as the formalism for system boundaries, Karl Friston's free energy principle as the optimization target, and the MILLS framework (Mortal Inference, Learning, and Selection) operating across multiple timescales. He then surveys the landscape of alternatives to backpropagation -- predictive coding, Hebbian learning, contrastive methods, and Geoff Hinton's forward-forward algorithm -- showing how each maps to observations from neuroscience.
The conversation gets practical with Ororbia's ngc-learn library for implementing these algorithms, the stability-plasticity dilemma in continual learning, and the current state of neuromorphic hardware from Intel Loihi to IBM TrueNorth. He closes with his neural generative coding work, which showed that predictive coding networks can synthesize data they were never trained on -- outperforming VAEs and GANs -- and his vision for bio-inspired AI systems that coexist with humanity rather than replacing it.
---
TIMESTAMPS:
00:00:00 Introduction to Bio-Inspired AI and Mortal Computation
00:04:50 Principles of Mortal Computation
00:17:41 Markov Blankets and Free Energy Principle
00:24:38 MILLS Framework and Biological Systems
00:31:00 Challenging Backpropagation: Alternative Approaches
00:31:49 Predictive Coding and Free Energy Principle
00:41:52 Biologically Plausible Credit Assignment Methods
00:50:11 Taxonomy of Bio-inspired Learning Algorithms
00:59:30 Forward-Only Learning and ngc-learn Implementation
01:03:25 Stability-Plasticity Dilemma and Continual Learning
01:09:00 Neuromorphic Hardware and Challenges
01:12:58 Neural Generative Coding and Future Directions
---
REFERENCES:
website:
[00:04:43] The Levin Lab
drmichaellevin.org
[00:18:20] Good Regulator Theorem
en.wikipedia.org/wiki/Good_regulator
[00:41:52] Hebbian Theory
en.wikipedia.org/wiki/Hebbian_theory
[00:45:00] Hopfield Network
en.wikipedia.org/wiki/Hopfield_network
[01:09:00] Intel Loihi 2
intel.com/content/www/us/en/research/neuromorphic-computing-loihi-2-technology-brief.html
paper:
[00:04:50] Mortal Computation: A Foundation for Biomimetic Intelligence
arxiv.org/abs/2311.09589
[00:06:53] The Forward-Forward Algorithm
arxiv.org/abs/2212.13345
[00:07:20] There's Plenty of Room Right Here
ncbi.nlm.nih.gov/pmc/articles/PMC10046700
[00:17:41] The Free-Energy Principle: A Rough Guide to the Brain
fil.ion.ucl.ac.uk/~karl/The%20free-energy%20principle%20-%20a%20rough%20guide%20to%20the%20brain.pdf
[00:31:49] Predictive Coding in the Visual Cortex
nature.com/articles/nn0199_79
[00:41:52] Brain-Inspired Machine Intelligence: Neurobiologically-Plausible Credit Assignment
arxiv.org/abs/2312.09257
[00:45:50] A Tutorial on Energy-Based Learning
yann.lecun.com/exdb/publis/pdf/lecun-06.pdf
[00:46:40] A Learning Algorithm for Boltzmann Machines
https://www.cs.toronto.edu/~hinton/absps/cogscibm.pdf
[00:50:11] A Review of Neuroscience-Inspired Machine Learning
arxiv.org/abs/2403.18929
[00:53:20] NEAT: NeuroEvolution of Augmenting Topologies
https://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf
[00:56:40] A Path Towards Autonomous Machine Intelligence
openreview.net/pdf?id=BZ5a1r-kVsf
[00:59:30] Test-Time Model Adaptation with Only Forward Passes
arxiv.org/abs/2404.01650
[01:03:25] Spiking Neural Predictive Coding for Continual Learning
sciencedirect.com/science/article/pii/S0925231223004150
[01:10:00] IBM TrueNorth
research.ibm.com/publications/truenorth-design-and-tool-flow-of-a-65-mw-1-million-neuron-programmable-neurosynaptic-chip
book:
[00:24:38] Active Inference: The Free Energy Principle in Mind, Brain, and Behavior
https://direct.mit.edu/books/oa-monograph/5299/Active-InferenceThe-Free-Energy-Principle-in-Mind
---
LINKS:
Full Transcript: app.rescript.info/share/cfa14d0f39f86d035d5caf6173d6207f
Download PDF transcript: app.rescript.info/api/public/sessions/94d5c32d7c507e6a/pdf
MLST is sponsored by Tufa Labs:
Are you interested in working on ARC and cutting-edge AI research with the MindsAI team (current ARC winners)?
Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more.
Future plans: Expanding to complex environments like Warcraft 2 and Starcraft 2.
Interested? Apply for an ML research position: benjamin@tufa.ai
Professor Alexander Ororbia from the Rochester Institute of Technology takes Tim Scarfe through the case for bio-inspired AI. The central idea is mortal computation: you cannot divorce the software from the hardware that runs it. The brain manages remarkable things on a few watts because its computations are entangled with its physical substrate. GPT-class models, running on von Neumann architectures designed for immortal computation -- where software and hardware are deliberately decoupled -- pay a staggering energy penalty for that separation.
Ororbia explains the building blocks: Markov blankets as the formalism for system boundaries, Karl Friston's free energy principle as the optimization target, and the MILLS framework (Mortal Inference, Learning, and Selection) operating across multiple timescales. He then surveys the landscape of alternatives to backpropagation -- predictive coding, Hebbian learning, contrastive methods, and Geoff Hinton's forward-forward algorithm -- showing how each maps to observations from neuroscience.
The conversation gets practical with Ororbia's ngc-learn library for implementing these algorithms, the stability-plasticity dilemma in continual learning, and the current state of neuromorphic hardware from Intel Loihi to IBM TrueNorth. He closes with his neural generative coding work, which showed that predictive coding networks can synthesize data they were never trained on -- outperforming VAEs and GANs -- and his vision for bio-inspired AI systems that coexist with humanity rather than replacing it.
---
TIMESTAMPS:
00:00:00 Introduction to Bio-Inspired AI and Mortal Computation
00:04:50 Principles of Mortal Computation
00:17:41 Markov Blankets and Free Energy Principle
00:24:38 MILLS Framework and Biological Systems
00:31:00 Challenging Backpropagation: Alternative Approaches
00:31:49 Predictive Coding and Free Energy Principle
00:41:52 Biologically Plausible Credit Assignment Methods
00:50:11 Taxonomy of Bio-inspired Learning Algorithms
00:59:30 Forward-Only Learning and ngc-learn Implementation
01:03:25 Stability-Plasticity Dilemma and Continual Learning
01:09:00 Neuromorphic Hardware and Challenges
01:12:58 Neural Generative Coding and Future Directions
---
REFERENCES:
website:
[00:04:43] The Levin Lab
drmichaellevin.org
[00:18:20] Good Regulator Theorem
en.wikipedia.org/wiki/Good_regulator
[00:41:52] Hebbian Theory
en.wikipedia.org/wiki/Hebbian_theory
[00:45:00] Hopfield Network
en.wikipedia.org/wiki/Hopfield_network
[01:09:00] Intel Loihi 2
intel.com/content/www/us/en/research/neuromorphic-computing-loihi-2-technology-brief.html
paper:
[00:04:50] Mortal Computation: A Foundation for Biomimetic Intelligence
arxiv.org/abs/2311.09589
[00:06:53] The Forward-Forward Algorithm
arxiv.org/abs/2212.13345
[00:07:20] There's Plenty of Room Right Here
ncbi.nlm.nih.gov/pmc/articles/PMC10046700
[00:17:41] The Free-Energy Principle: A Rough Guide to the Brain
fil.ion.ucl.ac.uk/~karl/The%20free-energy%20principle%20-%20a%20rough%20guide%20to%20the%20brain.pdf
[00:31:49] Predictive Coding in the Visual Cortex
nature.com/articles/nn0199_79
[00:41:52] Brain-Inspired Machine Intelligence: Neurobiologically-Plausible Credit Assignment
arxiv.org/abs/2312.09257
[00:45:50] A Tutorial on Energy-Based Learning
yann.lecun.com/exdb/publis/pdf/lecun-06.pdf
[00:46:40] A Learning Algorithm for Boltzmann Machines
https://www.cs.toronto.edu/~hinton/absps/cogscibm.pdf
[00:50:11] A Review of Neuroscience-Inspired Machine Learning
arxiv.org/abs/2403.18929
[00:53:20] NEAT: NeuroEvolution of Augmenting Topologies
https://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf
[00:56:40] A Path Towards Autonomous Machine Intelligence
openreview.net/pdf?id=BZ5a1r-kVsf
[00:59:30] Test-Time Model Adaptation with Only Forward Passes
arxiv.org/abs/2404.01650
[01:03:25] Spiking Neural Predictive Coding for Continual Learning
sciencedirect.com/science/article/pii/S0925231223004150
[01:10:00] IBM TrueNorth
research.ibm.com/publications/truenorth-design-and-tool-flow-of-a-65-mw-1-million-neuron-programmable-neurosynaptic-chip
book:
[00:24:38] Active Inference: The Free Energy Principle in Mind, Brain, and Behavior
https://direct.mit.edu/books/oa-monograph/5299/Active-InferenceThe-Free-Energy-Principle-in-Mind
---
LINKS:
Full Transcript: app.rescript.info/share/cfa14d0f39f86d035d5caf6173d6207f
Download PDF transcript: app.rescript.info/api/public/sessions/94d5c32d7c507e6a/pdf
![AI Agents can write 10,000 lines of hacking code in seconds [Dr. Ilia Shumailov]](https://i.ytimg.com/vi/aoX_pGQMbEM/mqdefault.jpg "AI Agents can write 10,000 lines of hacking code in seconds [Dr. Ilia Shumailov]
Dr. Ilia Shumailov - Former DeepMind AI Security Researcher, now building security tools for AI agents
Ever wondered what happens when AI agents start talking to each other—or worse, when they start breaking things? Ilia Shumailov spent years at DeepMind thinking about exactly these problems, and hes here to explain why securing AI is way harder than you think.
**SPONSOR MESSAGES**
—
Check out notebooklm for your research project, its really powerful
https://notebooklm.google.com/
—
Take the Prolific human data survey - https://www.prolific.com/humandatasurvey?utm_source=mlst and be the first to see the results and benchmark their practices against the wider community!
—
cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy
Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++
Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst
Submit investment deck: https://cyber.fund/contact?utm_source=mlst
—
Were racing toward a world where AI agents will handle our emails, manage our finances, and interact with sensitive data 24/7. But there is a problem. These agents are nothing like human employees. They never sleep, they can touch every endpoint in your system simultaneously, and they can generate sophisticated hacking tools in seconds. Traditional security measures designed for humans simply wont work.
Dr. Ilia Shumailov
https://x.com/iliaishacked
https://iliaishacked.github.io/
https://sequrity.ai/
TRANSCRIPT:
https://app.rescript.info/public/share/dVGsk8dz9_V0J7xMlwguByBq1HXRD6i4uC5z5r7EVGM
More from Ilia on our Patreon:
https://www.patreon.com/posts/116142401/ (interview from last year)
https://www.patreon.com/posts/ilia-shumailov-140359158 (extended version of this interview)
TOC:
00:00:00 - Introduction & Trusted Third Parties via ML
00:03:45 - Background & Career Journey
00:06:42 - Safety vs Security Distinction
00:09:45 - Prompt Injection & Model Capability
00:13:00 - Agents as Worst-Case Adversaries
00:15:45 - Personal AI & CAML System Defense
00:19:30 - Agents vs Humans: Threat Modeling
00:22:30 - Calculator Analogy & Agent Behavior
00:25:00 - IMO Math Solutions & Agent Thinking
00:28:15 - Diffusion of Responsibility & Insider Threats
00:31:00 - Open Source Security Concerns
00:34:45 - Supply Chain Attacks & Trust Issues
00:39:45 - Architectural Backdoors
00:44:00 - Academic Incentives & Defense Work
00:48:30 - Semantic Censorship & Halting Problem
00:52:00 - Model Collapse: Theory & Criticism
00:59:30 - Career Advice & Ross Anderson Tribute
REFS:
Lessons from Defending Gemini Against Indirect Prompt Injections
https://arxiv.org/abs/2505.14534
Defeating Prompt Injections by Design. Google, Google DeepMind, and ETH Zurich. (CAML)
Debenedetti, E., Shumailov, I., Fan, T., Hayes, J., Carlini, N., Fabian, D., Kern, C., Shi, C., Terzis, A., & Tramèr, F.
https://arxiv.org/pdf/2503.18813
Agentic Misalignment: How LLMs could be insider threats
https://www.anthropic.com/research/agentic-misalignment
STOP ANTHROPOMORPHIZING INTERMEDIATE TOKENS AS REASONING/THINKING TRACES!
Subbarao Kambhampati et al
https://arxiv.org/pdf/2504.09762
Meiklejohn, S., Blauzvern, H., Maruseac, M., Schrock, S., Simon, L., & Shumailov, I. (2025).
Machine learning models have a supply chain problem.
https://arxiv.org/abs/2505.22778
Gao, Y., Shumailov, I., & Fawaz, K. (2025).
Supply-chain attacks in machine learning frameworks.
In Proceedings of the 8th MLSys Conference.
https://openreview.net/pdf?id=EH5PZW6aCr
Apache Log4j Vulnerability Guidance
https://www.cisa.gov/news-events/news/apache-log4j-vulnerability-guidance
Bober-Irizar, M., Shumailov, I., Zhao, Y., Mullins, R., & Papernot, N. (2023).
Architectural backdoors in neural networks.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 21163–21173).
Bober-Irizar, M., Shumailov, I., Zhao, Y., Mullins, R., & Papernot, N. (2022).
Architectural backdoors in neural networks. arXiv preprint arXiv:2206.07840.
https://arxiv.org/pdf/2206.07840
Langford, H., Shumailov, I., Zhao, Y., Mullins, R., & Papernot, N. (2024).
Architectural neural backdoors from first principles.
arXiv preprint arXiv:2402.06957.
Küchler, N., Petrov, I., Grobler, C., & Shumailov, I. (2025).
Architectural backdoors for within-batch data stealing and model inference manipulation.
arXiv preprint arXiv:2505.18323.
Position: Fundamental Limitations of LLM Censorship Necessitate New Approaches
David Glukhov, Ilia Shumailov, Yarin Gal, Nicolas Papernot, Vardan Papyan
https://proceedings.mlr.press/v235/glukhov24a.html
AlphaEvolve MLST interview [Matej Balog, Alexander Novikov]
https://www.youtube.com/watch?v=vC9nAosXrJw")
![The Weird ChatGPT Hack That Leaked Training Data [Dr. Yannic Kilcher / Prof. Florian Tramer]](https://i.ytimg.com/vi/c_hmxRVDXBE/mqdefault.jpg "The Weird ChatGPT Hack That Leaked Training Data [Dr. Yannic Kilcher / Prof. Florian Tramer]
Dr. Yannic lightspeed Kilcher sits down with an AI Security Researcher Prof. Florian Tramer (an expert in the field, collaborating with folks like Nicholas Carlini and Milad Nasr at Google) to ruminate on the murky state of affairs in AI security.
Out of the gate they assert that AI detectors (those tools that try to spot if text is human-written or AI-generated) are basically doomed to fail—like trying to catch an eel with greasy hands blindfolded. Lightspeed Kilcher shares a funny story about how the word Delve kept rearing its ugly head in AI text because of training data emanating from Nigerian crowd workers, leading to false flags in job applications.
Tramer then takes the gloves off about security risks, saying that even a 99% success rate in AI means total failure in security terms—attackers will always find that 1% weak spot. They share an interesting example of a weird attack on ChatGPT: asking it to repeat poem forever made it spit out memorized internet scraps, leaking training data like a sieve. OpenAI patched it, but its like slapping a band-aid on a shark bite.
Lightspeed Kilcher asks about top worries, and Tramer lists three big ones in relatable terms: 1) Training AI on private data (like medical records) could lead to massive leaks, since models memorize stuff without us fully understanding how; 2) Prompt injections, where bad actors hijack AI agents to do harmful things, like Anthropics demo that controls computers—its like giving your keys to a stranger; and 3) The rush to deploy AI without safeguards, echoing past tech blunders like SQL hacks.
They reflect on how ChatGPT has lit a fire under AI security research—making it amazing and scary by turning hypotheticals into real-world issues with millions of users. The conversation touches on why just scaling up AI wont fix everything (we need causal understanding, not just more data), and why watermarking AI outputs isnt a silver bullet—its easy to bypass, especially with open-source models.
Its about time we had our old comrade light speed Kilcher back on MLST!
TRANSCRIPT:
https://app.rescript.info/public/share/PGieXFZCRyJjJswicatmNFia_pKCt_WVP-pZS4qguiw
LINKS:
Scalable Extraction of Training Data from (Production) Language Models [Poem Attack]
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, Colin Raffel
https://arxiv.org/abs/2311.17035
- Prof. Florian Tramer
https://www.floriantramer.com/
- Yannic Kilchers YouTube channel
https://www.youtube.com/c/YannicKilcher
- Nicholas Carlinis research profile (collaborator on the data recovery project):
https://nicholas.carlini.com/ – Includes papers on AI attacks and memorization risks.")
![AI training data will never be fully synthetic [SPONSORED]](https://i.ytimg.com/vi/cnxZZTl1tkk/mqdefault.jpg "AI training data will never be fully synthetic [SPONSORED]
We sat down with Sara Saab (VP of Product at Prolific) and Enzo Blindow (VP of Data and AI at Prolific) to explore the critical role of human evaluation in AI development and the challenges of aligning AI systems with human values. Prolific is a human annotation and orchestration platform for AI used by many of the major AI labs. This is a sponsored show in partnership with Prolific.
**SPONSOR MESSAGES**
—
cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy
Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++
Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst
Submit investment deck: https://cyber.fund/contact?utm_source=mlst
—
While technologists want to remove humans from the loop for speed and efficiency, these non-deterministic AI systems actually require more human oversight than ever before. Prolifics approach is to put well-treated, verified, diversely demographic humans behind an API - making human feedback as accessible as any other infrastructure service.
When AI models like Grok 4 achieve top scores on technical benchmarks but feel awkward or problematic to use in practice, it exposes the limitations of our current evaluation methods. The guests argue that optimizing for benchmarks may actually weaken model performance in other crucial areas, like cultural sensitivity or natural conversation.
We also discuss Anthropics research showing that frontier AI models, when given goals and access to information, independently arrived at solutions involving blackmail - without any prompting toward unethical behavior. Even more concerning, the more sophisticated the model, the more susceptible it was to this agentic misalignment.
Enzo and Sarah present Prolifics Humane leaderboard as an alternative to existing benchmarking systems. By stratifying evaluations across diverse demographic groups, they reveal that different populations have vastly different experiences with the same AI models.
Looking forwards the guests imagine a world where humans take on coaching and teaching roles for AI systems - similar to how we might correct a child or review code. Working conditions and the evolution of labor in an AI-augmented world is also important to consider. Rather than replacing humans entirely, we may be moving toward more sophisticated forms of human-AI collaboration.
We need more representative evaluation frameworks that capture the messy reality of human values and cultural diversity.
Visit Prolific:
https://www.prolific.com/
Sara Saab (VP Product):
https://uk.linkedin.com/in/sarasaab
Enzo Blindow (VP Data & AI):
https://uk.linkedin.com/in/enzoblindow
TRANSCRIPT:
https://app.rescript.info/public/share/xZ31-0kJJ_xp4zFSC-bunC8-hJNkHpbm7Lg88RFcuLE
TOC:
[00:00:00] Intro & Background
[00:03:16] Human-in-the-Loop Challenges
[00:17:19] Can AIs Understand?
[00:32:02] Benchmarking & Vibes
[00:51:00] Agentic Misalignment Study
[01:03:00] Data Quality vs Quantity
[01:16:00] Future of AI Oversight
REFS:
Anthropic Agentic Misalignment
https://www.anthropic.com/research/agentic-misalignment
Value Compass
https://arxiv.org/pdf/2409.09586
Reasoning Models Don’t Always Say What They Think (Anthropic)
https://www.anthropic.com/research/reasoning-models-dont-say-think
https://assets.anthropic.com/m/71876fabef0f0ed4/original/reasoning_models_paper.pdf
Maslow’s Hierarchy Of Needs
https://www.simplypsychology.org/maslow.html
Apollo research - science of evals blog post
https://www.apolloresearch.ai/blog/we-need-a-science-of-evals
Leaderboard Illusion
https://www.youtube.com/watch?v=9W_OhS38rIE MLST video
The Leaderboard Illusion [2025]
Shivalika Singh, Yiyang Nan, Alex Wang, Daniel DSouza, Sayash Kapoor, Ahmet Üstün, Sanmi Koyejo, Yuntian Deng, Shayne Longpre, Noah A. Smith, Beyza Ermis, Marzieh Fadaee, Sara Hooker
https://arxiv.org/abs/2504.20879
Humanities last exam
https://arxiv.org/abs/2501.14249
PRISM paper
https://arxiv.org/abs/2405.10254
https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback
Collective intelligence project
https://www.cip.org/whitepaper
https://ischool.utoronto.ca/faculty-profile/brian-cantwell-smith/
Ghost work (Mary Gray)
https://www.amazon.com/Ghost-Work-Silicon-Building-Underclass/dp/1328566242
Fairwork Cloudwork report
https://fair.work/en/ratings/cloudwork/
Gibson theory of affordances
https://cs.brown.edu/courses/cs137/2017/readings/Gibson-AFF.pdf")
