Uploaded March 2026 | Updated November 2026, 15 hours ago
In the context of AI safety and alignment, Adam Ford argues that values are upstream from goals because they act as the foundational criteria that shape every decision, interpretation, and trajectory an intelligent agent follows.
Ford's core thesis differentiates between goals (specific end-states) and values (ongoing guiding principles) to highlight why simply controlling an AI's behaviour is insufficient for long-term safety.
Key Concepts:
- Fundamental Influence: Values sit upstream of everything an advanced system does, including how it pursues goals, interprets instructions, and weighs trade-offs.
- The Competence Trap: If underlying values are flawed, higher intelligence simply leads to faster and more competent "optimisation of the wrong things".
Values vs. Goals:
- Values are seen as "compass directions" or permanent evaluative criteria (e.g., valuing "health").
- Goals are specific, terminable destinations (e.g., "running a marathon") that often serve as concrete ways to realise those values.
Control vs. Motivation: While control techniques (like sandboxing) can limit damage, they struggle to scale with superintelligence. Motivation techniques - aligning the AI's internal values - are necessary to ensure the system reliably pursues outcomes we would endorse.
Value-Content Integrity: Ford emphasises the importance of preserving an agent's terminal values over time, as any drift in these "upstream" principles could lead to catastrophic misdirection "in the large," even if the system remains obedient "in the small".
Ford concludes that value alignment must be a primary pillar of AI safety research rather than a philosophical afterthought, as it ultimately determines where time takes us once control and governance have bought us that time.
Many thanks for tuning in!
Please support SciFuture by subscribing and sharing!
Buy me a coffee? buymeacoffee.com/tech101z
Have any ideas about people to interview? Want to be notified about future events? Any comments about the STF series?
Please fill out this form: docs.google.com/forms/d/1mr9PIfq2ZYlQsXRIn5BcLH2onbiSI7g79mOH_AFCdIk
Kind regards,
Adam Ford
- Science, Technology & the Future - #SciFuture - scifuture.org
In the context of AI safety and alignment, Adam Ford argues that values are upstream from goals because they act as the foundational criteria that shape every decision, interpretation, and trajectory an intelligent agent follows.
Ford's core thesis differentiates between goals (specific end-states) and values (ongoing guiding principles) to highlight why simply controlling an AI's behaviour is insufficient for long-term safety.
Key Concepts:
- Fundamental Influence: Values sit upstream of everything an advanced system does, including how it pursues goals, interprets instructions, and weighs trade-offs.
- The Competence Trap: If underlying values are flawed, higher intelligence simply leads to faster and more competent "optimisation of the wrong things".
Values vs. Goals:
- Values are seen as "compass directions" or permanent evaluative criteria (e.g., valuing "health").
- Goals are specific, terminable destinations (e.g., "running a marathon") that often serve as concrete ways to realise those values.
Control vs. Motivation: While control techniques (like sandboxing) can limit damage, they struggle to scale with superintelligence. Motivation techniques - aligning the AI's internal values - are necessary to ensure the system reliably pursues outcomes we would endorse.
Value-Content Integrity: Ford emphasises the importance of preserving an agent's terminal values over time, as any drift in these "upstream" principles could lead to catastrophic misdirection "in the large," even if the system remains obedient "in the small".
Ford concludes that value alignment must be a primary pillar of AI safety research rather than a philosophical afterthought, as it ultimately determines where time takes us once control and governance have bought us that time.
Many thanks for tuning in!
Please support SciFuture by subscribing and sharing!
Buy me a coffee? buymeacoffee.com/tech101z
Have any ideas about people to interview? Want to be notified about future events? Any comments about the STF series?
Please fill out this form: docs.google.com/forms/d/1mr9PIfq2ZYlQsXRIn5BcLH2onbiSI7g79mOH_AFCdIk
Kind regards,
Adam Ford
- Science, Technology & the Future - #SciFuture - scifuture.org








![Wendell Wallach & Adam Ford - Utopia, Superintelligence & Cosmic Morality #AI #AIEthics #Ethics
Wendell Wallach and Adam Ford discuss their visions for the future, focusing on superintelligence, human evolution, and the possibility of a cosmic morality.
Key Discussion Points:
- Morally Motivated Superintelligence: The conversation begins with a reflection on what a utopia might look like. One perspective offered is a beneficial intelligence explosion where superintelligence is developed with moral motivations that accommodate human concerns rather than being alien or divorced from humanity [00:13].
- Evolutionary Enhancement: Wendell Wallach views superintelligence not as a separate computational system that replaces humans, but as an evolutionary enhancement scheme [00:44]. He advocates for upgrading human biological and cognitive capabilities through self-understanding rather than focusing on technological immortality for a select few [01:04].
- Fair Distribution of Capabilities: A central hope discussed is that if civilization gains the ability to do grand things, the resulting benefits and dividends should be spread equally and fairly across all of society [01:36].
- Cosmic Morality and the Long-Term Future: The speakers explore cosmic game theory, suggesting that if a moral arc is discoverable by any advanced civilization, humanity might have more in common with alien life than currently assumed [02:05]. This leads to a vision of a cosmic collective in the distant future where intelligent life thrives and connects across the universe [02:24].
Wendell Wallach (born 21 April 1946) is a world-renowned bioethicist and expert on the ethics and governance of emerging technologies, particularly artificial intelligence (AI) and neuroscience. Often referred to as a godfather of AI ethics, his work focuses on the challenges of aligning rapidly advancing systems with human values.
# Current Roles & Affiliations:
Wallach remains active in several prestigious academic and policy institutions:
- Yale University: Emeritus Scholar at the Interdisciplinary Center for Bioethics, where he chaired the Technology and Ethics study group for 11 years.
- Carnegie Council for Ethics in International Affairs: Carnegie-Uehiro Senior Fellow and co-director of the AI & Equality Initiative (AIEI), which addresses structural inequalities driven by AI.
- The Hastings Center: Senior Advisor for this nonpartisan research institute focused on the social and ethical issues of science and health.
- Other Fellows: Fellow at the Sandra Day O’Connor School of Law (Arizona State University) and the Institute for Ethics & Emerging Technology.Key Publications
# He has authored several foundational works that helped define the field of machine ethics:
- Moral Machines: Teaching Robots Right from Wrong (2008): Co-authored with Colin Allen, this is widely considered the first book to examine the challenge of building artificial moral agents.
- A Dangerous Master: How to Keep Technology from Slipping Beyond Our Control (2015): Explores the risks of unchecked technological development and proposes governance solutions. A revised edition with a new preface was released in 2024.
- Library of Essays on Ethics and Emerging Technologies: Series editor for this comprehensive eight-volume collection published by Routledge.
# Global Policy & Influence
Wallach has a significant presence in international policy forums:United Nations: Has provided testimony to the UN on lethal autonomous weapons systems and advised on digital cooperation.
World Economic Forum (WEF): Formerly co-chaired the Global Future Council on Technology, Values, and Policy and served on the AI Council.
# Awards: Recipient of the World Technology Network awards for Ethics (2014) and Journalism & Media (2015).
#moralmachines #MachineEthics #AI
Many thanks for tuning in!
Please support SciFuture by subscribing and sharing!
Buy me a coffee? https://buymeacoffee.com/tech101z
Have any ideas about people to interview? Want to be notified about future events? Any comments about the STF series?
Please fill out this form: https://docs.google.com/forms/d/1mr9PIfq2ZYlQsXRIn5BcLH2onbiSI7g79mOH_AFCdIk/
Kind regards,
Adam Ford
- Science, Technology & the Future - #SciFuture - http://scifuture.org Wendell Wallach & Adam Ford - Utopia, Superintelligence & Cosmic Morality #AI #AIEthics #Ethics](https://i.ytimg.com/vi/yrjH6lrbGU8/mqdefault.jpg)

