How to Stop the Coming AI Disaster Future of Life Institute 2026-09-21 | How to Stop the Coming AI Disaster
Annie Jacobsen on Nuclear War - a Second by Second Timeline Future of Life Institute 2024-04-05 | Annie Jacobsen joins the podcast to lay out a second by second timeline for how nuclear war could happen. We also discuss time pressure, submarines, interceptor missiles, cyberattacks, and concentration of power. You can find more on Annie's work at anniejacobsen.com Timestamps:00:00 A scenario of nuclear war06:56 Who would launch an attack? 13:50 Detecting nuclear attacks 19:37 The first critical seconds 29:42 Decisions under time pressure34:27 Lessons from insiders 44:18 Submarines 51:06 How did we end up like this? 59:40 Interceptor missiles 1:11:25 Nuclear weapons and cyberattacks1:17:35 Concentration of power
Katja Grace on the Largest Survey of AI Researchers Future of Life Institute 2024-03-14 | Katja Grace joins the podcast to discuss the largest survey of AI researchers conducted to date, AI researchers' beliefs about different AI risks, capabilities required for continued AI-related transformation, the idea of discontinuous progress, the impacts of AI from either side of the human-level intelligence threshold, intelligence and power, and her thoughts on how we can mitigate AI risk. Find more on Katja's work at aiimpacts.org/.Timestamps:0:20 AI Impacts surveys18:11 What AI will look like in 20 years22:43 Experts’ extinction risk predictions29:35 Opinions on slowing down AI development31:25 AI “arms races”34:00 AI risk areas with the most agreement40:41 Do “high hopes and dire concerns” go hand-in-hand?42:00 Intelligence explosions45:37 Discontinuous progress49:43 Impacts of AI crossing the human-level intelligence threshold59:39 What does AI learn from human culture?1:02:59 AI scaling1:05:04 What should we do?
Holly Elmore on Pausing AI, Hardware Overhang, Safety Research, and Protesting Future of Life Institute 2024-02-29 | Holly Elmore joins the podcast to discuss pausing frontier AI, hardware overhang, safety research during a pause, the social dynamics of AI risk, and what prevents AGI corporations from collaborating. You can read more about Holly's work at pauseai.info Timestamps: 00:00 Pausing AI10:23 Risks during an AI pause19:41 Hardware overhang29:04 Technological progress 37:00 Safety research during a pause54:42 Social dynamics of AI risk1:10:00 What prevents cooperation? 1:18:21 What about China? 1:28:24 Protesting AGI corporations
Sneha Revanur on the Social Effects of AI Future of Life Institute 2024-02-16 | Sneha Revanur joins the podcast to discuss the social effects of AI, the illusory divide between AI ethics and AI safety, the importance of humans in the loop, the different effects of AI on younger and older people, and the importance of AIs identifying as AIs. You can read more about Sneha's work at encodejustice.org Timestamps: 00:00 Encode Justice06:11 AI ethics and AI safety 15:49 Humans in the loop 23:59 AI in social media30:42 Deteriorating social skills? 36:00 AIs identifying as AIs 43:36 AI influence in elections 50:32 AIs interacting with human systems
Roman Yampolskiy on Shoggoth, Scaling Laws, and Evidence for AI being Uncontrollable Future of Life Institute 2024-02-02 | Roman Yampolskiy joins the podcast again to discuss whether AI is like a Shoggoth, whether scaling laws will hold for more agent-like AIs, evidence that AI is uncontrollable, and whether designing human-like AI would be safer than the current development path. You can read more about Roman's work at http://cecs.louisville.edu/ry/ Timestamps: 00:00 Is AI like a Shoggoth?09:50 Scaling laws 16:41 Are humans more general than AIs? 21:54 Are AI models explainable? 27:49 Using AI to explain AI32:36 Evidence for AI being uncontrollable40:29 AI verifiability 46:08 Will AI be aligned by default? 54:29 Creating human-like AI 1:03:41 Robotics and safety 1:09:01 Obstacles to AI in the economy 1:18:00 AI innovation with current models1:23:55 AI accidents in the past and future
Special: Flo Crivello on AI as a New Form of Life Future of Life Institute 2024-01-19 | On this special episode of the podcast, Flo Crivello talks with Nathan Labenz about AI as a new form of life, whether attempts to regulate AI risks regulatory capture, how a GPU kill switch could work, and why Flo expects AGI in 2-8 years. Timestamps: 00:00 Technological progress 07:59 Regulatory capture and AI11:53 AI as a new form of life15:44 Can AI development be paused? 20:12 Biden's executive order on AI22:54 How would a GPU kill switch work?27:00 Regulating models or applications? 32:13 AGI in 2-8 years42:00 China and US collaboration on AI
Carl Robichaud on Preventing Nuclear War Future of Life Institute 2024-01-06 | Carl Robichaud joins the podcast to discuss the new nuclear arms race, how much world leaders and ideologies matter for nuclear risk, and how to reach a stable, low-risk era. You can learn more about Carl's work here: longview.org/about/carl-robichaudTimestamps:00:00 A new nuclear arms race08:07 How much do world leaders matter? 18:04 How much does ideology matter?22:14 Do nuclear weapons cause stable peace? 31:29 North Korea 34:01 Have we overestimated nuclear risk? 43:24 Time pressure in nuclear decisions 52:00 Why so many nuclear warheads?1:02:17 Has containment been successful? 1:11:34 Coordination mechanisms 1:16:31 Technological innovations1:25:57 Public perception of nuclear risk 1:29:52 Easier access to nuclear weapons1:33:31 Reaching a stable, low-risk era
Frank Sauer on Autonomous Weapon Systems Future of Life Institute 2023-12-14 | Frank Sauer joins the podcast to discuss autonomy in weapon systems, killer drones, low-tech defenses against drones, the flaws and unpredictability of autonomous weapon systems, and the political possibilities of regulating such systems. You can learn more about Frank's work here: https://metis.unibw.de/en/Timestamps:00:00 Autonomy in weapon systems12:19 Balance of offense and defense 20:05 Killer drone systems28:53 Is autonomy like nuclear weapons? 37:20 Low-tech defenses against drones 48:29 Autonomy and power balance 1:00:24 Tricking autonomous systems 1:07:53 Unpredictability of autonomous systems 1:13:16 Will we trust autonomous systems too much? 1:27:28 Legal terminology 1:32:12 Political possibilities
Darren McKee on Uncontrollable Superintelligence Future of Life Institute 2023-12-01 | Darren McKee joins the podcast to discuss how AI might be difficult to control, which goals and traits AI systems will develop, and whether there's a unified solution to AI alignment. Timestamps:00:00 Uncontrollable superintelligence16:41 AI goals and the "virus analogy" 28:36 Speed of AI cognition39:25 Narrow AI and autonomy 52:23 Reliability of current and future AI 1:02:33 Planning for multiple AI scenarios 1:18:57 Will AIs seek self-preservation? 1:27:57 Is there a unified solution to AI alignment? 1:30:26 Concrete AI safety proposals
Mark Brakel on the UK AI Summit and the Future of AI Policy Future of Life Institute 2023-11-17 | Mark Brakel (Director of Policy at the Future of Life Institute) joins the podcast to discuss the AI Safety Summit in Bletchley Park, objections to AI policy, AI regulation in the EU and US, global institutions for safe AI, and autonomy in weapon systems. Timestamps:00:00 AI Safety Summit in the UK 12:18 Are officials up to date on AI? 23:22 Objections to AI policy 31:27 The EU AI Act 43:37 The right level of regulation 57:11 Risks and regulatory tools 1:04:44 Open-source AI 1:14:56 Subsidising AI safety research 1:26:29 Global institutions for safe AI 1:34:34 Autonomy in weapon systems
How two films saved the world from nuclear war Future of Life Institute 2023-11-13 | In an era shadowed by the Cold War's persistent dread, two films resonated powerfully with the public and world leaders. The 2023 Future of Life Award honors the visionaries behind these narratives: Walter F. Parkes and Larry Lasker are behind the riveting techno-thriller "WarGames," while the groundbreaking television event "The Day After" comes from the visionary Brandon Stoddard, with Edward Hume as the screenwriter and Nicholas Meyer as the director. Released in the tension-filled year of 1983, these films not only reflected the era's nuclear anxieties but also played a direct role in shaping policy. They went beyond mere entertainment, acting as significant catalysts for dialogue. These films influenced policymakers and raised public awareness about the grave risks associated with nuclear warfare, leading to tangible, preventative action from leaders.Witness how their creative vision transcended the silver screen to safeguard the future of our world, as we celebrate their achievement with a video tribute to the 2023 Future of Life Award recipients.–––Every year, the Future of Life Award honors unsung individuals whose achievements helped make today dramatically better than it might otherwise have been.Read more about this year’s award:futureoflife.org/FLAHear the Future of Life Award 2023 featuring on this StarTalk podcast episode:www.youtube.com/watch?v=Gn-2kwQ4h1kWhat would happen if AI were integrated into today’s nuclear command and control systems? youtube.com/watch?v=w9npWiTOHX0How bad would it be if there were a nuclear war between the USA and Russia today?youtube.com/watch?v=-xthzy1PxTAOur recent podcast ‘Imagine A World’ envisions a positive future with AI:youtube.com/playlist?list=PLpxRpA6hBNrxFgCcbC4h-nE3CZ4gvRppp
Dan Hendrycks on Catastrophic AI Risks Future of Life Institute 2023-11-03 | Dan Hendrycks joins the podcast again to discuss X.ai, how AI risk thinking has evolved, malicious use of AI, AI race dynamics between companies and between militaries, making AI organizations safer, and how representation engineering could help us understand AI traits like deception. You can learn more about Dan's work at safe.ai Timestamps:00:00 X.ai - Elon Musk's new AI venture02:41 How AI risk thinking has evolved12:58 AI bioengeneering 19:16 AI agents 24:55 Preventing autocracy 34:11 AI race - corporations and militaries 48:04 Bulletproofing AI organizations 1:07:51 Open-source models1:15:35 Dan's textbook on AI safety 1:22:58 Rogue AI 1:28:09 LLMs and value specification1:33:14 AI goal drift 1:41:10 Power-seeking AI 1:52:07 AI deception 1:57:53 Representation engineering
Before It Controls Us. Future of Life Institute 2023-10-27 | The ongoing, unchecked, out-of-control race to develop increasingly powerful AI systems puts humanity at risk. These threats are potentially catastrophic, including rampant unemployment, bioterrorism, widespread disinformation, nuclear war, and many more. We urgently need lawmakers to step in and ensure a safety-first approach with proper oversight, standards and enforcement. These are not only critical to protecting human lives and wellbeing, they are essential to safeguarding innovation and ensuring that everyone can access the incredible potential benefits of AI going forward. We must not let a handful of tech corporations jeopardise humanity’s shared future.We must take control of AI before it controls us. Regulate AI now.http://futureoflife.org/safety
Samuel Hammond on AGI and Institutional Disruption Future of Life Institute 2023-10-20 | Samuel Hammond joins the podcast to discuss how AGI will transform economies, governments, institutions, and other power structures. You can read Samuel's blog at https://www.secondbest.ca. Timestamps: 00:00 Is AGI close? 06:56 Compute versus data09:59 Information theory 20:36 Universality of learning 24:53 Hards steps in evolution 30:30 Governments and advanced AI40:33 How will AI transform the economy? 55:26 How will AI change transaction costs? 1:00:31 Isolated thinking about AI 1:09:43 AI and Leviathan 1:13:01 Informational resolution1:18:36 Open-source AI 1:21:24 AI will decrease state power 1:33:17 Timeline of a techno-feudalist future 1:40:28 Alignment difficulty and AI scale1:45:19 Solving robotics1:54:40 A constrained Leviathan 1:57:41 An Apollo Project for AI safety 2:04:29 Secure "gain-of-function" AI research 2:06:43 Is the market expecting AGI soon?
Imagine A World: What if AI advisors helped us make better decisions? Future of Life Institute 2023-10-17 | Are we doomed to a future of loneliness and unfulfilling online interactions? What if technology made us feel more connected instead?Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last yearIn the eighth and final episode of Imagine A World we explore the fictional worldbuild titled 'Computing Counsel', one of the third place winners of FLI’s worldbuilding contest.Guillaume Riesen talks to Mark L, one of the three members of the team behind 'Computing Counsel', a third-place winner of the FLI Worldbuilding Contest. Mark is a machine learning expert with a chemical engineering degree, as well as an amateur writer. His teammates are Patrick B, a mechanical engineer and graphic designer, and Natalia C, a biological anthropologist and amateur programmer.This world paints a vivid, nuanced picture of how emerging technologies shape society. We have advertisers competing with ad-filtering technologies and an escalating arms race that eventually puts an end to the internet as we know it. There is AI-generated art so personalized that it becomes addictive to some consumers, while others boycott media technologies altogether. And corporations begin to throw each other under the bus in an effort to redistribute the wealth of their competitors to their own customers. While these conflicts are messy, they generally end up empowering and enriching the lives of the people in this world. New kinds of AI systems give them better data, better advice, and eventually the opportunity for genuine relationships with the beings these tools have become. The impact of any technology on society is complex and multifaceted. This world does a great job of capturing that. While social networking technologies become ever more powerful, the networks of people they connect don't necessarily just get wider and shallower. Instead, they tend to be smaller and more intimately interconnected. The world's inhabitants also have nuanced attitudes towards A.I. tools, embracing or avoiding their applications based on their religious or philosophical beliefs. These additives change over time, with public sentiment shifting from an initial dismissal of Artifical General Intelligence as persons toward something more inclusive and respectful. While most of the world's inhabitants would probably consider things to be improved in 2045, there's still a clear sense of ongoing change, growth and moral reckoning. This isn't the end of our story, but it seems like a good start.Please note: This episode explores the ideas created as part of FLI’s worldbuilding contest, and our hope is that this series sparks discussion about the kinds of futures we want. The ideas present in these imagined worlds and in our podcast are not to be taken as FLI endorsed positions.Explore this worldbuild: worldbuild.ai/computing-counselThe podcast is produced by the Future of Life Institute (FLI), a non-profit dedicated to guiding transformative technologies for humanity's benefit and reducing existential risks. To achieve this we engage in policy advocacy, grantmaking and educational outreach across three major areas: artificial intelligence, nuclear weapons, and biotechnology. If you are a storyteller, FLI can support you with scientific insights and help you understand the incredible narrative potential of these world-changing technologies. If you would like to learn more, or are interested in collaborating with the teams featured in our episodes, please email worldbuild@futureoflife.org.You can find more about our work at www.futureoflife.org, or subscribe to our newsletter to get updates on all our projects.Media and Concepts referenced in the episode:Corpus Callosum - en.wikipedia.org/wiki/Corpus_callosumEliezer Yudkowsky on ‘Superstimulation’ - lesswrong.com/posts/Jq73GozjsuhdwMLEG/superstimuli-and-the-collapse-of-western-civilizationUniversal culture - slatestarcodex.com/2016/07/25/how-the-west-was-wonMax Harms’ Crystal Trilogy - goodreads.com/book/show/28678856-crystal-societyUBI - en.wikipedia.org/wiki/Universal_basic_incomeKim Stanley Robinson - en.wikipedia.org/wiki/Kim_Stanley_Robinson
Imagine A World: What if narrow AI fractured our shared reality? Future of Life Institute 2023-10-10 | Let’s imagine a future where AGI is developed but kept at a distance from practically impacting the world, while narrow AI remakes the world completely. Most people don’t know or care about the difference and have no idea how they could distinguish between a human or artificial stranger. Inequality sticks around and AI fractures society into separate media bubbles with irreconcilable perspectives. But it's not all bad. AI markedly improves the general quality of life, enhancing medicine and therapy, and those bubbles help to sustain their inhabitants. Can you get excited about a world with these tradeoffs? Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last yearIn the seventh episode of Imagine A World we explore a fictional worldbuild titled 'Hall of Mirrors', which was a third-place winner of FLI's worldbuilding contest.Michael Vasser joins Guillaume Riesen to discuss his imagined future, which he created with the help of Matija Franklin and Bryce Hidysmith. Vassar was formerly the president of the Singularity Institute, and co-founded Metamed; more recently he has worked on communication across political divisions. Franklin is a PhD student at UCL working on AI Ethics and Alignment. Finally, Hidysmith began in fashion design, passed through fortune-telling before winding up in finance and policy research, at places like Numerai, the Median Group, Bismarck Analysis, and Eco.com.Hall of Mirrors is a deeply unstable world where nothing is as it seems. The structures of power that we know today have eroded away, survived only by shells of expectation and appearance. People are isolated by perceptual bubbles and struggle to agree on what is real. Things are generally going okay, for now. This is partly due to this world's particularly slow and modest development of AI technologies. AI tools here are still dominated by extensions of today's fundamentally narrow systems, with the one true Artificial General Intelligence being developed under heavy quarantine. There are a number of reasons for this slow progress, including high computational costs and poor funding due to politicization. This team put a lot of effort into creating a plausible, empirically grounded world, but their work is also notable for its irreverence and dark humor. In some ways, this world is kind of a caricature of the present. We see deeper isolation and polarization caused by media, and a proliferation of powerful but ultimately limited AI tools that further erode our sense of objective reality. A deep instability threatens. And yet, on a human level, things seem relatively calm. It turns out that the stories we tell ourselves about the world have a lot of inertia, and so do the ways we live our lives.Please note: This episode explores the ideas created as part of FLI’s worldbuilding contest, and our hope is that this series sparks discussion about the kinds of futures we want. The ideas present in these imagined worlds and in our podcast are not to be taken as FLI endorsed positions.Explore this worldbuild: worldbuild.ai/hall-of-mirrorsThe podcast is produced by the Future of Life Institute (FLI), a non-profit dedicated to guiding transformative technologies for humanity's benefit and reducing existential risks. To achieve this we engage in policy advocacy, grantmaking and educational outreach across three major areas: artificial intelligence, nuclear weapons, and biotechnology. If you are a storyteller, FLI can support you with scientific insights and help you understand the incredible narrative potential of these world-changing technologies. If you would like to learn more, or are interested in collaborating with the teams featured in our episodes, please email worldbuild@futureoflife.org.You can find more about our work at www.futureoflife.org, or subscribe to our newsletter to get updates on all our projects.Media referenced in the episode:en.wikipedia.org/wiki/Neo-Confucianismen.wikipedia.org/wiki/Who_Framed_Roger_Rabbiten.wikipedia.org/wiki/Seigniorageen.wikipedia.org/wiki/Adam_Smithen.wikipedia.org/wiki/Hamleten.wikipedia.org/wiki/The_Golden_Notebooken.wikipedia.org/wiki/Star_Trek%3A_The_Next_Generationen.wikipedia.org/wiki/C-3POen.wikipedia.org/wiki/James_Baldwin
Steve Omohundro on Provably Safe AGI Future of Life Institute 2023-10-05 | Steve Omohundro joins the podcast to discuss Provably Safe Systems, a paper he co-authored with FLI President Max Tegmark. You can read the paper here: arxiv.org/pdf/2309.01933.pdf Timestamps:00:00 Provably safe AI systems 12:17 Alignment and evaluations21:08 Proofs about language model behavior27:11 Can we formalize safety? 30:29 Provable contracts 43:13 Digital replicas of actual systems 46:32 Proof-carrying code 56:25 Can language models think logically? 1:00:44 Can AI do proofs for us? 1:09:23 Hard to proof, easy to verify 1:14:31 Digital neuroscience 1:20:01 Risks of totalitarianism 1:22:29 Can we guarantee safety? 1:25:04 Real-world provable safety 1:29:29 Tamper-proof hardware 1:35:35 Mortal and throttled AI 1:39:23 Least-privilege guarantee 1:41:53 Basic AI drives 1:47:47 AI agency and world models 1:52:08 Self-improving AI 1:58:21 Is AI overhyped now?
Imagine A World: What if AI enabled us to communicate with animals? Future of Life Institute 2023-10-04 | What if AI allowed us to communicate with animals? Could interspecies communication lead to new levels of empathy? How might communicating with animals lead humans to reimagine our place in the natural world?Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last year.In the sixth episode of Imagine A World we explore the fictional worldbuild titled 'AI for the People', a third place winner of the worldbuilding contest.Our host Guillaume Reisen welcomes Chi Rainer Bornfree, part of this three-person worldbuilding team alongside her husband Micah White, and their collaborator, J.R. Harris. Chi has a PhD in Rhetoric from UC Berkeley and has taught at Bard, Princeton, and NY State Correctional facilities, in the meantime writing fiction, essays, letters, and more. Micah, best-known as the co-creator of the 'Occupy Wall Street' movement and the author of 'The End of Protest', now focuses primarily on the social potential of cryptocurrencies, while Harris is a freelance illustrator and comic artist.The name 'AI for the People' does a great job of capturing this team's activist perspective and their commitment to empowerment. They imagine social and political shifts that bring power back into the hands of individuals, whether that means serving as lawmakers on randomly selected committees, or gaining income by choosing to sell their personal data online. But this world isn't just about human people. Its biggest bombshell is an AI breakthrough that allows humans to communicate with other animals. What follows is an existential reconsideration of humanity's place in the universe. This team has created an intimate, complex portrait of a world shared by multiple parties: AIs, humans, other animals, and the environment itself. As these entities find their way forward together, their goals become enmeshed and their boundaries increasingly blurred.Please note: This episode explores the ideas created as part of FLI’s worldbuilding contest, and our hope is that this series sparks discussion about the kinds of futures we want. The ideas present in these imagined worlds and in our podcast are not to be taken as FLI endorsed positions.Explore this worldbuild: worldbuild.ai/ai-for-the-peopleThe podcast is produced by the Future of Life Institute (FLI), a non-profit dedicated to guiding transformative technologies for humanity's benefit and reducing existential risks. To achieve this we engage in policy advocacy, grantmaking and educational outreach across three major areas: artificial intelligence, nuclear weapons, and biotechnology. If you are a storyteller, FLI can support you with scientific insights and help you understand the incredible narrative potential of these world-changing technologies. If you would like to learn more, or are interested in collaborating with the teams featured in our episodes, please email worldbuild@futureoflife.org.You can find more about our work at www.futureoflife.org, or subscribe to our newsletter to get updates on all our projectsMedia and resources referenced in the episode:en.wikipedia.org/wiki/Life_3.0en.wikipedia.org/wiki/1_the_Roadignota.org/products/pharmako-aien.wikipedia.org/wiki/The_Ministry_for_the_Futurescientificamerican.com/article/how-scientists-are-using-ai-to-talk-to-animalsen.wikipedia.org/wiki/Occupy_Wall_Streeten.wikipedia.org/wiki/Sortitionen.wikipedia.org/wiki/Iroquoisen.wikipedia.org/wiki/The_Ship_Who_Sangen.wikipedia.org/wiki/The_Sparrow_(novel)en.wikipedia.org/wiki/After_Yang
Regulate AI Now Future of Life Institute 2023-09-28 | In the six months since FLI published its open letter calling for a pause on giant AI experiments, we have seen overwhelming expert and public concern about the out-of-control AI arms race - but no slowdown. In this video, we call for U.S. lawmakers to step in, and explore the policy solutions necessary to steer this powerful technology to benefit humanity.
Imagine A World: What if AI-enabled life extension allowed some people to live forever? Future of Life Institute 2023-09-26 | If you could extend your life, would you? How might life extension technologies create new social and political divides? How can the world unite to solve the great problems of our time, like AI risk? What if AI creators could agree on an inspection process to expose AI dangers before they're unleashed? Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last yearIn the fifth episode of Imagine A World, we explore the fictional worldbuild titled 'To Light’. Our host Guillaume Reisen speaks to Mako Yass, the first place winner of the FLI Worldbuilding Contest we ran last year. Mako lives in Auckland, New Zealand. He describes himself as a 'stray philosopher-designer', and has a background in computer programming and analytic philosophy. Mako’s world is particularly imaginative, with richly interwoven narrative threads and high-concept sci fi inventions. By 2045, his world has been deeply transformed. There’s an AI-designed miracle pill that greatly extends lifespan and eradicates most human diseases. Sachets of this life-saving medicine are distributed freely by dove-shaped drones. There’s a kind of mind uploading which lets anyone become whatever they wish, live indefinitely and gain augmented intelligence. The distribution of wealth is almost perfectly even, with every human assigned a share of all resources. Some people move into space, building massive structures around the sun where they practice esoteric arts in pursuit of a more perfect peace. While this peaceful, flourishing end state is deeply optimistic, Mako is also very conscious of the challenges facing humanity along the way. He sees a strong need for global collaboration and investment to avoid catastrophe as humanity develops more and more powerful technologies. He’s particularly concerned with the risks presented by artificial intelligence systems as they surpass us. An AI system that is more capable than a human at all tasks - not just playing chess or driving a car - is what we’d call an Artificial General Intelligence - abbreviated ‘AGI’. Ensuring that such a system actually behaves as we intend, and faithfully pursues goals that are beneficial rather than harmful to humans, is an enormous challenge that we do not yet know how to solve. Mako proposes that we could build safe AIs through radical transparency. He imagines tests that could reveal the true intentions and expectations of AI systems before they are released into the world. Please note: This episode explores the ideas created as part of FLI’s worldbuilding contest, and our hope is that this series sparks discussion about the kinds of futures we want. The ideas present in these imagined worlds and in our podcast are not to be taken as FLI endorsed positions.Explore this worldbuild: worldbuild.ai/to-lightThe podcast is produced by the Future of Life Institute (FLI), a non-profit dedicated to guiding transformative technologies for humanity's benefit and reducing existential risks. To achieve this we engage in policy advocacy, grantmaking and educational outreach across three major areas: artificial intelligence, nuclear weapons, and biotechnology. If you are a storyteller, FLI can support you with scientific insights and help you understand the incredible narrative potential of these world-changing technologies. If you would like to learn more, or are interested in collaborating with the teams featured in our episodes, please email worldbuild@futureoflife.org.You can find more about our work at www.futureoflife.org, or subscribe to our newsletter to get updates on all our projects.Media and concepts referenced in the episode:en.wikipedia.org/wiki/Terra_Ignotaen.wikipedia.org/wiki/The_Transparent_Societyen.wikipedia.org/wiki/Instrumental_convergence#Paperclip_maximizeren.wikipedia.org/wiki/The_Elephant_in_the_Brainen.wikipedia.org/wiki/The_Matrixaboutmako.makopool.com
Johannes Ackva on Managing Climate Change Future of Life Institute 2023-09-21 | Johannes Ackva joins the podcast to discuss the main drivers of climate change and our best technological and governmental options for managing it. You can read more about Johannes' work at http://founderspledge.com/climateTimestamps:00:00 Johannes's journey as an environmentalist 13:21 The drivers of climate change23:00 Oil, coal, and gas 38:05 Solar, wind, and hydro49:34 Nuclear energy 57:03 Geothermal energy 1:00:41 Most promising technologies1:05:40 Government subsidies1:13:28 Carbon taxation 1:17:10 Planting trees1:21:53 Influencing government policy 1:26:39 Different climate scenarios1:34:49 Economic growth and emissions1:37:23 Social stabilityReferences: Emissions by sector: ourworldindata.org/emissions-by-sector Energy density of different energy sources: nature.com/articles/s41598-022-25341-9 Emissions forecasts: lse.ac.uk/granthaminstitute/publication/the-unconditional-probability-distribution-of-future-emissions-and-temperatures and science.org/doi/10.1126/science.adg6248 Risk management: youtube.com/watch?v=6JJvIR1W-xICarbon pricing: cell.com/joule/pdf/S2542-4351(18)30567-1.pdf Why not simply plant trees?: https://climate.mit.edu/ask-mit/how-many-new-trees-would-we-need-offset-our-carbon-emissions Deforestation: science.org/doi/10.1126/science.ade3535 Decoupling of economic growth and emissions: globalcarbonproject.org/carbonbudget/22/highlights.htm Premature deaths from air pollution: unep.org/interactives/air-pollution-note
Imagine A World: What if we developed digital nations untethered to geography? Future of Life Institute 2023-09-19 | How do low income countries affected by climate change imagine their futures? How do they overcome these twin challenges? Will all nations eventually choose or be forced to go digital?Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last year.In the fourth episode of Imagine A World, we explore the fictional worldbuild titled 'Digital Nations'.Conrad Whitaker and Tracey Kamande join Guillaume Riesen on 'Imagine a World' to talk about their worldbuild, 'Digital Nations', which they created with their teammate, Dexter Findley. All three worldbuilders were based in Kenya while crafting their entry, though Dexter has just recently moved to the UK. Conrad is a Nairobi-based startup advisor and entrepreneur, Dexter works in humanitarian aid, and Tracey is the Co-founder of FunKe Science, a platform that promotes interactive learning of science among school children.As the name suggests, this world is a deep dive into virtual communities. It explores how people might find belonging and representation on the global stage through digital nations that aren't tied to any physical location. This world also features a fascinating and imaginative kind of artificial intelligence that they call 'digital persons'. These are inspired by biological brains and have a rich internal psychology. Rather than being trained on data, they're considered to be raised in digital nurseries. They have a nuanced but mostly loving relationship with humanity, with some even going on to found their own digital nations for us to join. In an incredible turn of events, last year the South Pacific state of Tuvalu was the first to “go virtual” in response to sea levels threatening the island nation's physical territory. This happened in real life just months after it was written into this imagined world in our worldbuilding contest, showing how rapidly ideas that seem ‘out there’ can become reality. Will all nations eventually go digital? And might AGIs be assimilated, 'brought up' rather than merely trained, as 'digital people', citizens to live communally alongside humans in these futuristic states?The promise of this world is that new technologies might provide new opportunities to bring people together. The challenges of safely navigating new kinds of intelligence may offer a chance to know ourselves and each other more deeply, leading to wiser and gentler societies.Please note: This episode explores the ideas created as part of FLI’s worldbuilding contest, and our hope is that this series sparks discussion about the kinds of futures we want. The ideas present in these imagined worlds and in our podcast are not to be taken as FLI endorsed positions.Explore this worldbuild: worldbuild.ai/digital-nations The podcast is produced by the Future of Life Institute (FLI), a non-profit dedicated to guiding transformative technologies for humanity's benefit and reducing existential risks. To achieve this we engage in policy advocacy, grantmaking and educational outreach across three major areas: artificial intelligence, nuclear weapons, and biotechnology. If you are a storyteller, FLI can support you with scientific insights and help you understand the incredible narrative potential of these world-changing technologies. If you would like to learn more, or are interested in collaborating with the teams featured in our episodes, please email worldbuild@futureoflife.org.You can find more about our work at www.futureoflife.org, or subscribe to our newsletter to get updates on all our projectsMedia and concepts referenced in the episode:tuvalu.tven.wikipedia.org/wiki/Trolley_problemen.wikipedia.org/wiki/Climate_change_in_Kenyaen.wikipedia.org/wiki/John_von_Neumannen.wikipedia.org/wiki/Brave_New_Worldthenetworkstate.com/the-network-stateen.wikipedia.org/wiki/Culture_series
Imagine A World: What if our response to global challenges led to a more centralized world? Future of Life Institute 2023-09-12 | What if we had one advanced AI system for the entire world? Would this lead to a world 'beyond' nation states - and do we want this?Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last year.In the third episode of Imagine A World, we explore the fictional worldbuild titled 'Core Central'.How does a team of seven academics agree on one cohesive worldbuild? That's a question the team behind 'Core Central' had to figure out as they went along. The results speak for themselves: this worldbuild came joint second-place in the FLI Worldbuilding Contest, and its realistic sense of multipolarity and messiness reflect positively its organic formulation. The team settled on one core, centralized AGI system as the governance model for their entire world. This eventually moves their world 'beyond' nation states. Could this really work?In this episode of 'Imagine a World', Guillaume Riesen speaks to John Burden and Henry Shevlin, representing the team that created 'Core Central', one of the second-place winners in the FLI Worldbuilding Contest. The full team includes seven members, three of whom (Henry, John and Beba Cibralic) are researchers at the Leverhulme Centre for the Future of Intelligence, University of Cambridge, and five of whom (Jessica Bland, Lara Mani, Clarissa Rios Rojas, Catherine Richards and John) work with the Centre for the Study of Existential Risk, also at Cambridge University.Please note: This episode explores the ideas created as part of FLI’s worldbuilding contest, and our hope is that this series sparks discussion about the kinds of futures we want. The ideas present in these imagined worlds and in our podcast are not to be taken as FLI endorsed positions.Explore this imagined world: worldbuild.ai/core-centralThe podcast is produced by the Future of Life Institute (FLI), a non-profit dedicated to guiding transformative technologies for humanity's benefit and reducing existential risks. To achieve this we engage in policy advocacy, grantmaking and educational outreach across three major areas: artificial intelligence, nuclear weapons, and biotechnology. If you are a storyteller, FLI can support you with scientific insights and help you understand the incredible narrative potential of these world-changing technologies. If you would like to learn more, or are interested in collaborating with the teams featured in our episodes, please email worldbuild@futureoflife.org.You can find more about our work at www.futureoflife.org, or subscribe to our newsletter to get updates on all our projectsMedia and Concepts referenced in the episode:en.wikipedia.org/wiki/Culture_seriesen.wikipedia.org/wiki/The_Expanse_(TV_series)vox.com/authors/kelsey-piperen.wikipedia.org/wiki/Gratitude_journalen.wikipedia.org/wiki/The_Diamond_Agescientificamerican.com/article/the-mind-of-an-octopusen.wikipedia.org/wiki/Global_workspace_theoryen.wikipedia.org/wiki/Alien_hand_syndromeen.wikipedia.org/wiki/Hyperion_(Simmons_novel)
Tom Davidson on How Quickly AI Could Automate the Economy Future of Life Institute 2023-09-08 | Tom Davidson joins the podcast to discuss how AI could quickly automate most cognitive tasks, including AI research, and why this would be risky. Timestamps:00:00 The current pace of AI 03:58 Near-term risks from AI 09:34 Historical analogies to AI 13:58 AI benchmarks VS economic impact 18:30 AI takeoff speed and bottlenecks31:09 Tom's model of AI takeoff speed36:21 How AI could automate AI research 41:49 Bottlenecks to AI automating AI hardware 46:15 How much of AI research is automated now? 48:26 From 20% to 100% automation 53:24 AI takeoff in 3 years 1:09:15 Economic impacts of fast AI takeoff1:12:51 Bottlenecks slowing AI takeoff 1:20:06 Does the market predict a fast AI takeoff? 1:25:39 "Hard to avoid AGI by 2060" 1:27:22 Risks from AI over the next 20 years 1:31:43 AI progress without more compute 1:44:01 What if AI models fail safety evaluations? 1:45:33 Cybersecurity at AI companies 1:47:33 Will AI turn out well for humanity? 1:50:15 AI and board games
Imagine A World: What if we designed and built AI in an inclusive way? Future of Life Institute 2023-09-05 | How does who is involved in the design of AI affect the possibilities for our future? Why isn’t the design of AI inclusive already? Can technology solve all our problems? Can human nature change? Do we want either of these things to happen?Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last yearIn this second episode of Imagine A World we explore the fictional worldbuild titled 'Crossing Points', a second place entry in FLI's worldbuilding contest.Joining Guillaume Riesen on the Imagine a World podcast this time are two members of the Crossing Points team, Elaine Czech and Vanessa Hanschke, both academics at the University of Bristol. Elaine has a background in art and design, and is studying the accessibility of technologies for the elderly. Vanessa is studying responsible AI practices of technologists, using methods like storytelling to promote diverse voices in AI research. Their teammates in the contest were Tashi Namgyal, a University of Bristol PhD studying the controllability of deep generative models, Dr. Susan Lechelt, who researches the applications and implications of emerging technologies at the University of Edinburgh, and Nicol Ogston, a British civil servant.This team put a lot of thought into how new technologies might work alongside human nature rather than trying to change us directly. The most advanced AI systems in this world are extremely large and expensive to run, so are mostly limited to political and corporate use. Crossing Points has a special focus on exploring how simpler tools are experienced by a range of diverse individuals. There's an emphasis on the unanticipated impacts of new technologies on those who weren't considered during their development. From urban families in Indonesia to anti-technology extremists in America, we're shown that there's something to learn from every human story. This world emphasizes the importance of broadening our lens and empowering marginalized voices in order to build a future that would be bright for more than just a privileged few. The world of Crossing Points looks pretty different from our own, with advanced AIs debating philosophy on TV and hybrid 3D printed meats and grocery stores. But the people in this world are still basically the same. There are still a dazzling array of languages and cultures which shape our attitudes toward technologies. Our hopes and dreams haven't fundamentally changed, and neither have our blindspots and shortcomings. Crossing Points embraces humanity in all its diversity and looks for the solutions that human nature presents alongside the problems. It shows that there's something to learn from everyone's experience and that even the most radical attitudes can offer insights that help to build a better world.Please note: This episode explores the ideas created as part of FLI’s worldbuilding contest, and our hope is that this series sparks discussion about the kinds of futures we want. The ideas present in these imagined worlds and in our podcast are not to be taken as FLI endorsed positions.Explore this worldbuild: worldbuild.ai/crossing-pointsThe podcast is produced by the Future of Life Institute (FLI), a non-profit dedicated to guiding transformative technologies for humanity's benefit and reducing existential risks. To achieve this we engage in policy advocacy, grantmaking and educational outreach across three major areas: artificial intelligence, nuclear weapons, and biotechnology. If you are a storyteller, FLI can support you with scientific insights and help you understand the incredible narrative potential of these world-changing technologies. If you would like to learn more, or are interested in collaborating with the teams featured in our episodes, please email worldbuild@futureoflife.org.You can find more about our work at www.futureoflife.org, or subscribe to our newsletter to get updates on all our projects.Works referenced in this episode:en.wikipedia.org/wiki/The_Legend_of_Zeldaen.wikipedia.org/wiki/Ainu_peoplegoodreads.com/book/show/34846958-radicalshttp://www.historyofmasks.net/famous-masks/noh-mask
Imagine A World: What if new governance mechanisms helped us coordinate? Future of Life Institute 2023-09-05 | Are today's democratic systems equipped well enough to create the best possible future for everyone? If they're not, what systems might work better? And are governments around the world taking the destabilizing threats of new technologies seriously enough, or will it take a dramatic event, such as an AI-driven war, to get their act together? Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last year.In this first episode of Imagine A World we explore the fictional worldbuild titled 'Peace Through Prophecy'.Host Guillaume Reisen speaks to the makers of 'Peace Through Prophecy', a second place entry in FLI's Worldbuilding Contest. The worldbuild was created by Jackson Wagner, Diana Gurvich and Holly Oatley. In the episode, Jackson and Holly discuss just a few of the many ideas bubbling around in their imagined future.At its core, this world is arguably about community. It asks how technology might bring us closer together, and allow us to reinvent our social systems. Many roads are explored, a whole garden of governance systems bolstered by Artificial Intelligence and other technologies. Overall, there's a shift towards more intimate and empowered communities. Even the AI systems eventually come to see their emotional and creative potentials realized. While progress is uneven, and littered with many human setbacks, a pretty good case is made for how everyone's best interests can lead us to a more positive future.Please note: This episode explores the ideas created as part of FLI’s worldbuilding contest, and our hope is that this series sparks discussion about the kinds of futures we want. The ideas present in these imagined worlds and in our podcast are not to be taken as FLI endorsed positionsExplore this imagined world: worldbuild.ai/peace-through-prophecyThe podcast is produced by the Future of Life Institute (FLI), a non-profit dedicated to guiding transformative technologies for humanity's benefit and reducing existential risks. To achieve this we engage in policy advocacy, grantmaking and educational outreach across three major areas: artificial intelligence, nuclear weapons, and biotechnology. If you are a storyteller, FLI can support you with scientific insights and help you understand the incredible narrative potential of these world-changing technologies. If you would like to learn more, or are interested in collaborating with the teams featured in our episodes, please email worldbuild@futureoflife.org.You can find more about our work at www.futureoflife.org, or subscribe to our newsletter to get updates on all our projects.Media and concepts referenced in the episode:en.wikipedia.org/wiki/Prediction_marketforum.effectivealtruism.org'Veil of ignorance' thought experiment: en.wikipedia.org/wiki/Original_positionen.wikipedia.org/wiki/Isaac_Asimoven.wikipedia.org/wiki/Liquid_democracyen.wikipedia.org/wiki/The_Dispossesseden.wikipedia.org/wiki/Terra_Ignotaequilibriabook.comen.wikipedia.org/wiki/John_Rawlsen.wikipedia.org/wiki/Radical_transparencyen.wikipedia.org/wiki/Audrey_Tangen.wikipedia.org/wiki/Quadratic_voting#Quadratic_funding
Imagine A World Podcast from the Future of Life Institute [TRAILER] Future of Life Institute 2023-08-29 | Coming Soon…The year is 2045. Humanity is not extinct, nor living in a dystopia. It has averted climate disaster and major wars. Instead, AI and other new technologies are helping to make the world more peaceful, happy and equal. How? This was what we asked the entrants of our Worldbuilding Contest to imagine last year.Our new podcast series digs deeper into the eight winning entries, their ideas and solutions, the diverse teams behind them and the challenges they faced. You might love some; others you might not choose to inhabit. FLI is not endorsing any one idea. Rather, we hope to grow the conversation about what futures people get excited about.Ask yourself, with each episode, is this a world you’d want to live in? And if not, what would you prefer?Don’t miss the first two episodes coming to your inboxes at the start of September!In the meantime, do explore the winning worlds, if you haven’t already: worldbuild.ai
Robert Trager on International AI Governance and Cybersecurity at AI Companies Future of Life Institute 2023-08-20 | Robert Trager joins the podcast to discuss AI governance, the incentives of governments and companies, the track record of international regulation, the security dilemma in AI, cybersecurity at AI companies, and skepticism about AI governance. We also discuss Robert's forthcoming paper International Governance of Civilian AI: A Jurisdictional Certification Approach. You can read more about Robert's work at https://www.governance.ai. Timestamps:00:00 The goals of AI governance08:38 Incentives of governments and companies18:58 Benefits of regulatory diversity 28:50 The track record of anticipatory regulation 37:55 The security dilemma in AI46:20 Offense-defense balance in AI 53:27 Failure rates and international agreements 1:00:33 Verification of compliance 1:07:50 Controlling AI supply chains 1:13:47 Cybersecurity at AI companies 1:21:30 The jurisdictional certification approach1:28:40 Objections to AI governance
Jason Crawford on Progress and Risks from AI Future of Life Institute 2023-07-21 | Jason Crawford joins the podcast to discuss the history of progress, the future of economic growth, and the relationship between progress and risks from AI. You can read more about Jason's work at rootsofprogress.org Timestamps:00:00 Eras of human progress 06:47 Flywheels of progress 17:56 Main causes of progress 21:01 Progress and risk 32:49 Safety as part of progress 45:20 Slowing down specific technologies? 52:29 Four lenses on AI risk 58:48 Analogies causing disagreement 1:00:54 Solutionism about AI 1:10:43 Insurance, subsidies, and bug bounties for AI risk 1:13:24 How is AI different from other technologies? 1:15:54 Future scenarios of economic growth
Artificial Escalation Future of Life Institute 2023-07-17 | This work of fiction seeks to depict key drivers that could result in a global Al catastrophe:- Accidental conflict escalation at machine speeds;- Al integrated too deeply into high-stakes functions;- Humans giving away too much control to Al;- Humans unable to tell what is real and what is fake, and;- An arms race that ultimately has only losers.The good news is, all of these risks can be avoided. This story does not have to be our fate.Please share this video and learn more at futureoflife.org/artificial-escalation.This video has been informed by a 2020 paper from the Stockhold International Peace Research Institute (SIPRI):Boulanin, Vincent et al. ‘Artificial Intelligence, Strategic Stability and Nuclear Risk’. sipri.org/publications/2020/other-publications/artificial-intelligence-strategic-stability-and-nuclear-riskThe sequel to this video: youtube.com/watch?v=-xthzy1PxTA
Special: Jaan Tallinn on Pausing Giant AI Experiments Future of Life Institute 2023-07-06 | On this special episode of the podcast, Jaan Tallinn talks with Nathan Labenz about Jaan's model of AI risk, the future of AI development, and pausing giant AI experiments.Timestamps: 0:00 Nathan introduces Jaan4:22 AI safety and Future of Life Institute5:55 Jaan's first meeting with Eliezer Yudkowsky12:04 Future of AI evolution14:58 Jaan's investments in AI companies23:06 The emerging danger paradigm26:53 Economic transformation with AI32:31 AI supervising itself 34:06 Language models and validation38:49 Lack of insight into evolutionary selection process41:56 Current estimate for life-ending catastrophe44:52 Inverse scaling law 53:03 Our luck given the softness of language models 55:07 Future of language models59:43 The Moore's law of mad science1:01:45 GPT-5 type project1:07:43 The AI race dynamics1:09:43 AI alignment with the latest models1:13:14 AI research investment and safety1:19:43 What a six-month pause buys us1:25:44 AI passing the Turing Test1:28:16 AI safety and risk 1:32:01 Responsible AI development.1:40:03 Neuralink implant technology
How would a nuclear war between Russia and the US affect you personally? Future of Life Institute 2023-06-29 | What would happen if a nuclear war were to be sparked between Russia and the United States today? Who would survive?In our most scientifically realistic simulation to date, we show what a nuclear war between Russia and the United States might look like today. It is based on detailed modeling of nuclear targets, missile trajectories, and the effects of blasts, EMPs, and smoke on the climate and food resources.We have just announced the results of our latest grant program focused on the humanitarian impacts of nuclear war – see the list of projects here: futureoflife.org/grant-program/nuclear-war-researchLearn more about the risks posed by nuclear weapons and find out how you can take action to reduce the risks here: nuclearweapons.infoDiscover the full story of Nuclear Winter: youtube.com/watch?v=EuzH43J0mrYThe 2022 Future of Life Award was given to 8 individuals for their contributions to the discovery of and raising awareness for Nuclear Winter. Learn about their stories: youtube.com/watch?v=gEwNWBnlhlwSee also:Union of Concerned Scientists: ucsusa.org/nuclear-weaponsThe International Campaign to Abolish Nuclear Weapons (ICAN): icanw.orgCopyright 2023 Future of Life Institute, Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0international license.- BIBLIOGRAPHY -Here are the resources used in the video. These papers and data sources present the most cutting-edge science we have on nuclear war and its effects:Nuclear arsenal data:Hans Kristensen (Federation of American Scientists); The video shows 3,641 of 11,133 US and Russian warheads used, and none from the UK, France, China, India, Pakistan, China, Israel or North Korea.Nuclear target model:Prof. Sharon Weiner (American University) & Dr. Moritz Kütt (IFSH Hamburg)Ballistic missile trajectories calculations:Prof. Anthony Aguirre (UCSC) & Prof. Tegmark (MIT); In reality, MIRVs of course separate only long after launch.Electromagnetic pulse modeling:U.S. Army unclassified report AD-A178230 (1994)Blast modeling:Prof. Alex Wellerstein, Stevens Inst. of Technology, https:/www.nukemap.orgSamuel Glasstone and Philip J. Dolan: "The Effects of Nuclear Weapons", published by U.S. Dept. of Defense & Dept. of Energy; Although the video mainly shows ground bursts to simplify visualization, most urban strikes are likely to instead be airbursts to destroy a greater area.Atmospheric modeling of global smoke spread and lofting:Dr. Charles G. Bardeen (National Center for Atmospheric Research) assuming 150 megatons of black carbon smokeSmoke modeling methodology:Joshua Coupe, Charles G. Bardeen, Alan Robock & Owen B. Toon 2019, J. of Geophysical Research: Atmospheres, 124, 8522-8543Owen B. Toon, Charles G. Bardeen, Alan Robock, Lili Zia, Hans Kristensen, Matthew McKinzie, R. Peterson, Cheryl S. Harrison, Nicole.Lovenduski & Richard P. Turco 2019, Science Advances 5: eaay5478Nuclear winter temperature modeling:.Alan Robock, Luke Oman & Georgiy L. Stenchikov 2007, J. Of Geophysical Research 112, D13107Famine modeling:Lili Xia, Alan Robock, Kim Scherrer, Cheryl Harrison, Benjamin Bodirsky, Isabelle Reindl, Jonas Jägermeyr, Charles Bardeen, Owen B. Toon & Ryan Heneghan 2022, Nature Food, 3, 586-596. nature.com/articles/s43016-022-00573-0
Joe Carlsmith on How We Change Our Minds About AI Risk Future of Life Institute 2023-06-22 | Joe Carlsmith joins the podcast to discuss how we change our minds about AI risk, gut feelings versus abstract models, and what to do if transformative AI is coming soon. You can read more about Joe's work at https://joecarlsmith.com. Timestamps: 00:00 Predictable updating on AI risk 07:27 Abstract models versus gut feelings22:06 How Joe began believing in AI risk29:06 Is AI risk falsifiable? 35:39 Types of skepticisms about AI risk 44:51 Are we fundamentally confused? 53:35 Becoming alienated from ourselves? 1:00:12 What will change people's minds? 1:12:34 Outline of different futures 1:20:43 Humanity losing touch with reality 1:27:14 Can we understand AI sentience? 1:36:31 Distinguishing real from fake sentience 1:39:54 AI doomer epistemology 1:45:23 AI benchmarks versus real-world AI 1:53:00 AI improving AI research and development2:01:08 What if transformative AI comes soon? 2:07:21 AI safety if transformative AI comes soon 2:16:52 AI systems interpreting other AI systems 2:19:38 Philosophy and transformative AI Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Dan Hendrycks on Why Evolution Favors AIs over Humans Future of Life Institute 2023-06-08 | Dan Hendrycks joins the podcast to discuss evolutionary dynamics in AI development and how we could develop AI safely. You can read more about Dan's work at safe.ai Timestamps: 00:00 Corporate AI race 06:28 Evolutionary dynamics in AI 25:26 Why evolution applies to AI 50:58 Deceptive AI 1:06:04 Competition erodes safety 10:17:40 Evolutionary fitness: humans versus AI 1:26:32 Different paradigms of AI risk 1:42:57 Interpreting AI systems 1:58:03 Honest AI and uncertain AI 2:06:52 Empirical and conceptual work 2:12:16 Losing touch with reality Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Roman Yampolskiy on Objections to AI Safety Future of Life Institute 2023-05-26 | Roman Yampolskiy joins the podcast to discuss various objections to AI safety, impossibility results for AI, and how much risk civilization should accept from emerging technologies. You can read more about Roman's work at http://cecs.louisville.edu/ry/ Timestamps: 00:00 Objections to AI safety 15:06 Will robots make AI risks salient? 27:51 Was early AI safety research useful? 37:28 Impossibility results for AI 47:25 How much risk should we accept? 1:01:21 Exponential or S-curve? 1:12:27 Will AI accidents increase? 1:23:56 Will we know who was right about AI? 1:33:33 Difference between AI output and AI modelSocial Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Nathan Labenz on How AI Will Transform the Economy Future of Life Institute 2023-05-11 | Nathan Labenz joins the podcast to discuss the economic effects of AI on growth, productivity, and employment. We also talk about whether AI might have catastrophic effects on the world. You can read more about Nathan's work at cognitiverevolution.ai Timestamps: 00:00 Economic transformation from AI 11:15 Productivity increases from technology 17:44 AI effects on employment 28:43 Life without jobs 38:42 Losing contact with reality 42:31 Catastrophic risks from AI 53:52 Scaling AI training runs 1:02:39 Stable opinions on AI? Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Nathan Labenz on the Cognitive Revolution, Red Teaming GPT-4, and Potential Dangers of AI Future of Life Institute 2023-05-04 | Nathan Labenz joins the podcast to discuss the cognitive revolution, his experience red teaming GPT-4, and the potential near-term dangers of AI. You can read more about Nathan's work at cognitiverevolution.ai Timestamps: 00:00 The cognitive revolution 07:47 Red teaming GPT-4 24:00 Coming to believe in transformative AI 30:14 Is AI depth or breadth most impressive? 42:52 Potential near-term dangers from AI Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Maryanna Saenko on Venture Capital, Philanthropy, and Ethical Technology Future of Life Institute 2023-04-27 | Maryanna Saenko joins the podcast to discuss how venture capital works, how to fund innovation, and what the fields of investing and philanthropy could learn from each other. You can read more about Maryanna's work at https://future.ventures Timestamps: 00:00 How does venture capital work? 09:01 Failure and success for startups 13:22 Is overconfidence necessary? 19:20 Repeat entrepreneurs 24:38 Long-term investing 30:36 Feedback loops from investments 35:05 Timing investments 38:35 The hardware-software dichotomy 42:19 Innovation prizes 45:43 VC lessons for philanthropy51:03 Creating new markets 54:01 Investing versus philanthropy56:14 Technology preying on human frailty 1:00:55 Are good ideas getting harder to find? 1:06:17 Artificial intelligence 1:12:41 Funding ethics research 1:14:25 Is philosophy useful? Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Connor Leahy on the State of AI and Alignment Research Future of Life Institute 2023-04-20 | Connor Leahy joins the podcast to discuss the state of the AI. Which labs are in front? Which alignment solutions might work? How will the public react to more capable AI? You can read more about Connor's work at https://conjecture.devTimestamps: 00:00 Landscape of AI research labs 10:13 Is AGI a useful term? 13:31 AI predictions 17:56 Reinforcement learning from human feedback 29:53 Mechanistic interpretability 33:37 Yudkowsky and Christiano 41:39 Cognitive Emulations 43:11 Public reactions to AI Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Connor Leahy on AGI and Cognitive Emulation Future of Life Institute 2023-04-13 | Connor Leahy joins the podcast to discuss GPT-4, magic, cognitive emulation, demand for human-like AI, and aligning superintelligence. You can read more about Connor's work at https://conjecture.devTimestamps: 00:00 GPT-4 16:35 "Magic" in machine learning 27:43 Cognitive emulations 38:00 Machine learning VS explainability 48:00 Human data = human AI? 1:00:07 Analogies for cognitive emulations 1:26:03 Demand for human-like AI 1:31:50 Aligning superintelligence Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Lennart Heim on Compute Governance Future of Life Institute 2023-04-06 | Lennart Heim joins the podcast to discuss options for governing the compute used by AI labs and potential problems with this approach to AI safety. You can read more about Lennart's work here: https://heim.xyz/about/Timestamps: 00:00 Introduction 00:37 AI risk 03:33 Why focus on compute? 11:27 Monitoring compute 20:30 Restricting compute 26:54 Subsidising compute 34:00 Compute as a bottleneck38:41 US and China 42:14 Unintended consequences 46:50 Will AI be like nuclear energy? Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Lennart Heim on the AI Triad: Compute, Data, and Algorithms Future of Life Institute 2023-03-30 | Lennart Heim joins the podcast to discuss how we can forecast AI progress by researching AI hardware. You can read more about Lennart's work here: https://heim.xyz/about/Timestamps: 00:00 Introduction 01:00 The AI triad 06:26 Modern chip production 15:54 Forecasting AI with compute 27:18 Running out of data? 32:37 Three eras of AI training 37:58 Next chip paradigm 44:21 AI takeoff speeds Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Liv Boeree on Poker, GPT-4, and the Future of AI Future of Life Institute 2023-03-23 | Liv Boeree joins the podcast to discuss poker, GPT-4, human-AI interaction, whether this is the most important century, and building a dataset of human wisdom. You can read more about Liv's work here: livboeree.comTimestamps: 00:00 Introduction 00:36 AI in Poker 09:35 Game-playing AI 13:45 GPT-4 and generative AI 26:41 Human-AI interaction 32:05 AI arms race risks 39:32 Most important century?42:36 Diminishing returns to intelligence? 49:14 Dataset of human wisdom/meaningSocial Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Liv Boeree on Moloch, Beauty Filters, Game Theory, Institutions, and AI Future of Life Institute 2023-03-16 | Liv Boeree joins the podcast to discuss Moloch, beauty filters, game theory, institutional change, and artificial intelligence. You can read more about Liv's work here: livboeree.comTimestamps: 00:00 Introduction 01:57 What is Moloch? 04:13 Beauty filters 10:06 Science citations 15:18 Resisting Moloch 20:51 New institutions 26:02 Moloch and WinWin 28:41 Changing systems 33:37 Artificial intelligence 39:14 AI acceleration Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Tobias Baumann on Space Colonization and Cooperative Artificial Intelligence Future of Life Institute 2023-03-09 | Tobias Baumann joins the podcast to discuss suffering risks, space colonization, and cooperative artificial intelligence. You can read more about Tobias' work here: https://centerforreducingsuffering.org. Timestamps: 00:00 Suffering risks 02:50 Space colonization 10:12 Moral circle expansion 19:14 Cooperative artificial intelligence 36:19 Influencing governments 39:34 Can we reduce suffering? Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Tobias Baumann on Artificial Sentience and Reducing the Risk of Astronomical Suffering Future of Life Institute 2023-03-02 | Tobias Baumann joins the podcast to discuss suffering risks, artificial sentience, and the problem of knowing which actions reduce suffering in the long-term future. You can read more about Tobias' work here: https://centerforreducingsuffering.org. Timestamps: 00:00 Introduction 00:52 What are suffering risks? 05:40 Artificial sentience 17:18 Is reducing suffering hopelessly difficult? 26:06 Can we know how to reduce suffering? 31:17 Why are suffering risks neglected? 37:31 How do we avoid accidentally increasing suffering? Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Neel Nanda on Math, Tech Progress, Aging, Living up to Our Values, and Generative AI Future of Life Institute 2023-02-23 | Neel Nanda joins the podcast for a lightning round on mathematics, technological progress, aging, living up to our values, and generative AI. You can find his blog here: neelnanda.ioTimestamps: 00:00 Introduction 00:55 How useful is advanced mathematics? 02:24 Will AI replace mathematicians? 03:28 What are the key drivers of tech progress? 04:13 What scientific discovery would disrupt Neel's worldview? 05:59 How should humanity view aging? 08:03 How can we live up to our values? 10:56 What can we learn from a person who lived 1.000 years ago? 12:05 What should we do after we have aligned AGI? 16:19 What important concept is often misunderstood? 17:22 What is the most impressive scientific discovery? 18:08 Are language models better learning tools that textbooks? 21:22 Should settling Mars be a priority for humanity? 22:44 How can we focus on our work? 24:04 Are human-AI relationships morally okay? 25:18 Are there aliens in the universe? 26:02 What are Neel's favourite books? 27:15 What is an overlooked positive aspect of humanity? 28:33 Should people spend more time prepping for disaster? 30:41 Neel's advice for teens. 31:55 How will generative AI evolve over the next five years? 32:56 How much can AIs achieve through a web browser? Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Neel Nanda on Avoiding an AI Catastrophe with Mechanistic Interpretability Future of Life Institute 2023-02-16 | Neel Nanda joins the podcast to talk about mechanistic interpretability and how it can make AI safer. Neel is an independent AI safety researcher. You can find his blog here: neelnanda.ioTimestamps: 00:00 Introduction 00:46 How early is the field mechanistic interpretability? 03:12 Why should we care about mechanistic interpretability? 06:38 What are some successes in mechanistic interpretability? 16:29 How promising is mechanistic interpretability? 31:13 Is machine learning analogous to evolution? 32:58 How does mechanistic interpretability make AI safer? 36:54 Does mechanistic interpretability help us control AI? 39:57 Will AI models resist interpretation? 43:43 Is mechanistic interpretability fast enough? 54:10 Does mechanistic interpretability give us a general understanding? 57:44 How can you help with mechanistic interpretability? Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Neel Nanda on What is Going on Inside Neural Networks Future of Life Institute 2023-02-09 | Neel Nanda joins the podcast to explain how we can understand neural networks using mechanistic interpretability. Neel is an independent AI safety researcher. You can find his blog here: neelnanda.ioTimestamps: 00:00 Who is Neel? 04:41 How did Neel choose to work on AI safety? 12:57 What does an AI safety researcher do? 15:53 How analogous are digital neural networks to brains? 21:34 Are neural networks like alien beings? 29:13 Can humans think like AIs? 35:00 Can AIs help us discover new physics? 39:56 How advanced is the field of AI safety? 45:56 How did Neel form independent opinions on AI? 48:20 How does AI safety research decrease the risk of extinction? Social Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
Connor Leahy on Aliens, Ethics, Economics, Memetics, and Education Future of Life Institute 2023-02-02 | Connor Leahy from Conjecture joins the podcast for a lightning round on a variety of topics ranging from aliens to education. Learn more about Connor's work at https://conjecture.devSocial Media Links:➡️ WEBSITE: futureoflife.org➡️ TWITTER: twitter.com/FLIxrisk➡️ INSTAGRAM: instagram.com/futureoflifeinstitute➡️ META: facebook.com/futureoflifeinstitute➡️ LINKEDIN: linkedin.com/company/future-of-life-institute
