Uploaded March 2026 | Updated September 2026, 1 day ago
Western Branch Meeting 2026-1 of the Royal Society of NSW — 25 March 2026
"The illusion of friendship: Why generative AI demands unprecedented ethical vigilance"
Professor Zahid Islam FRSN
Professor in Computer Science and
Director, AI and Cyber Futures Centre
Charles Sturt University
https://www.royalsoc.org.au/events/western-nsw-branch-meeting-2026-1/
The presentation commences at: 04:24
Timing Marks:
00:00:00 Introduction: Distinguished Professor Sarah O'Shea, Dean, Graduate Research, Charles Sturt University
00:04:24 Presentation: Zahid Islam
00:37:43 Audience Q&A moderated by Professor Sarah O'Shea
00:56:20 Meeting Close
This lecture was a joint presentation of Charles Sturt University, as part of its Provocations Public Lecture Series, and the Western NSW Branch of the Royal Society of NSW
Summary: Generative AI (GenAI) systems such as ChatGPT are increasingly used for drafting, summarisation, tutoring, and decision support, offering substantial gains in productivity and reduced cognitive load. However, the same natural-language fluency that makes these systems useful can also blur the boundary between tool and companion. This boundary confusion may also encourage some users to experience GenAI as empathic, benevolent, and relationally persistent.
Emerging reports and early findings suggest that some users may form emotionally significant attachments to conversational agents, in some cases with harmful consequences, including delayed help-seeking, dependency, and impaired judgment in high-stakes contexts.
In this lecture, Professor Zahid Islam develops a philosophical and ethical argument for why the resulting “illusion of friendship” is both understandable and ethically risky. Drawing on classical accounts of friendship (including Epicurean, Confucian, and Aristotelian perspectives), the presentation explains why users may understandably interpret sustained supportive interaction as friend-like and companionship-like.
The lecture then advances a counterargument that, despite relational appearances, GenAI lacks moral agency, i.e., consciousness, intention, and accountability, and therefore does not qualify as a moral agent or a true friend. To demystify the illusion, the talk presents for an everyday audience a mechanism-level explanation of how transformer-based GenAI generates responses via tokenisation, embeddings, self-attention, and probabilistic next-token prediction, often producing emotionally resonant language without inner states or commitments.
The lecture concludes by outlining a safeguard framework for safe and responsible GenAI use, education as a foundational defence, human-in-the-loop accountability, and design-level interventions to reduce possible anthropomorphic cues generated by the GenAI systems. The central contribution is to demystify the illusion of friendship and explain the computational background so that we can shift the emotional attachment with GenAI towards necessary human responsibility and thereby understand how institutions, designers, and users can preserve GenAI’s benefits while mitigating over-reliance and emotional misattribution.
About the speaker: Md Zahidul Islam (commonly known as Zahid Islam) is a Professor of Computer Science at Charles Sturt University. He is the Centre Director of the AI and Cyber Futures Centre and serves as Associate Dean (Research) in the Faculty of Business, Justice and Behavioural Sciences.
Professor Islam’s research spans data mining, privacy and cybersecurity, with a strong focus on applying advanced data analytics to real-world problems across industry and society. His work contributes to the responsible development and deployment of emerging digital technologies, particularly in high-impact and high-risk contexts.
Disclaimer: The opinions, beliefs and viewpoints expressed by the presenters or forum participants in this video, do not necessarily reflect those of the Royal Society of New South Wales.
Western Branch Meeting 2026-1 of the Royal Society of NSW — 25 March 2026
"The illusion of friendship: Why generative AI demands unprecedented ethical vigilance"
Professor Zahid Islam FRSN
Professor in Computer Science and
Director, AI and Cyber Futures Centre
Charles Sturt University
https://www.royalsoc.org.au/events/western-nsw-branch-meeting-2026-1/
The presentation commences at: 04:24
Timing Marks:
00:00:00 Introduction: Distinguished Professor Sarah O'Shea, Dean, Graduate Research, Charles Sturt University
00:04:24 Presentation: Zahid Islam
00:37:43 Audience Q&A moderated by Professor Sarah O'Shea
00:56:20 Meeting Close
This lecture was a joint presentation of Charles Sturt University, as part of its Provocations Public Lecture Series, and the Western NSW Branch of the Royal Society of NSW
Summary: Generative AI (GenAI) systems such as ChatGPT are increasingly used for drafting, summarisation, tutoring, and decision support, offering substantial gains in productivity and reduced cognitive load. However, the same natural-language fluency that makes these systems useful can also blur the boundary between tool and companion. This boundary confusion may also encourage some users to experience GenAI as empathic, benevolent, and relationally persistent.
Emerging reports and early findings suggest that some users may form emotionally significant attachments to conversational agents, in some cases with harmful consequences, including delayed help-seeking, dependency, and impaired judgment in high-stakes contexts.
In this lecture, Professor Zahid Islam develops a philosophical and ethical argument for why the resulting “illusion of friendship” is both understandable and ethically risky. Drawing on classical accounts of friendship (including Epicurean, Confucian, and Aristotelian perspectives), the presentation explains why users may understandably interpret sustained supportive interaction as friend-like and companionship-like.
The lecture then advances a counterargument that, despite relational appearances, GenAI lacks moral agency, i.e., consciousness, intention, and accountability, and therefore does not qualify as a moral agent or a true friend. To demystify the illusion, the talk presents for an everyday audience a mechanism-level explanation of how transformer-based GenAI generates responses via tokenisation, embeddings, self-attention, and probabilistic next-token prediction, often producing emotionally resonant language without inner states or commitments.
The lecture concludes by outlining a safeguard framework for safe and responsible GenAI use, education as a foundational defence, human-in-the-loop accountability, and design-level interventions to reduce possible anthropomorphic cues generated by the GenAI systems. The central contribution is to demystify the illusion of friendship and explain the computational background so that we can shift the emotional attachment with GenAI towards necessary human responsibility and thereby understand how institutions, designers, and users can preserve GenAI’s benefits while mitigating over-reliance and emotional misattribution.
About the speaker: Md Zahidul Islam (commonly known as Zahid Islam) is a Professor of Computer Science at Charles Sturt University. He is the Centre Director of the AI and Cyber Futures Centre and serves as Associate Dean (Research) in the Faculty of Business, Justice and Behavioural Sciences.
Professor Islam’s research spans data mining, privacy and cybersecurity, with a strong focus on applying advanced data analytics to real-world problems across industry and society. His work contributes to the responsible development and deployment of emerging digital technologies, particularly in high-impact and high-risk contexts.
Disclaimer: The opinions, beliefs and viewpoints expressed by the presenters or forum participants in this video, do not necessarily reflect those of the Royal Society of New South Wales.
