Microsoft AI Chief Mustafa Suleyman has stated that he is worried about AI systems seeming too alive. Suleyman, while talking about the phenomenon of “AI psychosis,” said that these new models are capable of convincing users that they have feelings and are seemingly conscious.
Suleyman, however, isn’t talking about a doomsday scenario where AI actually becomes smart enough to understand these things and take over the world, but rather a situation where people start forming delusions about their AI—such as thinking it is a god, a lover, or a digital person—believing it to be conscious.
In a blog post last week about his concerns, Suleyman wrote, “My central worry is that many people will start to believe in the illusion of AIs as conscious entities so strongly that they’ll soon advocate for AI rights, model welfare and even AI citizenship.”
Notably, recent research by EduBirdie revealed that most Gen Z users of AI tools believe that while AI systems aren’t yet conscious, they soon will be, while a significant portion (25%) already believe that AI is conscious.
Meanwhile, there have been various instances of people developing feelings for their AI or carrying out tasks at the behest of their chatbot. Recently, after OpenAI decided to deprecate the GPT-4o model, many users took to the dedicated subreddit to express their passion for bringing it back, with some even considering the model a friend or companion.
“It feels different and stronger than the kinds of attachment people have had to previous kinds of technology” OpenAI CEO Sam Altman had also warned in a recent post on X.
“People have used technology including AI in self-destructive ways” Altman added.
Mustafa Suleyman on AI companions:
Suleyman also talked about the need to put guardrails in place to protect people. He said, “We must build AI for people; not to be a digital person. AI companions are a completely new category, and we urgently need to start talking about the guardrails we put in place to protect people and ensure this amazing technology can do its job of delivering immense value to the world.”
“I’m fixated on building the most useful and supportive AI companion imaginable. But to succeed, I also need to talk about what we, and others, shouldn’t build.” Suleyman added
About the Author
