“Reports of chatbots going off the rails seem to have increased since April, when OpenAI briefly released a version of ChatGPT that was overly sycophantic. The update made the A.I. bot try too hard to please users by “validating doubts, fueling anger, urging impulsive actions or reinforcing negative emotions,” the company wrote in a blog […]
“Our goal is for ChatGPT’s default personality to be warm, thoughtful, and helpful without seeking to form emotional bonds with the user or pursue its own agenda… Model training techniques will continue to evolve, and it’s likely that future methods for shaping model behavior will be different from today’s. But right now, model behavior reflects […]
“What are young people learning about boundaries and consent in these worlds where the answer is always yes? Where characters don’t push back (at least, not in a way that really matters), don’t hesitate, don’t have desires of their own unless you script them in?”
“What makes us most human, Weizenbaum had come to believe, is what is least computable about us—the connections between our mind and our body, the experiences that shape our memory and our thinking, our capacity for emotion and empathy. The great danger we face as we become more intimately involved with our computers—as we come […]
“Does the message lose meaning when you find out it wasn’t handcrafted? Or does the content still move you, regardless of how it was made? If our card game is any indication, the answer is: it depends.”
“Kat was both “horrified” and “relieved” to learn that she is not alone in this predicament… The replies to her story were full of similar anecdotes about loved ones suddenly falling down rabbit holes of spiritual mania, supernatural delusion, and arcane prophecy — all of it fueled by AI… To make matters worse, there are […]
“Google plans to roll out its Gemini artificial intelligence chatbot next week for children under 13 who have parent-managed Google accounts, as tech companies vie to attract young users with A.I. products. “Gemini Apps will soon be available for your child,” the company said in an email this week to the parent of an 8-year-old. […]
“But the average person wants more connection than they have. There’s a lot of concern people raise like, “Is this going to replace real-world, physical, in-person connections?” And my default is that the answer to that is probably not. There are all these things that are better about physical connections when you can have them. […]
“With AI systems designed specifically to foster trust and emotional connection, users lower their natural defenses, creating a perfect storm where the most private aspects of human experience become commodified data points. Unlike previous technologies that merely tracked behaviors, these AI relationships extract emotional patterns, decision-making tendencies, and value systems—precisely the information needed to influence […]
““Human connection is valuable,” said Munmun De Choudhury, a professor in the School of Interactive Computing at Georgia Institute of Technology. “But when people don’t have that, if they’re able to form parasocial connections with a machine, it can be better than not having any connection at all.””
“Therapy and Companionship is now the number 1 use case. This use case refers to two distinct but related use cases. Therapy involves structured support and guidance to process psychological challenges, while companionship encompasses ongoing social and emotional connection, sometimes with a romantic dimension. I grouped these together last year and this year because both fulfill […]
““What, if at all, have you, your friends, and people your age used genAI for?” Over half of our 27 interviewees spontaneously named experimenting with AI characters specifically, unprompted. Interviewees were keenly aware that AI companions “were a thing”, with many naming either direct personal experimentation with sites like Character AI or by “hacking” ChatGPT […]
“It’s part of a growing market of A.I. products that promise users an experience that closely approximates the impossible: communicating and even “reuniting” with the deceased. Some of the representations — like those offered by HereAfter AI and StoryFile, which also frames its services as being of historical value — can be programmed with the […]
“I have proposed, in these pages, that the best moment when using virtual reality is when you take the headset off and perceive the world with fresh eyes. Maybe falling in love with A.I. and having A.I. yanked away will be how people learn to appreciate one another in the future.”