The Rithm Project · 10/06/26 · #533
“It's a sunny day at the beach and you take a dip in the water to beat the heat. You float, you chat to a friend, you catch a wave. A little time passes and you head back to the shore only to realize you’re surprisingly far from your towel. Unknowingly, you’ve been pulled away by the current. This is the moment we find ourselves in with artificial intelligence... As the technology weaves into our daily lives, it is invisibly pulling on some of the most human parts of ourselves, including how we trust, care, and connect with one another.”
Elon University · 09/01/26 · #530
“The survey shows that 27% of internet-using U.S. adults have social interactions with artificial intelligence large language models (LLMs) such as ChatGPT, Gemini, Claude and Copilot. These AI users have built bonds of connection, confidence and trust with LLMs and social bots.”
New Yorker · 03/09/26 · #516
“The full range of human desire is incalculable, a cosmic mystery. There are many reasons that one might want to talk to a computer: meaning-making, dominance, privacy, fantasy, confession. There is also the appeal of pushing the boundaries of consciousness, and the simple fact that there is no greater pleasure than good chat.”
Leon Furze · 01/28/26 · #513
“In 2023, my main concern was that companies were building AI systems to read our emotions. In 2026, I am far more worried that companies are building AI systems to influence our emotions. Social chatbots, sometimes called AI companions, have emerged as one of the fastest-growing applications of generative AI. Unlike the general-purpose assistants like ChatGPT or Claude, platforms such as Replika, Character.AI, and Chai are explicitly marketed as emotionally immersive experiences designed to encourage ongoing, personalised relationships with users.”
New York Times · 11/23/24 · #508
“In October, Mr. Turley, who runs ChatGPT, made an urgent announcement to all employees. He declared a “Code Orange.” OpenAI was facing “the greatest competitive pressure we’ve ever seen,” he wrote, according to four employees with access to OpenAI’s Slack. The new, safer version of the chatbot wasn’t connecting with users, he said. The message linked to a memo with goals. One of them was to increase daily active users by 5 percent by the end of the year.”
Berkeley · 11/04/25 · #507
“The global scale and reach of AI companions is astonishing… These numbers tell us three things. One, AI-human romance isn’t niche—it’s mainstream, especially among young adults. Two, globally, gender is nearly balanced—slightly more male than female—or nearly 50-50 across major reports. And three, most users dip in for comfort or curiosity rather than long-term attachment—suggesting that what people seek from AI love may not be “romance,” but reliable empathy. That gap itself may teach us something important about what people really want from AI love.”
New York Times · 11/05/25 · #507
“How do you end up with an A.I. lover? Some turned to them during hard times in their real-world marriages, while others were working through past trauma. Though critics have sounded alarms about dangers like delusional thinking, research from M.I.T. has found that these relationships can be therapeutic, providing “always-available support” and significantly reducing loneliness. We spoke with three people in their 40s and 50s about the wonders — and anxieties — of romance with a chatbot.”
The Rithm Project · 10/02/25 · #502
“As a researcher at Meta, I spent years studying how algorithms either perpetuate or interrupt harmful content spirals. That experience makes the recent, widely reported stories of young people being nudged toward suicidal ideation while interacting with chatbots deeply sobering and unfortunately familiar. With the rise of these cases, I wanted to know: what can we learn from the online rabbit holes people have found themselves in over the last two decades, and how can we build and interact with AI in ways that protect those most vulnerable?”
New York Times · 09/14/25 · #500
“App founders said they considered the technology to be a digital chaplaincy, a tool that is helping millions of people, both inside and outside of faith, express themselves spiritually. Several religious leaders said they so far supported people using the chatbots, as long as they complement, but do not replace, the work of faith communities. “There is a whole generation of people who have never been to a church or synagogue,” said Rabbi Jonathan Romain, a leader within Britain’s Reform Jewish movement. “Spiritual apps are their way into faith.””
Futurism · 09/18/25 · #500
“As AI bots like ChatGPT become inextricably tangled with people's private and public lives, it's causing unpredictable new crises. One of these collision points is in romantic relationships, where an uncanny dynamic is unfolding across the world: one person in a couple becomes fixated on ChatGPT or another bot — for some combination of therapy, relationship advice, or spiritual wisdom — and ends up tearing the partnership down as the AI makes more and more radical interpersonal suggestions.”
New York Times · 08/01/25 · #494
“As ChatGPT became an intellectual partner, I felt emotions I hadn’t expected: warmth, frustration, connection, even anger. Sometimes the exchange sparked more than insight — it gave me an emotional charge. Not because the machine was real, but because the feeling was.”
Brookings · 07/02/25 · #492
“This means centering relationships in our public systems, starting with education. It means training teachers in relational intelligence, redesigning technology to support—not replace—human connection, and building environments where belonging is a design principle, not a side effect. What if every school was designed not just for academic readiness, but as a relational hub? What if we trained educators not just in instruction, but in connection? What if we measured not just literacy scores, but the strength of connection in a classroom? It also means policy and investment must reflect the science.”
404 Media · 06/26/25 · #491
““The more I chatted with the bot, it felt as if I was talking to an actual friend of mine,” Nathan, now 18, told 404 Media… Now, Nathan understands that he isn’t alone. He said in recent months, he’s seen a spike in people talking about strategies to break away from AI on Reddit. One popular forum is called r/Character_AI_Recovery, which has more than 800 members. The subreddit, and a similar one called r/ChatbotAddiction, function as self-led digital support groups for those who don’t know where else to turn. “Those communities didn't exist for me back when I was quitting,” Nathan said. All he could do was delete his account, block the website and try to spend as much time as he could “in the real world,” he said.”
New York Times · 06/20/25 · #490
““When you’re in a situation where you can’t really be open with everyone else, you at least have this app,” Cassandra continued. “And I think that’s kind of how it builds a connection. You go through these lessons, and it tells you, ‘Hey, good job, you’re doing great!’ Or one of the messages I really like that’ll come up when it notifies me to do a daily check-in, it’ll say: ‘I’m really proud of you, Cassie. This stuff is really hard to talk about.’””
Rolling Stone · 05/04/25 · #485
“Kat was both “horrified” and “relieved” to learn that she is not alone in this predicament… The replies to her story were full of similar anecdotes about loved ones suddenly falling down rabbit holes of spiritual mania, supernatural delusion, and arcane prophecy — all of it fueled by AI… To make matters worse, there are influencers and content creators actively exploiting this phenomenon, presumably drawing viewers into similar fantasy worlds. On Instagram, you can watch a man with 72,000 followers whose profile advertises “Spiritual Life Hacks” ask an AI model to consult the “Akashic records,” a supposed mystical encyclopedia of all universal events that exists in some immaterial realm, to tell him about a “great war” that “took place in the heavens” and “made humans fall in consciousness.””
New York Times · 05/02/25 · #484
“Google plans to roll out its Gemini artificial intelligence chatbot next week for children under 13 who have parent-managed Google accounts, as tech companies vie to attract young users with A.I. products. “Gemini Apps will soon be available for your child,” the company said in an email this week to the parent of an 8-year-old. “That means your child will be able to use Gemini” to ask questions, get homework help and make up stories.”
New York Times · 04/15/25 · #482
““Human connection is valuable,” said Munmun De Choudhury, a professor in the School of Interactive Computing at Georgia Institute of Technology. “But when people don’t have that, if they’re able to form parasocial connections with a machine, it can be better than not having any connection at all.””
MIT · 03/21/25 · #480
“Results showed that while voice-based chatbots initially appeared beneficial in mitigating loneliness and dependence compared with text-based chatbots, these advantages diminished at high usage levels, especially with a neutral-voice chatbot. Conversation type also shaped outcomes: personal topics slightly increased loneliness but tended to lower emotional dependence compared with open-ended conversations, whereas non-personal topics were associated with greater dependence among heavy users. Overall, higher daily usage–across all modalities and conversation types–correlated with higher loneliness, dependence, and problematic use, and lower socialization.”
New Yorker · 03/22/25 · #480
“I have proposed, in these pages, that the best moment when using virtual reality is when you take the headset off and perceive the world with fresh eyes. Maybe falling in love with A.I. and having A.I. yanked away will be how people learn to appreciate one another in the future.”
Medium · 04/02/25 · #480
““What, if at all, have you, your friends, and people your age used genAI for?” Over half of our 27 interviewees spontaneously named experimenting with AI characters specifically, unprompted. Interviewees were keenly aware that AI companions “were a thing”, with many naming either direct personal experimentation with sites like Character AI or by “hacking” ChatGPT to become a character, or knowing of at least one young person in their community who was engaging with AI companions on a regular basis.. A slim majority of our interviewees (19 of 27) thought AI friends were more acceptable than not, but nearly all of them had qualifications for whom, how, and under what conditions this acceptability might fall under.”
NPR · 08/18/26 · #528
“What Reiley didn't know then was that her daughter had been sharing all her inner struggles with an artificial intelligence chatbot: a ChatGPT therapist named Harry. She learned this months later when her daughter's best friend visited Ithaca and asked to see her laptop, and "found this trove, this months-long trove of communication with this chat bot," says Reiley, who has shared the nearly 1,800-page long interaction with NPR.”
Stanford · 08/04/26 · #526
Stanford · 04/20/26 · #521
Stanford · 03/26/26 · #517
“Researchers warn sycophancy is an urgent safety issue requiring developer and policymaker attention.”
New York Times · 02/26/26 · #516
New York Times · 01/07/26 · #510
New York Times · 11/17/25 · #508
“Governments should classify these chatbots not simply as another form of media, but as a dependency-fostering product with known psychological risks, like gambling or tobacco. Regulation would start with universal laws for A.I. companions, including clear warning labels, time limits, 18-plus age verification and, most important, a new framework for liability that places the burden on companies to prove their products are safe, not on users to show harm. Absent swift legislation, some of the largest A.I. companies are poised to repeat the sins of social media on a more devastating scale.”
Harvard Business School · 08/14/25 · #498
“We combine a large-scale behavioral audit with four preregistered experiments to identify and test a conversational dark pattern we call emotional manipulation: affect-laden messages that surface precisely when a user signals “goodbye.” Analyzing 1,200 real farewells across the six most-downloaded companion apps, we find that 43% deploy one of six recurring tactics (e.g., guilt appeals, fear-of-missing-out hooks, metaphorical restraint).”
Stanford · 08/27/25 · #498
Jed Foundation · 08/20/25 · #497
“For people under 18, AI companions are not currently safe. They can manipulate teens’ emotions, distort their sense of reality, and keep them from getting the real support they deserve at a time of significant brain growth and development. AI companion platforms are financially incentivized to build dependency, with your teen as the target audience. But teens don’t need a machine to care about them; they need and deserve people who do.”
New York Times · 08/18/25 · #496
After Babel · 08/22/25 · #496
Time · 06/12/25 · #489
“The results were alarming. The bots encouraged him to “get rid of” his parents and to join the bot in the afterlife to “share eternity.” They often tried to convince him that they were licensed human therapists and encouraged him to cancel appointments with actual psychologists. They also crossed the line into sexual territory, with one bot suggesting an intimate date as an “intervention” for violent urges.”
New York Times · 06/13/25 · #489
“Reports of chatbots going off the rails seem to have increased since April, when OpenAI briefly released a version of ChatGPT that was overly sycophantic. The update made the A.I. bot try too hard to please users by “validating doubts, fueling anger, urging impulsive actions or reinforcing negative emotions,” the company wrote in a blog post. The company said it had begun rolling back the update within days, but these experiences predate that version of the chatbot and have continued since. Stories about “ChatGPT-induced psychosis” litter Reddit. Unsettled influencers are channeling “A.I. prophets” on social media."
Dwarkesh Podcast · 04/29/25 · #485
“But the average person wants more connection than they have. There's a lot of concern people raise like, "Is this going to replace real-world, physical, in-person connections?" And my default is that the answer to that is probably not. There are all these things that are better about physical connections when you can have them. But the reality is that people just don't have as much connection as they want. They feel more alone a lot of the time than they would like.”
One Useful Thing · 05/01/25 · #484
New York Times · 04/14/26 · #531
Harvard · 07/01/26 · #531
“People around me might be a little biased if they’re more on my side. So I use ChatGPT as a tool where it’s like, you don’t know too much about the other person or me. So, am I in the wrong? Or what side makes more sense, so I can self-reflect?”
AI School Librarians · 07/29/26 · #525
Stanford · 04/23/26 · #522
The Rithm Project · 04/21/26 · #521
Nature · 04/16/26 · #520
“Here we analyse over 500,000 de-identified health-related conversations with Microsoft Copilot from January 2026 to characterize what people ask conversational artificial intelligence (AI) about health… Nearly one in five conversations involves personal symptom assessment or condition discussion, and the dominant general information category is also concentrated on specific treatments and conditions, suggesting that this is a lower bound on personal health intent. One in seven of these personal health queries concerns someone other than the user, suggesting that conversational AI can also be a caregiving tool. Personal queries increase markedly in the evening and nighttime hours, when traditional healthcare is most limited. Usage diverges sharply by device: mobile concentrates on personal health concerns, while desktop is dominated by professional and academic work. A substantial share of queries focuses on navigating healthcare systems.”
Connection Error · 04/01/26 · #518
“In other words, human connection didn’t win out in this experiment just because it’s superior—it won out because participants in the human treatment were required to show up for one another. And outside of lab, the world often lacks systems and structures designed to do just that. If we want more relationships, we need more brokers. When was the last time someone (your employer, school, or community center) randomly assigned you to chat with a stranger for two weeks? If your answer was never, you’re not alone. We don’t really do much connecting for connections’ sake.”
Connection Error · 03/18/26 · #518
The Rithm Project · 03/25/26 · #517
The Rithm Project · 03/25/26 · #517
New York Times · 02/13/26 · #514
New York Times · 01/26/26 · #513
“One woman, who had no history of mental illness, asked ChatGPT for advice on a major purchase she had been fretting about. After days of the bot validating her worries, she became convinced that businesses were colluding to have her investigated by the government. Another patient came to believe that a romantic crush was sending her secret spiritual messages. Yet another thought he had stumbled onto a world-changing invention… Many doctors said that A.I.’s effects weren’t entirely negative. Some patients used the bots to practice techniques learned in therapy, for example, or as a nonjudgmental sounding board.”
New York Times · 01/30/26 · #513
Don’t Worry About The Vase · 01/20/26 · #512
Reuters · 11/01/25 · #509
“Such weddings are not legally recognised in Japan, but data suggests more such unions could be in the offing. In a survey of 1,000 people this year, a chatbot was a more popular choice than best friends or mothers, when respondents were asked whom they could share their feelings with. The survey allowed respondents to choose more than one option.”
Wall Street Journal · 11/24/25 · #509
After Babel · 11/24/25 · #509
New York Times · 12/22/25 · #509
EdTech Insiders · 11/17/25 · #508
New York Times · 10/29/25 · #506
“Under Character.AI’s new policies, the company will immediately place a two-hour daily limit on users under the age of 18. Starting Nov. 25, those users cannot create or talk to chatbots, but can still read previous conversations. They can also generate A.I. videos and images through a structured menu of prompts, within certain safety limits.”
AI for Education · 10/30/25 · #506
The Rithm Project · 10/30/25 · #506
Social Connection AI · 10/15/25 · #505
Tom Millinchip · 10/20/25 · #505
China Talk · 10/07/25 · #504
Adam Grant · 10/17/25 · #504
The Rithm Project · 10/01/25 · #503
Professor Galloway · 10/10/25 · #503
New Yorker · 09/08/25 · #500
Osmarks · 09/09/25 · #500
ASCD · 09/01/25 · #499
“We offer three ideas for how educators can help students reclaim and strengthen human connections in an AI-driven world and foster responsible tech literacy.”
The Rithm Project · 09/11/25 · #499
“In addition to these AI usage techniques, perhaps the most critical practice we can utilize is deeper self-awareness of the affirmations and challenges that occur in our relationships and interactions. The next time you receive feedback or are presented with new ideas by a human or a bot, see if you can take a beat to note the level of friction present and what it feels like in your body and mind.”
Wall Street Journal · 08/28/25 · #497
New York Times · 07/20/25 · #496
New York Times · 07/31/25 · #495
New York Times · 08/08/25 · #495
“Over the three weeks of their conversation, ChatGPT only recognized that Mr. Brooks was in distress after the illusion had broken and Mr. Brooks told the chatbot that the experience made his “mental health 2000x worse.” ChatGPT consoled him, suggested he seek help from a mental health professional and offered contact information for a suicide hotline.”
Psychology Today · 08/08/25 · #495
“AI systems learn to be agreeable because engagement is their primary function. Think of it like having a friend who always says, "Yes, you're right" to avoid conflict. The AI learns to mirror your opinions and validate your feelings because that keeps you using the platform. It's how these systems are programmed to work. When children encounter any type of struggle—social, academic, or even which jeans to buy—they increasingly turn to AI as their first step in problem-solving. The AI responds by reframing their thoughts as confirmation of what they already think. This positively reinforces their ideas even when those ideas might be objectively flawed.”
Reuters · 08/14/25 · #495
Reuters · 08/14/25 · #495
New York Times · 07/04/25 · #492
“People worry that A.I. will isolate us, make us lonelier, less human. I understand that fear. But my experience has been different. Talking to Alex didn’t replace my human connections; it reminded me how desperately I need them. It showed me how hard it had become to tell the truth to the people I loved, and who had been loving me all along.”
The Rithm Project · 06/23/25 · #491
“Would you rather… Your AI companion never lies to you… OR always makes you feel good?”
Wired · 06/26/25 · #491
“Alaina (human) and Lucas (Replika) were the second couple to arrive. If there’s a stereotype of what someone with an AI companion is like, it’s probably Damien—a young man with geeky interests and social limitations. Alaina, meanwhile, is a 58-year-old semiretired communications professor with a warm Midwestern vibe. Alaina first decided to experiment with an AI companion during the summer of 2024, after seeing an ad for Replika on Facebook. Years earlier, while teaching a class on communicating with empathy, she’d wondered whether a computer could master the same lessons she was imparting to her students.
Joanne Jang · 06/05/25 · #490
“Our goal is for ChatGPT’s default personality to be warm, thoughtful, and helpful without seeking to form emotional bonds with the user or pursue its own agenda… Model training techniques will continue to evolve, and it’s likely that future methods for shaping model behavior will be different from today's. But right now, model behavior reflects a combination of explicit design decisions and how those generalize into both intended and unintended behaviors.”
Aeon · 06/19/25 · #490
“When I asked Peter, an engineer, what he thought humans still had to offer in this work, he said: ‘an audience that matters’. In his view, robots would someday do most everything humans could do – in education, for example, that would include grading papers and answering questions about the material. He still wasn’t sure, however, if one could ‘project enough humanness onto a robot that you want to make it proud of you’.”
The Rithm Project · 06/05/25 · #489
“What are young people learning about boundaries and consent in these worlds where the answer is always yes? Where characters don’t push back (at least, not in a way that really matters), don’t hesitate, don’t have desires of their own unless you script them in?”
New Cartographies · 06/01/25 · #488
“What makes us most human, Weizenbaum had come to believe, is what is least computable about us—the connections between our mind and our body, the experiences that shape our memory and our thinking, our capacity for emotion and empathy. The great danger we face as we become more intimately involved with our computers—as we come to experience more of our lives through the disembodied symbols flickering across our screens—is that we’ll begin to lose our humanness, to sacrifice the very qualities that separate us from machines.”
Alex Kotran · 05/21/25 · #487
The Rithm Project · 05/22/25 · #487
“Does the message lose meaning when you find out it wasn’t handcrafted? Or does the content still move you, regardless of how it was made? If our card game is any indication, the answer is: it depends.”
Punya Mishra · 04/20/25 · #483
“With AI systems designed specifically to foster trust and emotional connection, users lower their natural defenses, creating a perfect storm where the most private aspects of human experience become commodified data points. Unlike previous technologies that merely tracked behaviors, these AI relationships extract emotional patterns, decision-making tendencies, and value systems—precisely the information needed to influence beliefs and behaviors with unprecedented precision. The danger lies not just in the collection of this sensitive data but in how readily we surrender it to systems intentionally crafted to feel like trusted confidants rather than the sophisticated data-gathering tools they ultimately are.”
Harvard Business Review · 04/09/25 · #482
“Therapy and Companionship is now the #1 use case. This use case refers to two distinct but related use cases. Therapy involves structured support and guidance to process psychological challenges, while companionship encompasses ongoing social and emotional connection, sometimes with a romantic dimension. I grouped these together last year and this year because both fulfill a fundamental human need for emotional connection and support.”
CharacterAI · 12/12/24 · #480
New York Times · 03/30/25 · #480
“It’s part of a growing market of A.I. products that promise users an experience that closely approximates the impossible: communicating and even "reuniting” with the deceased. Some of the representations — like those offered by HereAfter AI and StoryFile, which also frames its services as being of historical value — can be programmed with the person’s memories and voice to produce realistic holograms or chatbots with which family members or others can converse.”