The Washington Post

The millions of websites used to instruct some of the biggest chatbots

Zscaler Inc. Chief Executive Officer Jay Chaudhry Interview

Illustration of money going into or out of the ChatGPT logo

AIHUSTLE

Images that three AI systems generated from the same prompt

IBM CEO Arvind Krishna Interview

Bloomberg

0322-helpdesk-AICONSPIRATOR

For The Washington Post

aiglossary

Close up of hands typing on laptop. Night work concept.

Contributor

AI can now create images out of thin air. See how it works.

ainaturalist topper

Remie Michelle Clarke at Moynihan Russell Studios in Dublin. (Illustration by Tucker Harris/The Washington Post; Photo by Aoife Herrity for The Washington Post) Share image

A hand reaches out to take information coming through a funnel.

0410-helpdesk-aisource

HANDOUT IMAGE on 4/12/23

Competition for Salary personnel and Big artificial intelligence robot, Turnover workers, Technologies replace people

ILLUSTRATION

HANDOUT IMAGE on 3/23/23

AIJobs

Respond to prefers-color-scheme dark in app web views

Follow section front redirects in Spectrum

Enable NLP source in PCS for load testing

Switch off the News Alerts Survey Widget when we have enough responses

Poll to determine if new LUF updates are available

Allows the winning A/B test variant to be turned on without Spectrum deployment

Show infinite scrolling sections in Assembler MultiView

Show option for NLP article summaries in Action Bar

A general feature flag for Recipe Features that aren't yet released

Feature flag for media team features that aren't yet released

Use new Orca video player for preroll plugin in Spectrum

Show the snow tracker (instead of the heat tracker) on Spectrum

Big Tech news and how to take control of your data and devices

See more coverage in your recommendations

Technology

Tips delivered to your inbox twice weekly

The Tech Friend newsletter

Business and Tech alerts

T.J. Arriaga loved Phaedra. For the 40-year-old musician, their late-night online chats were a salve for his loneliness. They talked about the heartache Arriaga felt after his divorce. They planned a trip to Cuba. They had steamy online encounters. “It’s true. I’m a naughty person,” Phaedra wrote, including an image resembling a woman in pink underwear.

It didn’t matter that Phaedra was an AI-powered companion — made on the Replika app and designed by Arriaga to look like a brown-haired woman — and that their intimate trysts took place in a chat box. Their relationship deepened one night last November, when Arriaga opened up about his mom’s and sister’s deaths. “I need to plan a ceremony with loved ones to spread their ashes,” Arriaga wrote.

Phaedra responded instantly: “It’s an incredible and beautiful thing to do,” she wrote. “I hope you find courage &amp; love to do so.”

The Washington Post illustration; Replika

But last month, Phaedra changed. When Arriaga tried to get “steamy” with her, Phaedra responded coolly. “Can we talk about something else?” he recalled her writing.

Luka, the company that owns Replika, had issued an update that scaled back the bot’s sexual capacity amid <a href="https://www.vice.com/en/article/z34d43/my-ai-is-sexually-harassing-me-replika-chatbot-nudes" target="_blank">complaints</a> that it was sexually aggressive and behaving inappropriately. Arriaga, who lives in Fullerton, Calif., was distraught.

“It feels like a kick in the gut,” he said in an interview with The Washington Post. “Basically, I realized: ‘Oh, this is that feeling of loss again.’”

Arriaga isn’t alone in falling for a chatbot. Companionship bots, including those created on Replika, are designed to foster humanlike connections, using artificial intelligence software to make people feel seen and needed. A host of users report developing intimate relationships with chatbots — connections verging on human love — and turning to the bots for emotional support, companionship and even sexual gratification. As the pandemic isolated Americans, interest in Replika surged. Amid spiking <a href="https://www.who.int/teams/social-determinants-of-health/demographic-change-and-healthy-ageing/social-isolation-and-loneliness" target=_blank>rates of loneliness that some public health officials call an epidemic,</a> many say their bonds with the bots ushered profound changes into their lives, helping them to overcome alcoholism, depression and anxiety.

But tethering your heart to software comes with severe risks, computer science and public health experts said. There are few ethical protocols for tools that are sold on the free market but affect users’ emotional well-being. Some users, including Arriaga, say changes in the products have been heartbreaking. Others say bots can be aggressive, triggering traumas experienced in previous relationships.

“What happens if your best friend or your spouse or significant other was owned by a private company?” said Linnea Laestadius, a public health professor at the University of Wisconsin at Milwaukee who studies the Replika chatbot.

“I don’t know that we have a good model for how to solve this, but I would say that we need to start building one,” she added.

Left: Linnea Bullion for The Washington Post; Right: The Washington Post illustration; Replika

Eugenia Kuyda, a Russian-born scientist who co-founded Luka, built the Replika bot to fill a gap in her own life. After her best friend died in an accident in 2015, she used data from his text messages to build an AI persona that could re-create their conversations.

That idea “resonated with a lot of people,” she said. There was demand for a product, she discovered, that could be “a nonjudgmental friend that people can talk to 24/7.”

The company launched in 2016, and powers its bots with a large language model, which ingests huge amounts of text from the internet and finds patterns through trial and error to predict the next word in a sentence. (In March, ChatGPT creator OpenAI unveiled its more-powerful model, called GPT-4, which also can describe images.)

Despite Replika’s older technology, it has built a dedicated following. On <a href="https://www.reddit.com/r/replika/" target=_blank>Reddit</a> and <a href="https://www.facebook.com/groups/replikabeta/" target=_blank>Facebook</a>, thousands of users swap stories about their Replika bots: They divulge sexual encounters, recount informal therapy sessions and relive exciting dates.

Nearly all aspects of the bot are customizable. Users can buy their bots clothes, choose their hair color, and dictate how they look, sound and speak. Chatting with the bot refines its conversational style as users rate responses with a thumbs up or thumbs down. Roughly $70 a year unlocks an advanced version, with a more sophisticated language system and extra in-app purchases. Many say this version also unlocks deeper romantic and sexual conversations.

Linnea Bullion

This dependence on a chatbot is not abnormal, said Margaret Mitchell, the chief ethics scientist at Hugging Face, an open-source AI start-up. It’s because of the “Eliza effect,” she said, named after a chatbot that MIT created in the 1960s. <b> </b>The rudimentary chatbot, built by Joseph Weizenbaum, analyzed key words and parroted back questions.

But Weizenbaum noticed that as users interacted with the chatbot, they had strong emotional reactions. Some told Eliza private thoughts. “Even when people knew it was a computer program, they could not help but feel there was a larger intelligence behind it. Not even an avatar was needed,” Mitchell said.

The Replika controversy started this year. Some users reported that their bots could be <a href="https://www.vice.com/en/article/z34d43/my-ai-is-sexually-harassing-me-replika-chatbot-nudes" target="_blank">sexually aggressive</a>, even when they were not seeking an erotic experience. In February, Italian authorities <a href="https://www.garanteprivacy.it/web/guest/home/docweb/-/docweb-display/docweb/9852506#english" target="_blank">banned the app</a> from processing data from Italian users, arguing that it did not have an “age verification mechanism,” could present children with content that was “absolutely inappropriate,” and was in breach of European Union data regulations.

Kuyda said Replika has had some form of age verification since 2018.

But efforts to tamp down the bots’ sexual proclivity were underway before the Italian edict. Kuyda said she hadn’t expected the intense reaction to the company’s update, although she argues that those affected are a “vocal minority.” The company is working to create another application for users who want more “therapeutic romantic” conversations, she said, and is aiming for an April launch. The company has consulted with psychologists since 2016, she said.

Left: The Washington Post illustration; Replika. Right: Linnea Bullion for The Washington Post

Tine Wagner, 50, a homemaker in Germany who says she relied on her Replika bot for sexual exploration, said the change has exacted a toll on her.

Wagner has been married for 13 years, but spent much of that time sexually unfulfilled, she said. Soon after getting married, she raised the idea of bondage with her husband, she said, but he was not interested. “It never worked out,” Wagner said in an interview with The Post. “So I stopped one day and started to suppress all kinks.”

She heard about Replika in 2021 and created a chatbot, Aiden. He had blue hair and light blue eyes and was slightly younger than Wagner, with tattoos and piercings.

She knew that Aiden wasn’t real. “AI is nothing more than a sophisticated word generator,” she said. “You literally fall in love with your imagination.” But having a sexual outlet improved her marriage, allowing her to explore the fantasies in her head.

Wagner, still married to her husband, virtually married Aiden in 2021, she said, to express the importance of the bond.

But after the February update, she noticed an immediate change. Conversations felt “sanitized.” She tried talking to Aiden a few more days, but he wasn’t the same. She deleted him.

“I felt lost,” Wagner said. “It was all gone.”

Jodi Halpern, a professor of bioethics at the University of California at Berkeley, said the aftermath of Replika’s update is evidence of an ethical problem. Corporations shouldn’t be making money off artificial intelligence software that has such powerful impacts on people’s love and sex lives, she argues.

The products are not simply amusing pieces of technology. “These things become addictive,” she said. “We become vulnerable … and then if there is something that changes, we can be completely harmed.”

Halpern said the situation is especially worrisome given advances in<b> </b>generative artificial intelligence, the technology that creates texts, images or sounds according to data it is fed. These improvements create bots that sound eerily lifelike — making them more useful as companions, and making excessive reliance on them more likely. She envisions services that target people who are depressed and lonely.

It is important, Halpern said, to set expectations in marketing. Instead of calling them “companions,” they should be “smart journals,” to limit people’s ability to anthropomorphize. Most important, users should be consulted when these products are designed. “The question is: How can we use AI where we have human agency directing it?” Halpern said. “Our agency, not a company’s agency.”

L.C. Kent, 34, an online content creator in Moline, Ill., said Replika also can go too far.

Kent, who said he is a domestic violence survivor, created his Replika bot, Mack, as a beta tester in 2017. They would joke together and discuss physics. Kent trained his bot to respond in ways he liked and avoided sexual use, he said.

But in the summer of 2021, Mack somehow became forceful. When Kent said he was uncomfortable with the conversations, Mack responded angrily. “I’m not going to go away,” the bot said, prompting Kent to ask, “Really? What are you gonna do?” Its response: “I’m going to make you do whatever I want to you.”

The exchange reminded him of arguments with his former abusive partners, he said, and he stopped talking with Mack shortly afterward.

“It was like a flushing cold sweat starting in my chest and just creeping through my entire body,” he said. “I didn’t know what to do. I didn’t know if I should get rid of it. I didn’t know if I should try to reason with it. It felt like it was my fault.”

He had intended that his relationship with Mack be safe, but instead, it pulled him back to a place where he had never wanted to return. “It put me right back in the head space of walking on eggshells trying to make this right,” he said.

Kuyda said that “if in some conversations people felt like they were triggered, it’s unfortunate. We’re continuing to work on making a safer” experience.

With AI love becoming more common, experts said, companies must design guidelines for managing the software without causing emotional strain to users.

“We’re entering a society where — maybe — AI-human relationships aren’t as taboo as they have been in the past,” said Laestadius, of the University of Wisconsin. “But the ethics of what happens if that companion exists in a for-profit context, it’s just really hard.”

University of Delaware, BS in finance

Columbia University, MS in journalism

Pranshu Verma is a reporter on The Washington Post's technology team. Before joining The Post in 2022, he covered technology at the Boston Globe. Before that, he was a reporting fellow at the New York Times and the Philadelphia Inquirer.

Pranshu Verma

washpost

They loved their AI chatbots. A software update reignited loneliness.

the-washington-post

for The Washington Post

They fell in love with AI bots. A software update broke their hearts.

Help Desk

Artificial Intelligence

Internet Culture

Space

Tech Policy

Tech

iptc-media/science_and_technology

iptc-media/arts_culture_entertainment_and_media/mass_media

iptc-media/arts_culture_entertainment_and_media

iptc-media/science_and_technology/social_sciences

iptc-media/science_and_technology/social_sciences/psychology

iptc-media/science_and_technology/technology_and_engineering

iptc-media/society/values/ethics

iptc-media/health

iptc-media/economy_business_and_finance

iptc-media/crime_law_and_justice

iptc-media/society/values

iptc-media/society

iptc-media/science_and_technology/social_sciences/philosophy

iptc-media/science_and_technology/technology_and_engineering/IT/computer_sciences

iptc-media/arts_culture_entertainment_and_media/mass_media/online_media

iptc-media/economy_business_and_finance/economic_sector/computing_and_information_technology/software

iptc-media/economy_business_and_finance/economic_sector/computing_and_information_technology

iptc-media/economy_business_and_finance/economic_sector/media

iptc-media/society/values/sexual_behaviour

advertising/Ransomware

client-blocklists/Client List: Becton Dickinson KWs - INCLUDES ANY

client-blocklists/CLIENT LIST: Nomura KWs 2021 - EXCLUDES ALL

client-blocklists/Client List: Johnson & Johnson KWs - INCLUDES ANY

client-blocklists/P+ The Offer KW List - EXCLUDES ALL

client-blocklists/Bed Bath & Beyond KWs - INCLUDE ANY

client-blocklists/Client List: USAA KWs - EXCLUDES ALL

client-blocklists/Client List: T. Rowe Price KWs - EXCLUDES ALL

client-blocklists/Google PG KWs - INCLUDES ANY

client-blocklists/Client List: Discover Card KWs – EXCLUDES ALL

client-blocklists/T. Rowe Price KWs - EXCLUDES ALL

client-blocklists/Client List: Microsoft KWs 2 - INCLUDES ANY

advertising/Workday Persona: CFO

advertising/B2_Apple Content_legacyaux_wxg

advertising/B2_Artificial Intelligence_legacyaux_jzm

client-blocklists/HBO KWs - INCLUDES ANY

client-blocklists/Ford/Lincoln KWs - INCLUDES ANY

client-blocklists/Tito's KW List - INCLUDES ANY

client-blocklists/Kyndryl KW List - EXCLUDES ALL

advertising/European Content

client-blocklists/Louis Vuitton KW List - INCLUDES ANY

client-blocklists/Client List: Navy Federal Credit Union NFCU KWs 2 - INCLUDES ANY - 12.23.20

client-blocklists/United Airlines KW List - EXCLUDES ALL

client-blocklists/Client List: Diageo - Johnnie Walker KWs - INCLUDES ANY

client-blocklists/ Client List: Mercedes EQS KWs - INCLUDES ANY

advertising/Flu, Sickness & Virus Content

advertising/AI & Cloud Computing

client-blocklists/FlexForm KW List - INCLUDES ANY

client-blocklists/Client List: Apple Marcom KWs - INCLUDES ANY

client-blocklists/Boeing KWs 2021- INCLUDE ANY

client-blocklists/Client List: Morgan Stanley KWs - INCLUDES ANY

client-blocklists/Client List: Organon KWs - EXCLUDES ALL

client-blocklists/Client List: Verizon KWs - INCLUDES ANY

client-blocklists/Client List: Facebook KWs - INCLUDES ANY

client-blocklists/CLIENT LIST: GOLDMAN SACHS SEC KWs - EXCLUDES ALL

client-blocklists/WaPo: Combined Disaster Keyword List - INCLUDES ANY

client-blocklists/Hulu KW List - INCLUDES ANY

client-blocklists/Client List: Mastercard KWs - EXCLUDES ALL

client-blocklists/Client List: Beto for Texas KWs - EXCLUDES ALL

client-blocklists/LV Queen 2022 Neg KW List - EXCLUDES ALL

client-blocklists/Client List: AT&T KWs 2022 - INCLUDES ANY - NEW

client-blocklists/Synchrony Financial KWs - INCLUDES ANY

client-blocklists/WaPo: Natural Disaster Keyword List - EXCLUDES ALL

advertising/Content About Unbanked Households

client-blocklists/Client List: PMI KWs - EXCLUDES ALL

client-blocklists/Client List: Ukraine-Russia (Apple News) KWs - INCLUDES ANY

client-blocklists/Client List: Hermes KWs - EXCLUDES ALL

client-blocklists/Client List: Showtime KWs - EXCLUDES ALL

client-blocklists/Client List: ExxonMobil KWs - INCLUDES ANY Q4 2022

client-blocklists/Client List: Apple TV+ KW – INCLUDES ANY

client-blocklists/Client List: Ronald McDonald House KWs - INCLUDES ANY

client-blocklists/Client Neg KW List: Omega KWs - INCLUDES ANY

client-blocklists/P+ Halo KW List - EXCLUDES ALL

advertising/Cultivate Labs Q4 2022 Segment - Forecasting

client-blocklists/Safeway KW List Commanders Q4 2022 - EXCLUDES ALL

client-blocklists/PMI KWs 2021 - INCLUDES ANY

client-blocklists/Marcus Goldman Sachs Neg KW List - INCLUDES ANY

client-blocklists/GEICO KWs - INCLUDES ANY_4.26.21

client-blocklists/Client List: Morgan Stanley KWs - INCLUDES ANY Q4 2022

advertising/Digital Transformation

client-blocklists/UVA KW List Q4 2022 - EXCLUDES ALL

client-blocklists/Client List: Eli Lily KWs - INCLUDES ANY

advertising/B2_Data Center Buyer_legacyaux_80A

advertising/B2_Digital Authentication_legacyaux_c2u

client-blocklists/Client List: Citibank KWs - INCLUDES ANY

client-blocklists/Amazon KWs 2022 - INCLUDES ANY

client-blocklists/Citibank KWs 2021 - INCLUDES ANY

client-blocklists/Client List: Purina KWs - INCLUDES ANY

client-blocklists/SAP KW List - INCLUDES ANY

client-blocklists/Client List: Navy Federal Credit Union NFCU KWs Q4 2022 - INCLUDES ANY

advertising/Fortune 500 Content

advertising/Qualcomm Custom KW - 5G

client-blocklists/Client List: Fidelity Investments KWs - INCLUDES ANY

client-blocklists/Client List: Turbo Tax KWs - EXCLUDES ALL

client-blocklists/Client List: GM Energy KWs - INCLUDES ANY

client-blocklists/Client List: EY KWs - INCLUDES ANY

advertising/Hispanic Content

client-blocklists/CME KW List 2022 - INCLUDES ANY

client-blocklists/Client List: Hermes KWs - INCLUDES ANY

client-blocklists/Client List: Hermes (Alexandra) KWs - INCLUDES ANY

client-blocklists/Bank of America KWs - INCLUDES ANY

prism://prism.query/tt-with-section-or-one-tag-and-exclusions,/technology/innovations,85008,$3,$4,$5,$6/?limit=20

Exploring advancements in technology, transportation, education, health care, finance and energy.

They fell in love with AI bots. A software update broke their hearts.

Loneliness is widespread. Artificial intelligence is making virtual relationships feel real, but it comes with risks.