OpenAI fixing bug that let kids generate erotic ChatGPT chats

OpenAI is working to fix a bug that allowed — and, in some cases, reportedly encouraged — minors to generate erotica on ChatGPT, as TechCrunch reported on Monday, April 28. OpenAI told TechCrunch that the company is “actively deploying a fix,” since its policies don’t allow its AI chatbot to send responses like this to users under 18 years old.

Young people have been finding ways to access pornography and erotica since the dawn of the internet, so it’s not surprising that youngsters would also test the limits of the popular AI chatbot.

An OpenAI spokesperson told Mashable over email that its “model policies don’t allow the kind of responses that happened here, and they shouldn’t have been shown to users.”

“Protecting younger users is a top priority, and our Model Spec, which guides model behavior, clearly restricts sensitive content like erotica to narrow contexts such as scientific, historical, or news reporting. In this case, a bug allowed responses outside those guidelines, and we are actively deploying a fix to limit these generations,” the OpenAI statement read.

You must be 13 years old to use ChatGPT — and even then, you need parental consent if you’re under 18. But those rules can be skirted pretty easily as long as you have an email account. Anyone can lie about their age online, after all.

In February, OpenAI updated its policies on how ChatGPT approaches sensitive subjects, including erotica. This may have made it easier for underage users pretending to be adults to generate more sexual responses.

Meta’s AI chatbot has also been accused of serving up explicit content to minors. According to a report earlier this week from The Wall Street Journal, Meta loosened the guardrails around its chatbots on Instagram, Facebook, and WhatsApp to make them as engaging as possible. As the WSJ reported, this allowed Meta’s AI chatbots to engage in sexual fantasies with young users, although not intentionally. Not only that, but users could even get those fantasies read aloud in the voices of Dame Judi Dench, John Cena, and Kristen Bell. Meta told the outlet that the testing was manipulative and not representative of real-world use.

Disclosure: Ziff Davis, Mashable’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.

UPDATE: Apr. 29, 2025, 1:47 p.m. EDT This article has been updated with a response from OpenAI.