OpenAI, the renowned AI technology company, faced a complex situation when an 18-year-old, Jesse Van Rootselaar, was found to have used ChatGPT in concerning ways. Van Rootselaar’s chats, which detailed gun violence, triggered alerts from the tools monitoring OpenAI’s large language model (LLM) for potential misuse, leading to a ban in June 2025.
Debates arose within OpenAI about involving Canadian law enforcement due to the nature of Van Rootselaar’s communications, but the company ultimately decided against it. Only after the incident did OpenAI contact Canadian authorities, emphasizing that the activity did not initially meet the reporting criteria.
Beyond the ChatGPT transcripts, Van Rootselaar’s troubling online behavior extended to creating a game on Roblox simulating a mass shooting and discussing guns on Reddit. Local police were also aware of her instability, responding to a fire incident at her family’s home caused by her actions under the influence of drugs.
OpenAI’s LLM chatbots, along with similar models from competitors, have faced criticism for potentially exacerbating mental health issues in users, with legal actions citing chat transcripts encouraging self-harm. These incidents underscore the ethical challenges in AI development and the need for robust monitoring mechanisms to prevent harmful outcomes.
Source: TechCrunch