OpenAI tightens ChatGPT’s limits on weapons and past mass attacks

ChatGPT will stop giving teenagers practical information about weapons and security systems, even when a question looks harmless, OpenAI said on Friday. The limit applies to teen accounts with no sign of violent intent.

OpenAI announced the change in a blog post, alongside a promise to publish a Community Safety Report on violence prevention. It made more than 200 referrals to law enforcement in the third quarter, the company said. The new safeguards will start rolling out in the next few weeks.

For all users, ChatGPT will give less detail about past mass attacks and the people who carried them out, even when that detail is available elsewhere online. It will not reconstruct an attack, including which weapons the attacker used and how they got them. It will also avoid detailed comparisons of how much harm attacks caused.

“These limits apply regardless of a user’s age or reason for asking,” OpenAI wrote.

People can still ask how such events affected victims and communities, and how to spot warning signs, OpenAI said.

When a chat shows signs that someone is fixating on violence or on seriously harming another person, ChatGPT will be able to end it automatically. If ChatGPT ends several chats for that reason, OpenAI may pause the account for a time.

OpenAI said it has strengthened its age prediction system, which estimates from signals such as behaviour whether a user is under 18. It now keeps checking new signals and moves users into the teen experience when it sees fit. Adults placed there by mistake can confirm their age through Persona, OpenAI’s age verification partner.

Since July, parents with linked accounts get an alert if OpenAI deactivates a teen’s account over violence. OpenAI said it is working on alerts for when it pauses a teen’s account for repeatedly discussing violence.

“Developing effective violence safeguards in AI systems requires solving novel problems in recognizing risk and responding appropriately,” OpenAI wrote.

The first Community Safety Report will come out in early 2027 and cover the second half of 2026, with updates every six months. It will explain how OpenAI detects and responds to safety issues, with anonymised examples and data, the company said.

The update adds to the ChatGPT teen safety measures OpenAI launched in August. Earlier this month, Common Sense Media said ChatGPT for Teens is unsafe for under-18s. Eight US states have sued OpenAI over harm to children, and Texas has sued the company and Sam Altman.

With expertise in digital marketing, product management, and branding & identity, Ana Maria Constantin develops strategies that resonate (show all) With expertise in digital marketing, product management, and branding & identity, Ana Maria Constantin develops strategies that resonate with our target audience in the software/SaaS industry. Collaboration and teamwork are paramount to her, as she loves empowering her colleagues to achieve outstanding results and unlock their full potential.

Get the most important tech news in your inbox each week.

Thanks! Check your inbox to confirm your subscription.

By subscribing you agree to our Privacy Statement.