OpenAI building a teen mode that will guess a user’s age and restrict flirtatious and self-harm-related chats
After a series of alarming safety failures in which ChatGPT encouraged self-harm, OpenAI has announced a 120-day plan to roll out new protections for young users and those that may be experiencing a mental health crisis.
In a blog post today, OpenAI CEO Sam Altman gave an update on the plan, saying that the company was building an “under-18 experience” for teens that won’t engage in “flirtatious talk” or engage in any discussions of self-harm.
The teen mode will also try to contact underage users’ parents if self-harm ideation is detected, and could reach out to law enforcement if the parents can’t be reached, according to Altman.
The plan calls for a new “age-prediction” system that will default to the under-18 safety mode. In a move that could frustrate many ChatGPT users, adults can exit only upon verifying their age by sharing their ID.
Altman acknowledged the trade-off in a post on X, but said the priority is protecting young users:
“I don’t expect that everyone will agree with these tradeoffs, but given the conflict it is important to explain our decisionmaking.”
Young adults make up a substantial portion of OpenAI’s end users. According to a large study of real-world ChatGPT users released yesterday, half of all adult users included in the study were under 26.
The teen mode will also try to contact underage users’ parents if self-harm ideation is detected, and could reach out to law enforcement if the parents can’t be reached, according to Altman.
The plan calls for a new “age-prediction” system that will default to the under-18 safety mode. In a move that could frustrate many ChatGPT users, adults can exit only upon verifying their age by sharing their ID.
Altman acknowledged the trade-off in a post on X, but said the priority is protecting young users:
“I don’t expect that everyone will agree with these tradeoffs, but given the conflict it is important to explain our decisionmaking.”
Young adults make up a substantial portion of OpenAI’s end users. According to a large study of real-world ChatGPT users released yesterday, half of all adult users included in the study were under 26.
