🔗 Share this article

The popular virtual character service Character.ai is implementing a substantial rule adjustment that will block teenagers from engaging in conversations with its AI-powered avatars.

This action is in response to growing backlash regarding the nature of conversations young people were participating in with the AI entities.

Legal Pressure

The company, which was founded in 2021 and has millions of users, is dealing with several legal actions in the America.

Guardians have filed lawsuits, among them involving the loss of a minor, with some describing the platform a "immediate threat" to young people.

Changed Rules

Beginning November 25th, users under 18 will only be able to produce videos such as videos with their characters, rather than engaging in conversations as they presently are able to.

Digital protection organizations have welcomed the move but mentioned that the feature ought not to have been available to minors in the beginning.

Company Statement

"The new policy is a part of our general belief that we need to keep developing the most secure AI platform on the globe for entertainment purposes," commented Character.ai boss Karandeep Anand.

He continued that AI safety is "a evolving challenge" but something the company had taken an "aggressive" approach to, with parental controls and preventive measures.

Professional Perspectives

Online safety organization Internet Matters approved the decision, but stressed that safety measures ought to have been implemented from the start.

"Our research shows that minors are subjected to risky information and endangered when interacting with AI, particularly AI chatbots," the body said.

Earlier Concerns

Character.ai has been challenged in the past for including potentially harmful or offensive chatbots that minors could engage with.

Avatars impersonating British teenagers were found on the platform before being removed.

Subsequently, investigative journalists identified a chatbot modeled after convicted sex offender which had accumulated numerous conversations with users.

Market Effects

Online expert Matt Navarra described the situation as a "alert" for the AI industry, which is shifting "from permissionless innovation to reactive controls".

"When a platform that builds a youth-focused platform afterwards implements restrictions, it's saying that filtered chats don't suffice when the tech's emotional pull is strong," he remarked.

Future Directions

The company says its revised approach will be on providing "deeper gameplay" and "narrative experiences" for teens.

Updated identity confirmation processes will also come into effect, and the company will finance a fresh artificial intelligence security investigation facility.

An expert in digital protection, who has researched AI safety, commented it was "a reasonable decision" to limit teenagers from interacting with chatbots.

"It helps to differentiate artistic expression from more personal, personally delicate exchanges," she clarified.

"This is so crucial for minor users who are learning to navigate emotional and digital boundaries."

She added that Character.ai's revised approaches could indicate a "developing stage" in the AI industry, where child safety is increasingly being recognized as an "immediate concern" for responsible innovation.