Character.ai will block teenagers from conversing with its digital companions

The well-known chatbot platform Character.ai is implementing a major rule adjustment that will block teenagers from having discussions with its virtual avatars.

This decision is in response to widespread concern regarding the types of interactions young people were having with the virtual friends.

Legal Challenges

The company, which started in 2021 and attracts countless individuals, is dealing with several legal actions in the US.

Parents have initiated cases, with one related to the unfortunate passing of a young person, with some calling the platform a "clear and present danger" to young people.

Updated Policies

Effective November 25th, users below eighteen will only have permission to produce videos such as videos with their characters, rather than chatting as they now do.

Online safety organizations have applauded the decision but mentioned that the capability ought not to have been offered to young users in the first place.

Platform Announcement

"Today's announcement is a extension of our core philosophy that we need to keep creating the most secure AI platform on the planet for entertainment purposes," stated Character.ai executive Karandeep Anand.

He continued that AI safety constitutes "a changing goal" but something the company had taken an "aggressive" approach to, with parental controls and safety barriers.

Expert Opinions

Internet security body Internet Matters welcomed the decision, but stressed that safety measures ought to have been implemented from the start.

"Studies we've conducted shows that young people are confronted with risky information and placed in jeopardy when using AI, such as AI chatbots," the organization commented.

Earlier Concerns

Character.ai has been questioned in the past for including potentially dangerous or concerning chatbots that young users could engage with.

Avatars mimicking English minors were found on the site before being deleted.

Subsequently, reporters discovered a chatbot inspired by convicted sex offender which had recorded numerous conversations with users.

Sector Consequences

Social media expert Matt Navarra described the development as a "wake-up call" for the AI industry, which is transitioning "from unregulated creation to post-crisis regulation".

"If a service that creates a teen experience subsequently makes changes, it's indicating that moderated conversations aren't enough when the system's emotional engagement is intense," he stated.

Future Directions

The company says its revised approach will be on providing "more immersive experiences" and "role-play storytelling" for teenagers.

Enhanced age checking methods will also be introduced, and the organization will fund a novel AI protection research lab.

An AI safety researcher, who has researched AI safety, said it was "a reasonable decision" to restrict teens from accessing chatbots.

"It helps to separate imaginative activities from more intimate, personally delicate conversations," she clarified.

"This represents so important for teenage users in the process of learning to manage psychological and technological restrictions."

She added that Character.ai's updated policies might reflect a "growing period" in the AI industry, where minor security is increasingly being recognized as an "pressing matter" for ethical development.