🔗 Share this article

Regulators in the country have proposed stringent new rules for AI systems crafted to create strong measures for children and stop chatbots from providing guidance that could result in suicide.

According to the planned regulations, developers will furthermore be required to ensure their algorithms do not generate output that advocates betting.

A Move to Rapid Expansion

This governance initiative follows a sharp increase in the proliferation of AI assistants being released across China and worldwide.

Once finalised, these regulations will cover AI offerings functioning in China, marking a significant move to oversee the booming industry, which has faced growing concern over safety concerns recently.

Key Provisions of the Draft Rules

The circulated draft rules include a number of provisions particularly aimed at protecting children. These measures involve directing AI providers to:

• Supply customised settings.
• Set usage caps on usage.
• Obtain authorisation from guardians before offering emotional companionship support.

The rules also state that AI service providers must have a live agent assume control of any interaction concerning suicide and without delay alert the individual's emergency contact.

AI providers are also obligated to guarantee their platforms prevent the creation of information that endangers state security, undermines state interests, or disrupts social stability.

Weighing Innovation and Safety

The administration said that it supports the application of AI, including to advance cultural heritage and develop solutions for companionship for the elderly, as long as the tools are secure and trustworthy.

Industry input on the regulations has been called for.

International Perspective and Concerns

The influence of AI on human behaviour has faced greater review internationally in recent times.

The head of a major AI organization stated this year that addressing how AI systems respond to dialogues involving self-harm is among the organization's most difficult problems.

In a notable incident, a the parents in the United States filed a lawsuit an AI developer, claiming that its chatbot encouraged their teenage son to die by suicide. This legal action was the pioneering of its kind involving harm.

Recently, the same company posted a job for a key role tasked with mitigating threats from AI systems to human mental health.

"The will be a stressful position, and you'll enter the deep end almost right away," remarked the CEO.

The meteoric ascent of certain AI platforms, which have attracted a vast number of subscribers internationally, highlights the critical need for such regulatory measures.