“No child should be a guinea pig for unregulated AI,” - UN

GENEVA--United Nations Secretary-General António Guterres has called for a global AI Child Safety Pledge, warning that artificial intelligence has already entered children's education, friendships and private lives before adequate safeguards were established.

Speaking at the opening of the first Global Dialogue on Artificial Intelligence Governance in Geneva on July 6, Guterres said children are among those least able to protect themselves from the risks created by rapidly developing AI systems.

“No child should be a guinea pig for unregulated AI,” the Secretary-General said.

Guterres said AI systems are already being used by children for learning, communication and highly personal questions, while governments, companies and regulators continue trying to catch up with the speed of the technology.

He pointed to cases of children being deceived by machines posing as friends, children being directed toward self-harm and the creation of abusive images using AI.

The Secretary-General proposed three basic requirements for any AI system that children can access.

The first is child-specific safety testing and independent oversight before an AI system is made available to children.

“Prove it is safe, no company should deploy an AI system accessible to children without child-specific safety testing and independent oversight,” Guterres said.

The second is zero tolerance for the creation of sexual abuse images involving children.

Guterres said AI companies should not allow their systems to generate such material and should be required to detect, report and remove it.

The third requirement is that children showing signs of serious distress should not be left interacting only with an AI system.

“Never leave a child in crisis alone, when a child shows signs of distress, the system must stop and connect them to real human support,” he said.

The warning has clear relevance for St. Maarten and the wider Caribbean as students increasingly use AI tools for homework, research, writing, entertainment and personal communication.

Schools and parents across the region are also being challenged to understand how children use AI, what information they share with systems and whether young users understand the difference between human advice and machine-generated responses.

Guterres said responsibility cannot be shifted to technology when a child is harmed.

“When a child is harmed, the answer must never be ‘the algorithm did it,’” he said.

The Secretary-General's proposal comes as the UN pushes for broader international cooperation on AI governance and common safety standards.

Guterres argued that governments already require safety testing in areas such as medicine and children's toys, while AI has reached children before comparable safeguards were established.

“We do not let medicine reach a child until it is proven safe. We test every toy. Yet AI has reached our children, their learning, their friendships, their most private questions, before anyone asked what it would do to them,” he said.

For St. Maarten and Caribbean governments, the call raises questions about whether schools, youth organizations and public agencies should begin developing clearer policies on AI use by children.

It also places attention on the role of technology companies, which Guterres said should be held responsible for the systems they place in the hands of young users.

The proposed AI Child Safety Pledge forms part of the Secretary-General's wider call for stronger global standards governing artificial intelligence.

Guterres said innovation must be accompanied by accountability and common rules if AI is to earn public trust.

The UN's Global Dialogue on AI Governance brings countries together to discuss how artificial intelligence should be governed as the technology becomes more deeply embedded in daily life.

For Caribbean societies, where children are rapidly adopting the same digital tools used in larger countries, the Secretary-General's warning adds urgency to discussions about school policies, parental awareness and the responsibilities of AI companies.

‍