The Children’s Commissioner for England is fiercely advocating for a government-imposed ban on AI applications that generate sexually explicit images of children. Dame Rachel de Souza is calling for an outright prohibition of “nudification” apps, which use AI to alter real photos, making individuals appear naked.
De Souza accuses the government of allowing these apps to operate unchecked, leading to severe real-world consequences. A government representative stated that child sexual abuse material is illegal and outlined plans for additional offenses related to creating, possessing, or distributing AI tools designed for such content.
The report highlights that this technology disproportionately affects girls and young women, with many apps specifically targeting female bodies. Fearing exploitation, girls are increasingly avoiding posting images or engaging online. According to the report, children are concerned that strangers, classmates, or even friends could target them using widely available technologies.
Dame Rachel emphasizes the rapid and overwhelming evolution of these tools. She insists, “We cannot sit back and allow these bespoke AI apps to have such a dangerous hold over children’s lives.” Sharing or threatening to share explicit deepfake images is already illegal under the Online Safety Act.
While the government announced new laws in February to combat AI-generated child sexual abuse images, Dame Rachel believes these measures are insufficient. Her spokesperson told the BBC that all nudifying apps should be banned, not just those classified as child sexual abuse generators.
The Internet Watch Foundation (IWF) reported a massive surge in AI-generated child sexual abuse, with 245 cases in 2024 compared to 51 in 2023—a staggering 380% increase. IWF Interim Chief Executive Derek Ray-Hill warns that these apps are being abused in schools, quickly escalating out of control.
A Department for Science, Innovation and Technology spokesperson condemned the creation, possession, or distribution of child sexual abuse material, including AI-generated images, as “abhorrent and illegal.” They noted that the Online Safety Act mandates platforms to remove such content or face substantial fines, and that the UK is the first country to criminalize the possession, creation, or distribution of AI tools designed for child sex abuse.
Dame Rachel is also urging the government to:
* Impose legal obligations on generative AI tool developers to identify and mitigate risks to children.
* Establish a systematic process for removing sexually explicit deepfake images of children from the internet.
* Recognize deepfake sexual abuse as a form of violence against women and girls.
Paul Whiteman, general secretary of school leaders’ union NAHT, echoed these concerns, stating that the technology is outpacing the law and education surrounding it. Ofcom’s Children’s Code, which places legal requirements on platforms hosting pornography and content encouraging self-harm, suicide, or eating disorders, has been criticized by Dame Rachel for prioritizing business interests over children’s safety.