Meta’s “supreme court” has ordered the technology company to remove deepfake videos involving a UK politician and a young Muslim woman from Facebook, while calling for stronger action against AI-generated fake imagery.
The Oversight Board said a fabricated video showing a Labour councillor in Scotland making inflammatory remarks about refugees should not have remained on Facebook. It also ruled that an AI-generated video featuring a Muslim campaign volunteer should have been removed after falsely portraying her giving health advice while performing absurd exercises and eating junk food.
The board said Meta’s safeguards were “consistently and fundamentally inadequate” in dealing with the rapid growth of AI-generated deepfakes.
“From politicians to private citizens, AI-generated deepfakes are increasingly being used to harass and silence women from engaging in public discourse,” said Pamela San Martin, an oversight board co-chair.
“These cases demonstrate a broader, troubling pattern in which women who engage publicly on issues are disproportionately subjected to harassment and misinformation. Meta and other social media platforms need more robust policies to address the proliferation of deepfakes.”
In the Scottish video, the councillor is falsely depicted as saying: “Refugees are welcome here, even if they rape our women, because white people do that too.”
The Oversight Board, a quasi-independent body whose decisions are binding on Meta, said the video appeared to have been generated using AI. It pointed to audio that was not fully synchronised with the unnamed councillor’s facial movements as an indication.
Even after the issue was brought directly to Meta’s attention by the board, the company decided that the video did not breach its content policies and did not require an AI-generated content label.
In a blogpost published on Thursday, the board said the video should have been removed because it breached Meta’s hateful conduct rules by attributing criminal and predatory sexual behaviour to refugees as a group rather than to individuals. It also said the video should have carried a “high risk AI” label and called for stronger protections against deepfakes.
“The majority [of the board] finds that Meta needs more robust policies on deepfakes, including expanding the situations when ‘high risk’ labels can be applied, more measures to reduce the spread of deceptive AI content, increasing the penalties for accounts that repeatedly share it, and more transparency on data around when AI labels are applied,” it said.
The board recommended nine policy changes for Meta. These included ensuring that algorithms reduce the visibility of content marked as “high risk”, making such posts less likely to appear in users’ feeds. It also called for measures that would make AI-generated content more difficult to access, such as warning screens requiring users to click through before viewing it.
In a separate ruling, the board also ordered Meta to remove from Facebook an AI-generated video mocking a young Muslim woman in Europe who was involved in a campaign promoting menstrual health education and reducing stigma around health conditions affecting women and girls from ethnic minority backgrounds. AI-manipulated videos and images targeting the woman had received tens of millions of views online, including across Meta’s platforms.
The board highlighted one of the videos, which showed the woman performing absurd exercises and eating junk food, as a violation of Meta’s bullying and harassment policy. It recommended removing the post and issued three policy recommendations, including changing the definition of “unwanted manipulated imagery” to cover deepfakes showing private individuals saying or doing things they never said or did.
Meta established the Oversight Board in 2020 to act as an independent referee for content decisions across its platforms. Meta has been approached for comment.
