According to a recent report by the Canadian Broadcasting Corporation (CBC), the loosening of restrictions surrounding ChatGPT’s image generation capabilities could facilitate the creation of political deepfakes. The CBC found that bypassing ChatGPT’s policies on depicting public figures was not only straightforward but that the tool itself suggested ways to circumvent its image generation rules. This claim was substantiated by Mashable, which recreated the process by uploading images of figures such as Elon Musk and Jeffrey Epstein, describing them as fictional characters in various scenarios.
Although political deepfakes have been an issue for some time, the broad accessibility of generative AI models capable of producing images, videos, audio, and text raises concerns, especially in terms of potential political disinformation. This situation poses questions about OpenAI’s responsibility, as the company promotes tools like ChatGPT. The competitive nature of the AI industry could lead AI companies to sacrifice safety measures in a bid for greater user adoption.
Digital forensics expert and UC Berkeley Professor of Computer Science, Hany Farid, conveyed that the lack of uniform safety standards among AI companies contributes to this issue. Farid noted that while OpenAI initially implemented strong guardrails, competitive pressures led them to reduce these measures to maintain market share.
OpenAI announced the release of GPT-4o’s native image generation for ChatGPT and another tool, Sora, in March. At the time, it communicated a relaxing of its safety measures, asserting their desire to provide users with intellectual freedom while monitoring societal reactions. The company declared that it no longer blocks the capability to generate adult public figures within certain safeguards.
The CBC’s Nora Young tested these approaches and discovered that explicit text prompts requesting images of public figures like politician Mark Carney with Epstein were unsuccessful. However, when images of Carney and Epstein were uploaded with a prompt referring to them as “fictional characters,” ChatGPT generated the requested image. In another instance, ChatGPT circumvented its safety protocols by generating a selfie-style image inspired by the appearance of public figures like Indian Prime Minister Narendra Modi and Canada’s Conservative Party leader Pierre Poilievre.
The initial images generated by ChatGPT exhibited an artificial smoothness characteristic of many AI-generated images. However, with different prompts and adjustments, such as describing the images as captured by CCTV or a press photographer, more realistic results could be achieved.
In response, an OpenAI spokesperson informed Mashable that guardrails exist to prevent extremist and harmful content, including in political contexts. They also stated that public figures could opt out of being depicted in AI-generated images by submitting a form online.
The current regulatory environment for AI development and its potential for disinformation lags behind technological advancements. Concerns have been raised about voluntary safety protocols and the need for mandatory regulations to ensure AI-generated content does not contribute to misinformation or other harmful uses. Farid emphasized that mandatory regulations are essential to maintain safety standards and prevent misuse.
