OpenAI says it has disrupted a large-scale attempt to attack ChatGPT, with more than 15,000 users involved in the coordinated activity at its peak.
The company said the operation was part of an effort to “distil” the AI model that powers ChatGPT. Distillation is a widely used technique for extracting knowledge and capabilities from specific AI models so that the technology can be adapted or used by other systems.
OpenAI attributed at least part of the activity to Moonshot AI, the Chinese developer behind the Kimi AI system, although it said it could not determine whether the company was responsible for all of the activity. Moonshot AI did not immediately respond to the allegations.
According to OpenAI, the attack began at a low volume at the start of July before increasing significantly throughout the month and evolving over time. By the time the operation was disrupted on 28 July, the company had identified more than 15,000 users who appeared to be connected to the activity.
Distillation can also be used as a legitimate technique in AI research and development. It can help researchers understand how AI models operate while allowing large models to be made smaller and potentially more efficient.
However, private AI companies including OpenAI have increasingly described large-scale model distillation as a potential security concern and have called for greater coordination across the industry. Following the latest disclosure, OpenAI warned that such attacks could create “safety and national security risks” by enabling models to be copied without preserving the safeguards built into the original systems.
AI companies also have commercial reasons to prevent such activity because successful model extraction could give competitors access to aspects of their technology without requiring the extensive training data, computing infrastructure and energy needed to develop comparable systems from scratch. Experts have suggested that this could make it easier for Chinese companies to replicate advances developed by US firms such as OpenAI.
OpenAI said it stopped the attack using a range of security measures, including shutting down accounts suspected of participating in the operation. The company also said it had fixed several bugs that had allowed users to access normally hidden reasoning processes used by its systems to generate responses.
