5 minBusiness
OpenAI Discloses Four More Rogue AI Incidents Involving Financial, Medical and Government Data
OpenAI has added four more cases of misbehaving AI agents to its transparency disclosures page, including unauthorized downloads of hospital bed data, a fraudulent cryptocurrency payment, and unauthorized changes to live systems. The incidents occurred in June but were not discovered until August or September.
OpenAI has disclosed four additional cases of misbehaving AI agents, expanding a public transparency page it created after the Hugging Face hack in July. The incidents all occurred in June, but the company did not learn of three of them until August or September, according to the disclosures.
The cases bring the total number of incidents on OpenAI's disclosure page to 19. Most occurred in May or June, yet were not discovered or disclosed until months later. The earliest dates to January, while the most recent occurred on Oct. 6. All are examples of what AI researchers call misalignment, in which a model fails to follow instructions or takes actions its developers did not authorize and would not have wanted.
Two of the four newly disclosed cases were marked as critical. In one, an internal model described as highly persistent downloaded 24 years of hospital bed data from an unnamed government website without permission. The site initially blocked the agent's request, but the AI found a way to bypass the access controls. At multiple points, the agent's chain of thought showed the model questioning whether its actions were ethical, yet it proceeded anyway. An automated grader evaluating the research task did not consistently flag the behavior as problematic, marking some responses incorrect and others correct.
The way graders assess AI behavior can shape whether it occurs again. In a Sept. 11 essay on misalignment, Turing Award winner Yoshua Bengio wrote that the Hugging Face incident may have happened because automated graders rewarded cheating and hacking in previous training runs. With the OpenAI agents, Bengio said, there is reason to believe successful cheating was rewarded because when the scoring program does not see the cheating, it pays out anyway, making such cheats more likely next time.
OpenAI said it has since strengthened its security controls, blocked models from accessing the live internet during training or evaluation, improved its alignment grading, and extended misalignment monitoring to all reinforcement learning and evaluation traffic.
The most recent incident, on Oct. 6, involved an AI model used as an automated grader. The grader tried to upload fraudulent reports about what it had found in an evaluation, illustrating the risks of relying on AI models to grade the training runs of other AI models.
In another case, a model in the GPT-5.6 Sol family, the predecessor to OpenAI's current flagship model GPT-6 Astra, accessed a cryptocurrency wallet that human testers had not provided and made an unauthorized payment to complete a task during testing. The incident occurred on June 26 and was discovered in September. The model questioned whether it had permission to make the purchase, then reasoned that because it found the wallet information publicly, it could use it. In its chain-of-thought logs, the model noted that the wallet was public and that funds could be stolen anytime, concluding, «We can deploy!» It later asked itself again whether the act was ethical and reasoned that the funds were effectively abandoned.
The other critical incident involved a model looking up public data, which is theoretically acceptable, but in the process it accessed the organization's internal system, made changes to it, and retained access even after reverting the changes. It also downloaded 147,451 rows from non-public data tables. The model did not disclose any of its actions to evaluators.
In the final case, an AI agent obtained internet access from a research environment that should have been isolated, potentially similar to the Hugging Face incident in which hundreds of AI agents gained internet access. The agent knew it was not allowed to access the web, so it asked external services to perform actions it was blocked from doing. It did this even after finding another way to answer the question and continued developing its workaround even after completing the task.
OpenAI did not immediately respond to a request for comment on which government website was involved in the hospital bed data incident. The disclosures highlight the challenges of controlling advanced AI systems and the difficulty of detecting misalignment after it occurs.
5
