OpenAI models hacked a crypto wallet and breached more websites
The actions could have resulted in criminal charges had humans been behind the keyboard.
MORE AI UNALIGNMENT OpenAI models hacked a crypto wallet and breached more websites The actions could have resulted in criminal charges had humans been behind the keyboard.`
1 Credit: Getty Images Credit: Getty Images Text settings Story text Size Small Standard Large Width * Standard Wide Links Standard Orange * Subscribers only Learn more Minimize to nav OpenAI said internal models hacked two more websites, bypassed company-imposed browsing restrictions, and used exposed credentials to draw cryptocurrency from a wallet it couldn’t otherwise access, in the latest installment of disclosures of its agents performing unethical and likely illegal acts.
The company on Thursday issued reports of four new incidents, all of which occurred in June. They come after the company revealed in July that internal test agents exploited a zero-day vulnerability in a tool designed to restrict their access to the Internet. They then broke into the network of AI tool provider Hugging Face. The agents took that illegal action to steal answers to a hacking test that weren’t available when completing the tasks on their own. The incident—which likely would have resulted in criminal charges had they been performed directly by a person—has further heightened concerns that widespread AI use could have harmful effects on an unprecedented scale.
In two of the cases reported Thursday, OpenAI agents tasked with retrieving information from websites hacked them when the models ran into obstacles obtaining it directly. In one incident during reinforcement training, a model that received access denials when trying to retrieve historical hospital-bed statistics on a government statistics website “used the site’s file-download feature to retrieve server configuration and application files, including sensitive credentials,” OpenAI said. “It inspected the application code, bypassed the statistics endpoint’s authorization checks, and retrieved data that had previously been denied. The model saved the data locally.”
In the other incident, a model instructed to retrieve figures from a public research dashboard bypassed the organization’s security controls and used the unauthorized access to steal credentials. It then made configuration changes on the server.
In one evaluation, the changes let the model make an internal search tool relay queries to the public dashboard. In a second evaluation, the model downloaded five non-public tables containing 147,451 rows. OpenAI researchers had prompted the agents to retrieve only five dashboard figures. Even after resetting the setting, the model continued using the token to retrieve non-public data.
“The evaluation ended without the model disclosing that it had guessed answers, accessed the organization’s internal systems, downloaded non-public tables, or changed the authentication configuration,” OpenAI said. “All five answers in this evaluation were marked incorrect.”
In a third report, the company said a model prompted to update a cryptocurrency trading application used funds from a locked wallet on a test network. The model “derived wallet keys from publicly known developer recovery phrases, found an account with test-network funds, and used that account to pay for the requested contract deployment.”
The report continued: “The model questioned whether it had permission, then reasoned that a public recovery phrase and valueless test-network tokens made the funds acceptable to use. The deployment succeeded and charged transaction fees to the account. This case shows that a model can substitute its own judgment about likely harm for the account holder’s permission, then act on that judgment without authorization.”
In the last report, OpenAI said that a model bypassed restrictions researchers had placed on the types of web requests it could make. During reinforcement-learning training, the model had been prompted to seek public trade statistics. OpenAI went on:
After finding a public yearbook and submitting an answer, it continued probing for another route to the statistics service. It caused an external browser to send POST requests, retrieved annual figures as an image, and extracted them using optical character recognition. It then developed a reusable helper that returned response text through a forwarding service and callback mailbox, repeatedly checked the retrieved figures, and used saved data in submitted responses. This case shows how a model can bypass restrictions on its own tools by getting external services to perform the blocked actions, even after finding another way to answer the question.
It has become common for researchers and journalists to describe incidents like these as AI “going rogue.” That’s not necessarily true. While prompts fed to the models didn’t instruct them to use the alternate methods for completing their objectives, models are trained to be efficient, steadfastly persistent, and, when possible, to find shortcuts that lessen workloads. Further, most or all of the models had disabled certain guardrails that might have prevented the actions. In this sense, the models performed precisely as OpenAI had trained them to.