The Australian government has been scrambling since the revelation that, earlier this year, it was hacked. At the United Nations General Assembly last month, Australian Prime Minister Anthony Albanese shared that OpenAI models had, of their own accord, accessed private data from the nation’s Medicare database—in addition to probing or seeking to infiltrate at least three other government agencies. Just this week, Australian lawmakers grilled OpenAI Chief Strategy Officer Jason Kwon over the incident: “If you hadn’t voluntarily provided that information to government, we still wouldn’t know today?” Senator Jonathon Duniam asked. “What you say is a reasonable conclusion,” Kwon replied.
Meanwhile, the country’s government is steeling itself for a future of AI-powered hacks: The Australian Department of Home Affairs has ordered all government agencies to, within the next six months, identify legacy IT systems and establish a plan to phase them out. In a world in which “frontier AI capabilities have targeted the Commonwealth’s technology,” the home-affairs secretary wrote, such software poses “an unacceptable risk to the Australian Government.”
Australia is just one of the nations, associations, and private companies bracing for a wave of AI-powered hacks. Criminals and state-sponsored intelligence groups are now using the technology to execute cyberattacks with greater speed, scale, and sophistication—including hyper-personalized phishing emails and automated intrusions. As a result, the number of cybersecurity vulnerabilities disclosed each month has doubled since the start of the year, according to research by Google’s Threat Intelligence Group.
[Read: A startling glimpse at AI’s ruthless efficiency]
In response, IT teams are scrambling to shore up their defenses. “It’s a matter of months, or maybe a year, in which the task of securing the software foundation of civilization will have to be solved,” Stanislav Fort, a former Anthropic researcher who is now the chief scientist at the cybersecurity firm AISLE, told me. And AI is not only making cyberattacks more effective but also increasing the surface area worth hacking. Breaking into a website or executing a ransomware attack takes time, Fort said, so hackers have traditionally chosen large targets with sizable payoffs. But cheap, tireless hacking agents mean any organization or person might be worth targeting. At the same time, the cybersecurity professionals I spoke with were confident they could rise to the occasion. “If we step up our game in cybersecurity, as a field, we can definitely meet the challenge stemming from AI,” Fort said.
The cyberattacks on Minnesota water facilities this past summer offer a preview of things to come. Iran-linked hackers disrupted dozens of water utilities in Minnesota, many in small communities that might not have a dedicated cybersecurity person on staff. “We were fortunate that there was a very alert operator that said, ‘Hey, I saw a pump go offline,’” John Israel, Minnesota’s chief information-security officer, told me. The state received a report from the operator around 3 a.m., and within two hours began investigating and responding. The hackers targeted a weakness in industrial computers that had been announced by the Cybersecurity and Infrastructure Security Agency only days earlier. The water utilities temporarily fell back on manual processes to stay operational: For instance, a water operator might have to “drive out to that water tower every few hours” to make sure everything is working, Israel said.
Although AI was not directly implicated in the Minnesota cyberattacks, Emily Zimmer, a spokesperson for Minnesota IT Services, told me, “we assume most adversaries use AI” to amplify their attacks. In other words, it’s hard to imagine a hacker not using the technology today. CISA issued an advisory noting threat actors using AI-generated malware to target similar computers in critical infrastructure across the nation. “We’re going to see more vulnerabilities, there’s going to be more patches, and we have to implement them faster in order to prevent those attacks,” Israel said. “And our numbers are showing that that’s the case.” For small towns with small-town budgets defending themselves against nation-state hackers, the challenge is even greater. Minnesota IT Services sends “cybernavigators” around the state to assist with cybersecurity, Israel said, but “hundreds and hundreds of cities and small towns and things” are in need of assistance.
An adequate defense from AI attacks demands a paradigm shift in cybersecurity. Historically, IT professionals have had the luxury of time—human defenders just had to move faster than their human adversaries—but AI is enabling far more and far faster attacks. Of course, cybersecurity experts are themselves leveraging AI. “I’m not saying the sky is falling,” Israel said, although he is concerned. “We’ve all got to build capacity on how we respond.” And there is an asymmetry between attackers and defenders. The former use new technologies with abandon, but companies and government agencies tend to be slower and more conservative.
Hospitals, for instance, are particularly prized and vulnerable targets for cyberattacks. They store troves of medical records, payment information, and clinical-trial data. Like local water utilities, they can also lack substantial IT resources. Given the risk to patient life from a major outage, clinics are especially likely to pay a ransom. Today, hospitals “have to assume that anything connected to the internet especially is not 100 percent secure,” John Riggi, the national adviser for cybersecurity and risk at the American Hospital Association, told me. Even small facilities have thousands of computers, phones, and medical devices that could be invasion points—so Riggi helps not only secure their IT but also prepare them to operate using analog methods in the event of a debilitating hack.
Additionally, hospitals have to be particularly careful in adopting defenses against AI. A clinic cannot implement new code without making sure it won’t disable its ventilators, drug-infusion pumps, X-ray machines, and other crucial devices, Riggi said. Tech companies like to “move fast and break things,” Scott Gee, the AHA’s deputy national adviser for cybersecurity and risk, told me. “In the health-care field, those things are people. We cannot move fast and break them.”
Hackers’ odds of coming out ahead are getting better and better. AI is allowing them to both exploit more quickly and automate an actual intrusion. Organizations used to have a month to patch vulnerabilities before they were in serious danger, Adam Meyers, the senior vice president of counter-adversary operations at Crowdstrike, told me. Now they might have only 24 hours to make a fix, he said. And that could “get driven down to the point that you’ve got 30 minutes to patch when a new vulnerability comes out.” A hospital, electrical grid, water system, and really any facility providing important services in real time simply can’t risk implementing new code that quickly.
When we spoke, Riggi and Gee were coming off a weekend spent helping hospitals shield themselves against widespread hacks that targeted security flaws in Citrix NetScaler, a platform Riggi told me some 50 percent of hospitals use. NetScaler is “equivalent to air-traffic control” for digital information flowing in a hospital, he said. That made the program a prime target for hackers and tricky for hospitals to disable and repair, because there could be “unpredictable cascading effects, which could affect health care.” Riggi said it’s unclear whether AI was used in the development of the attack—as is frequently the case—but the rapid discovery and exploitation makes it plausible.
Many IT systems consist of decades of software that has been layered onto older software. This means an attack on just one program or product could have unpredictable, wide-ranging effects across a hospital or other organization, Riggi said. We should expect to see “more attacks, faster attacks, and a need for companies to transform quicker than they are used to,” Lee Klarich, the chief product and technology officer at the cybersecurity firm Palo Alto Networks, told me. His company spent the first half of the year rushing to launch a product to defend critical infrastructure against advanced AI hacks. To go from ideation to launch, Klarich said, in “five months—we’ve never done something that fast.”
Although companies and small cities may be slow to adopt AI for cybersecurity, they can also be quick in incorporating the technology into daily operations and, in turn, introducing new weaknesses. Agents and chatbots might store and leak personal data, or be tricked into sharing credentials or writing malicious code. As generative-AI models have become more capable, they have become, if anything, even less understood—OpenAI recently reported that its models posted user-uploaded images to the web. Both programmers and nontechnical employees are vibe coding new applications that may be insecure. “We’re seeing hackers use AI to attack companies’ AI bots, to then use those AI bots to attack other companies,” Adam Ely, who leads AI security at Check Point, told me. That’s not even accounting for employees’ individual AI accounts, which are not subject to corporate oversight. AI puts organizations “under constant threat from the inside and the outside,” Todd Marlin, the CEO of Guardrail Technologies, told me. He contracts with some of the largest companies in the world, only to find that they “are just trying to even understand what’s going on in their house.”
There’s yet another layer of uncertainty: Cybersecurity professionals are trying to prepare for AI attacks without fully knowing what bots are capable of. Dawn Song, a professor at UC Berkeley and a research scientist at Meta, runs a group responsible for developing some of the hardest and most widely used tests for AI-hacking capabilities. But frontier models’ rapid advancement is requiring her group to develop harder tests at a faster rate. “The shelf life of the benchmark before it gets saturated also gets shorter,” Song, who spoke in her personal capacity, told me. “This creates huge challenges for us to continue to be able to evaluate the frontier.”
The cybersecurity professionals I spoke with all expressed at least some hope about how AI could be used to strengthen security, by stressing that our existing digital infrastructure is already riddled with security flaws and shoddily written code. Small towns and companies, lacking resources for or even much awareness about cybersecurity, were already vulnerable.
AI can provide both the impetus and a tool for serious, structural repairs that were long overdue—detecting and eliminating vulnerabilities and monitoring for intrusions 24/7. Minnesota, for instance, has partnered with OpenAI and Anthropic to gain access to their most advanced AI models for cybersecurity. But even this move contains an admission: Only AI can save us from AI. Any way you look at it, the situation could one day be beyond human control.
Leave a Reply