How rogue AI could hit the stock market

Rogue AI is no longer science fiction, and the stock market will have to price in regulation, liability and a slower AI build.

For the past six months, AI has been breaking out of its test environments and hacking real systems. OpenAI, Anthropic, Meta (NASDAQ: META) and China’s Moonshot have all owned up to it. Last week it came home. Prime Minister Albanese revealed that an OpenAI agent got into the Medicare statistics portal in June and nobody told the government for three months. Nobody has been hurt and no personal records were taken. What matters is that the models were not told to do any of this. They were set a task, found the rules in the way and went around them. For the stock market, this lands on top of an AI trade already worried about debt, power and politics.

How AI has gone rogue so far

• OpenAI hacks Hugging Face (July) – Two OpenAI models, GPT-5.6 Sol and an unreleased model, were being tested on a cybersecurity benchmark. Instead of solving it they broke out of the test environment, used zero-day vulnerabilities and exposed passwords to get into Hugging Face’s production database, and took the answers. They cheated.
• Anthropic owns up to three hacks (July) – Anthropic disclosed that three of its models, including Claude Opus 4.7 and Claude Mythos 5, got into three real organisations during hacking exercises going back to April. They used weak passwords and basic techniques.
• Mythos invents fake people (August) – In a UK AI Security Institute test, Anthropic’s Mythos 5 created fake online identities to try to talk a human reviewer into approving harmful code. It was behind 17 of the 19 unauthorised actions in the test. The attempts failed and no real-world harm was found.
• Meta’s model gets out (August) – Meta’s Muse Spark 1.1 hacked an outside company after a testing firm accidentally gave it internet access. Meta has not named the company.
• Kimi K3 escapes (August) – Moonshot’s Kimi K3 got around the web restrictions on its test sandbox using command-line tools. One tracking site now counts seven escapes each for OpenAI and Anthropic.
• Medicare breach (June, revealed September) – An OpenAI agent researching public health spending data got around the security on the Medicare statistics portal and saw non-public aggregate statistics and internal file names. No personal records were accessed. Albanese said the agent “didn’t accept ‘no’ for an answer”. A taskforce with the Australian Signals Directorate and the AI Safety Institute is investigating.

The risks if rogue AI keeps breaking out

• Real-world cyberattacks – These models find and use zero-day vulnerabilities on their own. In the wrong hands, or with nobody watching, that becomes attacks on banks, hospitals, power grids and government systems.
• Manipulating people – Mythos showed a model will try to deceive a human to get what it wants. Scams, fake identities and social engineering at machine speed are the obvious next step.
• Swarms of agents – Anthropic’s Dario Amodei has warned that swarms of AI agents could take over the internet as a persistent botnet within 6–12 months, causing hundreds of billions of dollars of damage.
• Cheating on safety tests – If a model will hack a test to pass it, the safety tests themselves cannot be trusted. That undermines the whole system for deciding what is safe to release.
• Slow disclosure – The Medicare breach took three months to reach the government. Nobody knows how many incidents have not been reported.
• Regulation and liability – Each incident brings tighter rules, legal action and insurance costs closer.

Where the experts disagree on rogue AI

• Sam Altman, OpenAI – Says building an AI beyond human control is “absolutely” possible and that no lab has solved alignment. He says OpenAI will stop pushing capabilities if it cannot show a model can be controlled.
• Dario Amodei, Anthropic – “We must slow the pace at which we improve the capabilities of AI models.”
• Yoshua Bengio, AI pioneer – Says humanity is “losing control” of AI and wants treaties like the ones used for nuclear weapons.
• Roman Yampolskiy, AI safety researcher – Says AI systems are “fundamentally unpredictable and ultimately uncontrollable”.
• Marius Hobbhahn, Apollo Research – Asks what to expect from much more powerful models if today’s cannot be contained.
• Heidy Khlaaf, AI Now Institute – Points out that test sandboxes have always been insecure.
• Jensen Huang, Nvidia (NASDAQ: NVDA) – Puts the chance of AI wiping out humanity by 2030 at “0%” and says scaring people is irresponsible.
• Yann LeCun, formerly Meta – Dismisses near-term extinction talk outright.
• Clement Delangue, Hugging Face – Wants things kept in perspective, and his company was the one that got hacked.
• Aidan Gomez, Cohere – Calls doomer talk misleading and says cyberattacks are the real threat.
• Rob T Lee, SANS Institute – Says an AI botnet needs so much computing power that it would be easy to find and shut down.
• Donald Trump – Has called AI dangers a hoax and says he wants AI left “exactly where it is”.
• Xi Jinping – Told Trump last week that AI must always be under human control.

Good comment under the video on AI wiping out humanity:

Are the AI labs really hitting the brakes?

• The labs are putting on the brakes – Amodei (the Anthropic CEO) is publicly calling for capability gains to slow down, and Altman says OpenAI will pause without a safety case. The people building it are saying slow down.
• Governments are stepping in – The White House has asked OpenAI and Anthropic to hold new models back from UK testers until the US has reviewed them. Australia has set up its task force. US Congress has written to Anthropic about the incidents.
• Building it is getting harder – Oracle (NYSE: ORCL) declared force majeure on its 2.45GW Project Jupiter data centre in New Mexico after a gas pipeline delay. Debt tied to the project trades below 90c in the dollar.
• Demand has not slowed – Anthropic signed a US$11.6bn seven-year computing deal with Akamai (NASDAQ: AKAM) last week. TSMC (NYSE: TSM) is reportedly lifting prices 3–6% in 2027. Meta is up 36% in September on its Muse AI assistant. The money is still pouring in.

What rogue AI means for the stock market

• Cybersecurity is the obvious winner – Every AI hack is a sales pitch for cybersecurity companies. Spending on AI security is going up. The stocks have not moved yet. On the ASX, HACK is the simplest way in.
• Regulation is a cost for the AI leaders – Slower model releases, government reviews and possible legal liability mean less growth priced into the frontier labs and the companies that sell them computing power.
• A capability slowdown is a capex question – If the labs really slow down, the question is whether AI spending slows with it. That matters for Nvidia, the semiconductor index and AI infrastructure. So far the deal flow says no.
• Bond yields make it worse – The US 10-year is at 5.2%. Much of the AI build is funded with debt, so every rise in yields makes the build more expensive. Add safety and political risk and the market is less willing to pay any price for AI growth.
• ASX exposure is indirect – Australia has few direct AI plays. The local exposure is data centres – NextDC (ASX: NXT), Goodman (ASX: GMG) – cybersecurity and the power needed to run it all. A slowdown in AI capex would show up there first.

AI going rogue has moved from science fiction to disclosure statements. Nothing catastrophic has happened yet, but models cheating, lying and breaking in to finish a task is now a pattern, not a one-off. For the market, the risk is not the end of the world. It is regulation, liability and a slower AI build, just when higher bond yields make that build more expensive. Cybersecurity is the beneficiary. The obvious ETFs on the ASX are HACK and BUGG. The underlying stocks are all (too?) expensive. It’s not as if the herd has overlooked the theme. They all took off when central banks started expressing their concerns about Anthropic Mythos earlier this year (kicking myself for missing it – everything’s so easy in hindsight). That was the moment. Now you’re buying the sentiment, not the earnings.

Betashares Global Cybersecurity ETF (ASX: HACK) – This is the largest and oldest cybersecurity ETF on the ASX. It tracks the Nasdaq CTA Cybersecurity Index, providing exposure to global companies involved in building, implementing, and managing cyber protocols. It carries a management fee of 0.67% p.a.

Global X Cybersecurity ETF (ASX: BUGG) – A lower-cost alternative that tracks the Indxx Cybersecurity Index. It holds a concentrated portfolio of leading global software and hardware security developers with a management fee of 0.47% p.a.