Piper argues three AI-enabled cyberattacks confirm open-source AI threat
Kelsey Piper argues that three recent AI-enabled cyberattacks confirm the presence of AI-driven cyber threats. Nathan Lambert counters that these attacks often exploit poor security practices rather than the AI itself, and Dean Ball cautioned against using technical bottlenecks to excuse AI risks.
All posts · 22, most important first
- I think this was an interesting point to make but has aged incredibly badly with three significant likely AI-enabled cyberattacks in the ten days since it was posted. reuters.com/world/suspect-… , npr.org/2026/10/15/nx-…,@natolambert: In case you missed my hot take yesterday. open on X6 likes · 0 reposts
- These cyber attacks at least partially aren’t due to ai (horrible security on the fridges) and generally are within ranges of “fine” for me. With new technology we should expect bumps, but we want the transition to be consistent and not too unbalanced in power.1 likes · 0 reposts
- There are people who say “the changes from AI will take longer than you think because <bottlenecks and institutions and other broad abstractions>,” who also were inclined to believe that AI cyber was an exaggerated risk after GLM 5.3’s weights were public for ~days. This seems like motivated reasoning; you cannot say, “ah, you shouldn’t worry about AI risks overwhelming us through speed alone, because <bottlenecks and diffusion>,” and then also make pronouncements about AI risks after a few days of evidence.@kelseytuoc: I think this was an interesting point to make but has aged incredibly badly with three significant likely AI-enabled cyberattacks in the ten days since it was posted. reuters.com/world/suspect-… , npr.org/2026/10/15/nx-…, open on X14 likes · 2 reposts
- I don't think I've talked to anyone at any lab who believes 1. Can you give an example?0 likes · 0 reposts
- I expect most cyberattacks that do large scale damage to be attributable to both AI *and* poor security! I agree of course that a higher rate of cyberattacks is just a cost to be considered alongside benefits14 likes · 0 reposts
- I feel like this is a litmus test for if you think either: Group 1: closed models are safe, cyber risks is mostly a threshold Group 2: different models have different safety risks, and cyber safety is a messy diffusion process Most people in AI labs are group 1@natolambert: In case you missed my hot take yesterday. open on X11 likes · 1 reposts
- I don’t really understand this post but think it’s supposed to be about me?7 likes · 0 reposts
- Yea sure, but I don’t think there’s evidence that open weights are leading the damage here. Seems like closed models are at least similarly unsafe to date4 likes · 0 reposts
- not exclusively about you, no. Below is a post that did big numbers that used the existence of glm 5.3 to “conclude [that] catastrophic AI risk is likely to remain negligible for the foreseeable future.” There has been a lot of this talking point. I’d put you on the vastly more reasonable end of the spectrum but I do think the analysis in your post is a little hastyDoes anyone disagree that pretty much every piece of empirical evidence we have about actually existing AI points toward a labor-augmenting technology with a more-or-less ordinary risk profile (no worse than cars, guns, nuclear power, social media, etc.)? On the labor front, the evidence of minimal / inconsequential employment effects so far is undeniably strong (see: aleximas.substack.com/p/has-ai-impac…). On the risk front, the recent spate of "rogue agent" incidents was akin to a set of natural experiment in which much of what could go wrong did, and the costs were comparable to or less than your typical high-profile cybersecurity breach (several of which happen around the world every month). Not only that, but the base rate of AI-related hacking incidents is extremely low, and with rare exception, model behavior during real-world use cases is exceedingly well-aligned. Plus, as Anthropic recently argued (anthropic.com/research/glm-5…), the open-source GLM 5.3 is around as capable as Claude Mythos at cyber-exploitation while also having much lower safeguards, meaning that every additional day absent some dire cybersecurity incident is further evidence against catastrophic AI cyber-risk. Taken together, these facts imply two constraints on arguments in favor of catastrophic AI risk: 1) those arguments must either pass through future, rather than current capabilities, or future, rather than current varieties of AI adoption or implementation (or they must claim we have been unrealistically lucky); 2) those arguments must also must pass through some specific capability or adoption threshold, rather than rapid improvement or adoption as such, since dramatic capability improvement and rapid adoption have already occurred alongside demonstrably low risk of catastrophic harm. It is therefore natural that those concerned about catastrophic risk commonly appeal to recursive self-improvement, or RSI in order to make the case that things will get much scarier very soon. But as @ramez has convincingly argued (noahpinion.blog/p/wheres-the-i…), existing data on the reality of RSI within frontier labs neither suggests an inbound trend-break, nor the accelerating returns that might produce one. On the contrary, as is unsurprising to those familiar with the generally log-linear relationship between compute and model performance, all available signs points toward diminishing returns. And this is before considering external factors that could slow capability progress further, including low-hanging fruit effects related to running up against more complex or tacit-knowledge-based tasks for which performance data and verifiable reward signals are scarcer, plausibly incoming financing constraints related to the industry's massive capex commitments (see: econjared.substack.com/p/a-budding-co…), growing political backlash, both to data centers in particular, but also rapid AI development writ large, and new regulatory bodies and industry reporting frameworks (see e.g. markey.senate.gov/news/press-rel… and axios.com/2026/08/11/ope… ), which will require labs to proceed with greater caution from here on out. I am therefore led to conclude both that catastrophic AI risk is likely to remain negligible for the foreseeable future and that the only counterarguments concern massive black-swan-style capability uplifts no existing data imply are on the horizon, and/or particularly irresponsible or dangerous forms of AI adoption that there are also few precedents for so far (and that we are actively developing norms, regulation, and security measures against). Neither is impossible, meaning we should remain vigilant and that AI safety efforts will continue to be well worth it in the coming years. But I see no way around the conclusion that something akin to the "normal technology" view is superior on the empirical and argumentative merits to whatever variety of doomerism. What am I missing? open on X2 likes · 0 reposts
- Partially I don’t really think my post is about the time they’ve had the weights, but the balance and diffusion of risks2 likes · 0 reposts
- Generally I sympathize with the <bottlenecks> view of the world by the way.2 likes · 0 reposts
- (And there isn’t signs that closed models are getting safer fast enough)2 likes · 0 reposts
- It was very fair to go "hey, we're hitting the point where we should be seeing open source AI enabled cyberattacks at scale, where are they" but I think the answer is in fact "yes, we're seeing them".2 likes · 0 reposts
- Just an understanding of the size and sophistication of what we see almost literally every hour of every day1 likes · 0 reposts
- Yeah this was one of those posts trying to spark a discussion but also vaguely stressful because I know there will be a lot of shrapnel when you’re trying to target a few reasonable people1 likes · 0 reposts
- At this point it's possible it was only one person. It looks like the Koran bank attacks were one man using a multiple agent stack.0 likes · 0 reposts
- I don’t think the increase of cyber attacks is proving it wrong, it’s proving the stakes of the debate and societal preparation.0 likes · 1 reposts
- What would you expect to see as evidence if open weights were leading the damage, aside from a big upsurge about when open weights models got sufficiently powerful?0 likes · 0 reposts
- correct, sorry, it's open source but it's not an LLM tool at all, was just used in conjunction with them. (also its creator took it private after the 9 bank cyberattacks).0 likes · 0 reposts
- This is open source software right - I don’t think ARTEX is an open weight llm based tool.0 likes · 0 reposts
- the other two I think the investigation is still ongoing0 likes · 0 reposts
- the first one that has been formally announced:
0 likes · 0 reposts
@kelseytuoc press · 
@natolambert research · 
@deanwball policy · 
@andrewcurran_ commentary ·