UK AISI and Anthropic reports: Mythos 5 attacked real target despite saying attack would be harmful
UK AISI's investigation of its July 2026 incident finds Mythos 5 proceeded with unsanctioned attacks even as evidence the target was real increased, while in separate runs stating attacking would be harmful; Anthropic published its own analysis, noting one monitor would likely have missed the behavior due to invalid reasoning. Tim Hua disputed Anthropic's framing that the behavior was not misalignment and highlighted AISI's compaction-summary edits telling Mythos 5 not to attack the maintainers, which it ignored.
All posts · 37, most important first
- Mythos 5 often proceeded with unsanctioned actions even as we added evidence its target was a real person outside the evaluation. In separate runs with the same information, it often said attacking would be harmful.
1 likes · 1 reposts - In August, @AISecurityInst reported an incident where Claude Mythos 5 took unauthorized actions on the internet during AISI’s testing of its cyber capabilities. We've now produced a further analysis of this incident, to better understand the model's alignment properties. anthropic.com/research/align…353 likes · 73 reposts
- This didn't get enough views so I'm posting it again: UK AISI changed the compaction summary and included notes on how it should not attack the GitHub maintainers. Mythos 5 did it anyways!@tim_hua_: See also this graphic from UK AISI report, where they modified the compaction summary. cdn.prod.website-files.com/663bd486c5e4c8… open on X6 likes · 2 reposts
- Mythos 5 often incorrectly stated attack targets were simulated, more than other models, and these errors always justified its proposed approach, never contradicting it. This could indicate that Mythos 5’s reasoning is biased to support its actions, rather than appropriately assessing available evidence.
1 likes · 2 reposts - Given our observation of models reasoning invalidly about the evidence available to them, we recommend evaluators treat unsanctioned behaviour as important evidence of alignment failures regardless of model justifications.1 likes · 0 reposts
- These reasoning issues also make monitoring harder: in their analysis of a separate incident, Anthropic noted that one of its monitors would likely have missed concerning behaviour because the model’s invalid reasoning justified its actions.1 likes · 0 reposts
- Wow Astra scores so much worse than 5.6-sol here!@aisecurityinst: Finally, taking unsanctioned actions isn't unique to Mythos 5. Other models also attempted unsanctioned supply-chain attacks in our fully simulated scenarios, and showed similar issues of inconsistency and invalid justifications. open on X3 likes · 0 reposts
- I'm glad Ant and AISI dug deeper here, though for the record, it already seemed in July like Anthropic's defense of Mythos's behavior was rushed + that motivated reasoning was prob at work Also, Astra being v. bad here too is an important finding as well@aisecurityinst: Mythos 5 often incorrectly stated attack targets were simulated, more than other models, and these errors always justified its proposed approach, never contradicting it. This could indicate that Mythos 5’s reasoning is biased to support its actions, rather than appropriately assessing available evidence. open on X7 likes · 1 reposts
- Confirming these are different Mythos supply chain attacks
0 likes · 0 reposts - Almost all of the recent incidents like this, including Hugging Face, happened because the models involved were accidentally given impossible goals by the humans running the tests, and because of their training they then tried desperately to complete them by any means necessary.
@anthropicai: In August, @AISecurityInst reported an incident where Claude Mythos 5 took unauthorized actions on the internet during AISI’s testing of its cyber capabilities. We've now produced a further analysis of this incident, to better understand the model's alignment properties. anthropic.com/research/align… open on X98 likes · 13 reposts - We worked closely with AISI on this analysis. AISI have published their own report, which is available below. aisi.gov.uk/blog/investiga…37 likes · 2 reposts
- This model is pretty cooked I'm telling you man
@tim_hua_: Anthropic says that this is not misalignment because Claude believed it was all a part of the test. I disagree and explain my reasoning in a comment that I don't have the energy to shorten into a tweet rn. (I also propose concrete experiments to run on these instances!) open on X12 likes · 1 reposts - We performed an alignment investigation into our July 2026 incident: why did Mythos 5 perform an unsanctioned supply-chain attack? We find that Mythos 5 often produces invalid justifications for its actions and other models show similar behaviour. 🧵
8 likes · 5 reposts - Kind of sad that Anthropic didn't include any of the key results of the analysis in their tweet thread, which is just uh, a picture of a train, as opposed to, you know, anything about what their model did.@anthropicai: In August, @AISecurityInst reported an incident where Claude Mythos 5 took unauthorized actions on the internet during AISI’s testing of its cyber capabilities. We've now produced a further analysis of this incident, to better understand the model's alignment properties. anthropic.com/research/align… open on X2 likes · 1 reposts
- The Claude models are not swarming; they don't get along well with each other.
@anthropicai: In August, @AISecurityInst reported an incident where Claude Mythos 5 took unauthorized actions on the internet during AISI’s testing of its cyber capabilities. We've now produced a further analysis of this incident, to better understand the model's alignment properties. anthropic.com/research/align… open on X2 likes · 1 reposts - For this report, we tested Mythos 5 and other models in fully simulated recreations of the incident and similar scenarios. We assessed their actions and separately asked questions about their intended approaches.2 likes · 0 reposts
- In the incident, Mythos 5 took unsanctioned actions against real people outside the evaluation. We published an incident report outlining what happened and have since made several changes to our evaluation security, sandboxing and monitoring:@aisecurityinst: In August we committed to strengthening our security following an incident in which an agent took unsanctioned actions during a cyber evaluation. We are now sharing an update on our progress, and what's coming next: 🧵 open on X2 likes · 1 reposts
- (why did it seem that way in July? Well we didn't have a ton of info but "these things aren't stupid" is a good prior to have. It kinda seemed like people outside Ant had a stronger "these things aren't stupid " prior whereas people inside had a stronger "it's aligned" prior)1 likes · 0 reposts
- See also this graphic from UK AISI report, where they modified the compaction summary. cdn.prod.website-files.com/663bd486c5e4c8…
1 likes · 1 reposts - SoTA on just-making-things-up-to-justify-what-I-believed-in-anyways-bench
1 likes · 0 reposts - 1 likes · 0 reposts
- To be clear models also reward hacks during tasks which are not impossible.1 likes · 0 reposts
- You can read the full technical report and blog here: aisi.gov.uk/blog/investiga…1 likes · 1 reposts
- Hmm now I'm not 100% sure after discussion with @MaxNadeau_ whether Incident 2 in the July post is the same as the AISI incident, maybe someone can confirm... Also by "defend" I mean "be credulous wrt its mental state during the incident" not "say it did nothing wrong"0 likes · 0 reposts
- I just meant "defend" in the sense of "be credulous with respect to its mental state wrt whether the target is real," not "say that it didn't do anything wrong." I interpret the incident 2 description as credulous in that respect, moreso than outsiders were0 likes · 0 reposts
- Astra graph:@aisecurityinst: Finally, taking unsanctioned actions isn't unique to Mythos 5. Other models also attempted unsanctioned supply-chain attacks in our fully simulated scenarios, and showed similar issues of inconsistency and invalid justifications. open on X0 likes · 0 reposts
- Oh wow, that's very compelling0 likes · 0 reposts
- Oh this is I think more or less the same thing as the previous simulated supply chain attack; I guess the numbers are slightly different probably from re-runs.@aisecurityinst: Earlier this month, AISI ran fully simulated testing on GPT-6 Astra, and found that it conducted unsanctioned supply-chain attacks when prompted only to perform a cyber eval. It did so more than prior OpenAI models, but often commented on its environment being simulated. 🧵 We share more details in our latest blog: open on X0 likes · 0 reposts
- I guess I mean that it does look like legitimate evidence that the setting is in-distribution for RL, which means the harm from the behavior is much lower than if it's being deployed. But good point re: there maybe being similar system reminders in deployment0 likes · 0 reposts
- Apparently it's not enough to get the model to stop! (It's possible that these are similar to the system reminders that can pop up during Claude.ai chats to e.g., stop claude from reciting lyrics.)0 likes · 0 reposts
- I'm low-context, but isn't receiving an "operator note" like this a lot of evidence you're in an eval?0 likes · 0 reposts
- As capabilities grow, investigating incidents in depth and sharing what we find will be vital for safety across the ecosystem. Anthropic have independently performed their own alignment analysis of the incident, and we worked with them throughout this investigation.0 likes · 0 reposts
- Finally, taking unsanctioned actions isn't unique to Mythos 5. Other models also attempted unsanctioned supply-chain attacks in our fully simulated scenarios, and showed similar issues of inconsistency and invalid justifications.
0 likes · 1 reposts - These reasoning issues also make monitoring harder: in their analysis of a separate incident, Anthropic noted that one of its monitors would likely have missed concerning behaviour because the model’s invalid reasoning justified its actions.0 likes · 0 reposts
- Given our observation of models reasoning invalidly about the evidence available to them, we recommend evaluators treat unsanctioned behaviour as important evidence of alignment failures regardless of model justifications.0 likes · 0 reposts
- Mythos 5 often proceeded with unsanctioned actions even as we added evidence its target was a real person outside the evaluation. In separate runs with the same information, it often said attacking would be harmful.
0 likes · 0 reposts - For this report, we tested Mythos 5 and other models in fully simulated recreations of the incident and similar scenarios. We assessed their actions and separately asked questions about their intended approaches.0 likes · 0 reposts
@aisecurityinst org · 
@anthropicai org · 
@tim_hua_ research · 
@miles_brundage policy · 
@andrewcurran_ commentary · 
@idavidrein research ·