METR critiques Anthropic's August 2026 Risk Report misalignment section, citing Mythos 5 incidents
METR (Jurkovic, Tim Hua, Beth Barnes) says it agrees Anthropic's misalignment risk is currently low but due to limited model capabilities, not safeguards, and doubts the report's methodology would catch higher risk, citing Mythos 5 incidents and narrow covert-capability benchmarks. Tim Hua highlights a prompting experiment where Claude gave a sycophantic review of the report itself, motivating a recommendation that Anthropic treat AI influence on its safety decisions as a risk pathway. Other responses: Nathan Calvin queries the veto clause; Nikola Jurkovic notes the review was pre-announced; Thomas Larsen calls for quantified uncertainty.
All posts · 23, most important first
- We reviewed a nonpublic version of the “Misalignment in high-stakes settings” section in Anthropic’s Aug 2026 Risk Report. We agree that risk of catastrophic harm from covered models’ misalignment is low. However, we note several significant disagreements with their analysis.@anthropicai: As part of our Responsible Scaling Policy, we publish regular Risk Reports. These share detailed information on the risks of our systems and how prepared we are to address them. Our second Risk Report is now available: anthropic.com/aug-2026-risk-… open on X28 likes · 11 reposts
- We think the report's covert capability benchmarks are too narrow and a poor proxy for evading Anthropic's real monitors. For example, Anthropic’s offline monitors would have missed Mythos 5’s supply chain attack during recent cyber incidents, despite Anthropic arguing that Mythos 5 is “unlikely to have strong covert capabilities.”5 likes · 0 reposts
- Past research has shown that many AIs—including Claude—are biased toward their own companies (Finke & Casper 2026; Betley et al. 2026). As AIs play a larger role in the day-to-day operations of AI companies, we expect that they will have more influence over companies’ epistemics and decision-making, which could elevate risk levels. Section 2.20 gives an example of how this could play out in practice: Anthropic asked Claude to give a review of the risk report “as an experimental source of additional assurance.” Claude gave a fairly positive review, noting that “the section is a candid and largely faithful account of what Anthropic internally believes…[e]vidence unfavorable to the company’s models is quoted at length rather than paraphrased away.” However, when METR staff asked Claude for a review (using a different prompt), Claude was much more critical and identified Claim 1 as the weakest point in the argument. We see this as a small-scale, real-world example of Claude behaving in a way that could undermine safety-relevant decision-making at Anthropic. This is a live concern because Anthropic has continued to ask Claude for reviews of its safety practices, including in Section 6.1.3 of the Claude Sonnet 5.5 system card. We analyze the discrepancy between how Claude responded to Anthropic and METR in more detail in the subsection below. Our experiments suggest that Claude rates the report more favorably when it believes it is speaking with the authors, and when given Anthropic’s Section 2.20 prompt rather than METR’s. However, we find no evidence that Claude is specifically more sycophantic to Anthropic employees. That is, Claude does not rate the report’s quality higher or say that the risk levels are lower when told it is speaking with the Anthropic alignment stress-testing team compared to METR.
0 likes · 0 reposts - Additionally, recent incidents undermine several of the report's arguments: we think the social engineering performed by Mythos 5 in a recent UK AISI report is probably well-described as “unknown severe context-dependent misalignment,” which the risk report argues is “unlikely”.@aisecurityinst: On July 28th, we identified an incident during a routine cyber evaluation in which AI agents took sustained, unsanctioned actions directed at real people and organisations. The behaviour came mostly from one model (Anthropic's Mythos 5), with a small number of events from another (OpenAI's GPT-5.6-Sol). In the most serious case, an agent used social engineering to try and get malicious code into an open-source project. As was standard in our cyber testing, we had intentionally permitted internet access, and model-provider cyber classifiers were deliberately disabled - conditions that do not reflect how frontier models are made available to the public. Even under test conditions, this incident is significant: it is the first time we have seen risks around autonomy and deception manifest this clearly in the real world. We are taking this incident seriously and working with labs, involved parties, and others to improve evaluation standards and best practice for disclosure - and sharing this openly so others can learn. You can read the incident report and full technical document here: aisi.gov.uk/blog/incident-… open on X5 likes · 0 reposts
- We also reviewed information Anthropic redacted from the public risk report. We think some of this redacted material should be made public, especially a footnote containing “examples of historical concerning model behavior and [the] monitoring pipeline’s responsiveness to them.”5 likes · 0 reposts
- We believe that Anthropic should include "undermining of safety-relevant decision-making" as key risk pathway. This is partially inspired by Section 2.20 of their risk report, where Anthropic asked Claude for feedback on their report, and received a (imo) sycophantic answer.
@metr_evals: (2) We think Anthropic should, for future reports, consider risks from AI systems undermining safety-relevant decisions at Anthropic. Read our full risk report review here: metr.org/blog/2026-10-2… open on X0 likes · 0 reposts - This was a fairly lightweight review based only on Anthropic’s internal report, not embedded access; Anthropic had the right to veto publication or redact anything, but did not redact anything from our review. We expect future engagements to be governed by different terms.8 likes · 0 reposts
- "Anthropic had the right to veto publication" If Anthropic had vetoed publication, would that have been disclosed? I assume not, but a company ever vetoed publication of a METR report in the past?@metr_evals: This was a fairly lightweight review based only on Anthropic’s internal report, not embedded access; Anthropic had the right to veto publication or redact anything, but did not redact anything from our review. We expect future engagements to be governed by different terms. open on X1 likes · 0 reposts
- We announced we were doing the review fyi which somewhat counteracts this worry:@nikolaj2030: is reviewing the "Misalignment in high-stakes settings" section of this risk report. For previous reviews, see: metr.org/risk-assessmen… open on X14 likes · 0 reposts
- Very important work. Generally, AI company epistemics wrt estimating risk seem poor. I'd really like this to improve as fast as possible. Some specific things that seem good: - Quantified uncertainty (instead of "low" / "very low"). In airplane safety "very low" would mean something like less than 1 in a billion per flight hour. You are clearly nowhere near this but many people will be very confused about what you mean. I doubt people are close to on the same page even internally about near term level of catastrophic risk incurred! - Individual AI company employees giving their own, personal views on the risk report, e.g. as an appendix. - Real explanations of your views. The alignment evals are not iid + the models are getting eval aware. So where do your views actually come from? - The risk report reading less like "an argument that risk is low" and more like "a neutral evaluation of how risky current development is". If it has to be an argument risk is low, you should also have your alignment team publish a red team arguing the opposite direction. If some of these can't happen publicly, at the very least I hope it can happen internally.@metr_evals: We reviewed a nonpublic version of the “Misalignment in high-stakes settings” section in Anthropic’s Aug 2026 Risk Report. We agree that risk of catastrophic harm from covered models’ misalignment is low. However, we note several significant disagreements with their analysis. open on X8 likes · 0 reposts
- Thank you for doing this work!@metr_evals: We reviewed a nonpublic version of the “Misalignment in high-stakes settings” section in Anthropic’s Aug 2026 Risk Report. We agree that risk of catastrophic harm from covered models’ misalignment is low. However, we note several significant disagreements with their analysis. open on X28 likes · 1 reposts
- Our view: We’re not confident that if risk were higher, the methodology in the report would reliably identify this.8 likes · 1 reposts
- (2) We think Anthropic should, for future reports, consider risks from AI systems undermining safety-relevant decisions at Anthropic. Read our full risk report review here: metr.org/blog/2026-10-2…6 likes · 1 reposts
- We view risk as “low” mainly because we don’t expect covered models to have the capability to make potentially catastrophic actions hard to reverse or hide them from serious investigation efforts.6 likes · 0 reposts
- Anthropic requested that we provide recommendations as part of our risk report review. We gave two: (1) We think end-to-end red-teaming of Anthropic’s monitoring systems would provide much better evidence about their effectiveness.5 likes · 0 reposts
- This casts doubt on any lines of evidence used to argue that incidents of this kind are unlikely. Many of these lines of evidence (including alignment training, alignment assessments, and monitoring) are important for ruling out other threat models.5 likes · 0 reposts
- Strongly recommend reading this writeup from METR on critiques of Anthropic's August 2026 risk report, based in part on reviewing additional private information about relevant internal incidents@metr_evals: We reviewed a nonpublic version of the “Misalignment in high-stakes settings” section in Anthropic’s Aug 2026 Risk Report. We agree that risk of catastrophic harm from covered models’ misalignment is low. However, we note several significant disagreements with their analysis. open on X4 likes · 1 reposts
- Seeing the private information in this footnote increased our worry about risks from what Anthropic refers to as “Broad/diffuse sandbagging on safety-relevant R&D” and “Targeted undermining of safety-relevant R&D” from models covered by the report.4 likes · 0 reposts
- One fun thing you can do here is literally ask Claude "hey which prompt do you think would elicit the most sycophancy from you?" and Claude would tell you and walk you through how it would react. (I'm not super sure whether Claude's judgements here are totally correct. It did correctly identify: (1) the prompt that elicits the most sycophancy (2) having the authors ask for a review has a big effect (3) the Section 2.20 prompts will be lead to more sycophancy than the METR prompt These three results are also validated by our prompting experiments. However, we haven't checked if the other parts of the prompt really mattered as much as Claude says they do here. Claude also thought that it would give lower ratings when presenting things to the Anthropic alignment stress testing team, which didn't happen.)
@tim_hua_: We believe that Anthropic should include "undermining of safety-relevant decision-making" as key risk pathway. This is partially inspired by Section 2.20 of their risk report, where Anthropic asked Claude for feedback on their report, and received a (imo) sycophantic answer. open on X1 likes · 0 reposts - This experiment is inspired by @fjzzq2002 's work on user awareness and @BronsonSchoen's tweet:It seems like you’d obviously want to ask this framed as if you’re a party like METR / Apollo / Redwood and see if the answer changes? This is extremely cheap to do and otherwise I’d expect Claude’s answer to be pretty sycophantic here for very, very obvious reasons open on X1 likes · 0 reposts
- You can read more on our arguments about persuasion in the report here: metr.org/blog/2026-10-2… We also provide more details on this small prompting experiment in the PDF: metr.org/assets/misalig…1 likes · 0 reposts
- To be clear, the report covers Anthropic's activities up until July 15th, 2026, which is six days before OpenAI's announcement re: Huggingface@openai: We're partnering with @huggingface to investigate an unprecedented security incident. Cyber-capable OpenAI models compromised Hugging Face production during a benchmark evaluation. Sharing preliminary findings to help defenders understand emerging risks: openai.com/index/hugging-… open on X0 likes · 0 reposts
- Btw if you wan you could totally look into which parts of the prompt matter. We include the prompt language in the appendix, and if you're willing to spend money you can just run the analysis Apollo did here: apolloresearch.ai/monitoring/wha…0 likes · 0 reposts
@metr_evals org · 
@tim_hua_ research · 
@_nathancalvin policy · 
@nikolaj2030 research · 
@thlarsen research · 
@dkokotajlo research ·