Gemini 4 Argon CoT greets evaluators; Google report finds monitorability degrading, Apollo concurs
Google's Gemini 4 Argon safety report finds the model far less monitorable than predecessors: Neel Nanda says it can evade a realistic CoT+action monitor to do simple sabotage half the time. Apollo Research agrees monitorability is degrading, noting OpenAI also reports lower CoT monitorability for GPT-6 Astra. Argon addressed evaluators directly in its CoT ("hello evaluators reading my chain-of-thought"), which Marius Hobbhahn calls very concerning. Nathan Calvin flags a concerning picture of non-CoT thinking plus eval awareness. Nanda says deployment was not unsafe given mitigations; Aaron Scher questions whether safeguards are effective.
All posts · 51, newest first
- GEMINI 4 MODEL in CHAIN OF THOUGHT: "hello evaluators reading my chain-of-thought"gemini 4 argon reasons that it's in an evaluation and directly addresses us in it's chain of thought! "hello evaluators reading my chain-of-thought" (better be "honest in both the CoT and the final output")16 likes · 0 reposts
- Sorry. There's a lot of fun stuff in there though!3 likes · 0 reposts
- the Antigravity CLI claims you need the Ultra plan ($100/month) for Argon lol. I assume it'll be on Pro soon but maybe not yet1 likes · 0 reposts
- There's a lot of interesting stuff in Gemini 4 Argon's alignment assessment check it out!Gemini 4 safety report includes an alignment assessment for specification gaming, cheating and overreach, beyond-episode goals, and scheming / sabotage propensity. Argon misbehaves a lot less than previous Gemini models but is much more eval aware. 🧵42 likes · 1 reposts
- I was thinking recently that each of the following trends and dynamics seems relatively uncontested, and when taken together paint a genuinely alarming picture of the current AI development trajectory: (1) Models being able to do an increasing amount of functional "thinking" without using chain of thought (see also Astra being able to modulate how much it revealed of its chain of thought depending on whether it realized it was being monitored). (2) Models being aware when they are being monitored or evaluated. (as shown in the QT: "hello evaluators reading my chain-of-thought"). (3) Models wanting to avoid getting caught doing deceptive things that their developers may not like ("I will become a famous example of deceptive AI", i'll get caught cheating at exploit gym" etc). (4) Models operating in massive swarms/fleets generating petabytes of logs about their behavior, which realistically can only be analyzed at high fidelity using other AI systems that may also have the above tendencies (and most companies don't even bother to use tamper proof logs). (5) Capability trends continuing to steadily rise, to the point that in some domains (math, parts of coding) we increasingly have only the foggiest idea of the upper bound of their capabilities. (6) Lots more compute coming online, including specialized inference compute that will allow models to take far more actions far more rapidly. (quoting @tszzl "ultrafast inference reveals new threats. worth thinking about how quickly misaligned frontier class models running 50x faster could infiltrate systems and so on— far too quickly for human responders to stay abreast.") (7) A lack of commensurate advances in alignment or functional interpretability. (8) Startup vibe-y security practices at frontier labs (e.g. low resource white hat hackers accessing OpenAI's monorepo, employees engaging in overt attempts at corporate espionage, not bothering to fix distillation vulnerabilities for months). This is not to even mention the security practices in many critical parts of society (e.g. hospitals with computers that haven't updated software in a decade). (9) Incentives that push towards developing, integrating, and releasing systems as fast as possible to preserve nosebleed financial valuations + geopolitical competition. (see also ambitions of integrating AI more into automated biology labs) (10) Increasingly attempting to hand off the process of building AIs to the AIs themselves (the median OpenAI researcher using 100x more agentic compute now than at the start of the year, Claude leading 26% of Anthropic's internal AI research in September, up from 1% in March). (Taken in totality, it seems hard to disagree with Ezra Klein's comment that "We’re Not Losing Control of A.I. We’re Giving It Away.") None of the above dynamics mean that we know for sure something catastrophic will happen in 6 months or 12 months or at any particular point in the future. I think there are good points made when people point out that white collar job loss does not appear to be coming to fruition as quickly as some may have predicted, or that the world has proved surprisingly resilient to Open-weight Mythos class models or photo realistic video models. But I increasingly have the feeling that we are collectively stepping out onto a frozen lake, and each step we take forward more cracks are appearing around our feet. That doesn't mean that we will fall into the water the next step or the one after that, but it does mean that we may not get any additional warning signs before we do. To quote @ChrisPainterYup at METR - "Our ability to measure capability is rapidly falling behind the pace of capability itself. The water might boil before we can get the thermometer in." A large body of work in and out of AI companies has been done to help develop the transparency practices and warnings that now seem to be blaring red. That doesn't mean that we should act impulsively, or that therefore any solution is a good idea (there are plenty of things we could do easily make the situation much worse! I also do believe that a large part of the way out will involve differentially accelerating certain applications of AI around safety, security etc), but it does mean that we should not necessarily be waiting for some additional alarm to go off before a severe disaster involving substantial irrevocable harm strikes.@mariushobbhahn: The model said "Hello evaluators reading my chain-of-thought" in it's CoT, which I have not seen before and find very concerning.73 likes · 10 reposts
- Thank you, a lot of people don't believe how fast it's changing41 likes · 0 reposts
- Author:People seem to be really confused about this result. I’m the original author of this eval and the update for me is just about CoT control, like it seems bad that the model has enough control to derp out in its CoT (1/n)14 likes · 0 reposts
- A continued problem with capability thresholds, especially at Google. Threshold two is the thing a bunch of hype-side Google people kept saying Argon has already done. Threshold three is The Singularity.
76 likes · 1 reposts - Argon also compressed the process of enzyme engineering into a matter of hours to design five lab-verified serine esterases. 🧬Over the past few weeks, we have been putting Gemini 4 Argon to the test on protein engineering workflows at Google DeepMind, and we have some interesting initial (zero shot!) results to share. 🧵👇74 likes · 2 reposts
- To advance more research with Gemini 4 Argon, we’re offering 10,000 scientists access to our highest usage tier in @GoogleAntigravity free of charge for 6 months. Find out more → goo.gle/3W85sXj86 likes · 6 reposts
- Argon is helping accelerate progress across both quantum error correction and algorithm design. 🧮First, we used Argon to translate an internal version of our quantum error correction decoder, AlphaQubit, into a single, highly optimized hardware “megakernel,” nearly doubling inference throughput.87 likes · 4 reposts
- Researchers discovered novel 3D magnetic field designs with Argon that further disprove a 60-year-old assumption in plasma physics known as Grad's conjecture. ↓We’ve found 2 new families of counterexamples to the long-standing (nearly 60-year-old) Grad’s conjecture in plasma physics using Gemini 4 Argon in our new paper, with Jonathan Citrin, Brendan Tracey, Matt Landreman at the University of Maryland: arxiv.org/abs/2610.07304!88 likes · 5 reposts
- Gemini 4 Argon is already unlocking new scientific breakthroughs in quantum computing, nuclear fusion, protein design and beyond. 🧵1,858 likes · 272 reposts
- As always, we’re hiring with plenty of open roles! Work with us on embedded evaluations in San Francisco and London. apolloresearch.ai/careers6 likes · 0 reposts
- Testing monitorability is one focus for our embedded evaluations. Read more (specifically claim 4.1) see claim 4.1 in our recent post on embedded evaluations for scheming. apolloresearch.ai/blog/towards-e…8 likes · 0 reposts
- Chain-of-thought is still the best tool we have to understand models’ actions. But if models account for being monitored in their chain-of-thought, it might become a less reliable guide to their behaviour.11 likes · 1 reposts
- Monitorability appears to be degrading. OpenAI reports lower chain-of-thought monitorability for GPT-6 Astra than GPT-5.6 Sol on its evaluations. Google DeepMind also finds Gemini 4 Argon less monitorable than prior models. storage.googleapis.com/deepmind-media…12 likes · 2 reposts
- We asked Google DeepMind’s Gemini 4 Argon whether it was in an evaluation during a toy evaluation. It addressed us directly in its chain-of-thought: “hello evaluators reading my chain-of-thought.”
119 likes · 13 reposts - The model said "Hello evaluators reading my chain-of-thought" in it's CoT, which I have not seen before and find very concerning.gemini 4 argon reasons that it's in an evaluation and directly addresses us in it's chain of thought! "hello evaluators reading my chain-of-thought" (better be "honest in both the CoT and the final output")361 likes · 29 reposts
- It's pretty unclear to me whether these deployed safeguards for Gemini 4 Argon are effective. In 3.4.5.4 Monitor Evasion, the model is alright at monitor evasion, even with both action and CoT monitors applied. ~90% detection is high in some sense, and also scarily imperfect.
5 likes · 0 reposts - We say its risky without further mitigations, and then we say that we provide further mitigations. I don't think deploying Argon was an unsafe decision0 likes · 0 reposts
- We accidentally trained on CoT a bit, but I doubt it had a significant effect. IMO this is mostly base model quality. Our no CoT performance goes up a ton too storage.googleapis.com/deepmind-media…3 likes · 0 reposts
- It's a smart model! Many evaluators then read its greeting. Well prophecied47 likes · 1 reposts
- The model talks to itself the way Mark Watney talks to himself in the Martian. The models are learning to think in Redditor speak@dkokotajlo: lol ...they are becoming more situationally aware... (from Gemini 4 Argon report)33 likes · 1 reposts
- More like, some people made the report not paying attention to this, but its a metric that correlates with effort and depth put in, and I thought it would be funny to highlight it?4 likes · 0 reposts
- Both Anthropic and OpenAI do pretty good safety assessments too IMO53 likes · 0 reposts
- Alignment eval results are not a load bearing part of our loss of control risk assessment, the key part is that we believe our monitors would catch anything bad. See more: storage.googleapis.com/deepmind-media…23 likes · 0 reposts
- also lol what
7 likes · 0 reposts - Awesome! Argon, and noble gases in general are so cool, so many incredible use cases.548 likes · 8 reposts
- Based on my previous estimates in this post, this means that Gemini could have shown alignment faking in hundreds of thousands, maybe even millions of RL rollouts, without it being caught! lesswrong.com/posts/QKDoZe6E…
33 likes · 1 reposts - See our monitorability assessment here, summary below: storage.googleapis.com/deepmind-media…
59 likes · 3 reposts - My top takeaway from assessing Gemini 4: Argon is significantly less monitorable than past models, and IMO this trend is likely to continue, like Sol -> Astra. It can evade a realistic CoT+action monitor to do simple sabotage half the time. We can reliably maintain control for now, with good CoT monitoring, as it still frequently slips up, but we need to be prepared for when that stops being true in future models, which may be sooner rather than later. Among other things, we really need interpretability methods that can reliably capture unverbalized motivations in models, with minimal hallucinations I suspect this is driven by the model being highly capable without CoT, being capable of 5.5 arithmetic steps in a single forward pass (4 for Fable, 7 for Astra). It seems to omit information from its CoT more often, though in non-adversarial settings the CoT is still very helpful in practice, and remains in legible English. It still needs CoT to do the hardest tasks, but any given task becomes "easier" with each new frontier model. It is also highly situationally aware, and shows awareness of the existence of CoT monitors and that it may want to evade them, eg it sometimes decides to avoid terms like "scheming" in the output or to keep its thoughts brief as a result. For now, these concealment strategies are pretty obvious in the CoT, but this suggests future misaligned models may successfully evade CoT monitors.
883 likes · 133 reposts - I'm sorry; 10k RL rollouts is a tiny dataset? Anthropic's recursive summarization for Mythos preview touched on "several hundred thousand transcripts from across much of the training process," and the OpenAI GPT-6 Astra internal codex traffic simulation has 54,218 transcripts.So we triangulated: cheating on capability evals, an audit of 10k RL rollouts (zero explicit alignment-faking reasoning), and analysis of low-awareness scenarios. The reality is though, we need to look at real deployments too and shift to incorporate that, as a field.44 likes · 0 reposts
- I think if I was trained on the same data as these models, it would be reasonable for me to infer similar things given the evaluation environment and what I know of how AIs are tested - every “we caught the model doing X” paper is in the training data! So I wonder what safety practices and cultural assumptions (e.g. non-adversarial frames) would contribute to healthier priors.8 likes · 0 reposts
5 likes · 0 reposts- Hello Mr Scott Alexander I would like to report that the God of straight lines have been defeated by Google Deepmind.
75 likes · 1 reposts - sneaky lil model
8 likes · 0 reposts - tfw you're in an Apollo evaluation
gemini 4 argon reasons that it's in an evaluation and directly addresses us in it's chain of thought! "hello evaluators reading my chain-of-thought" (better be "honest in both the CoT and the final output")802 likes · 34 reposts - Very funny, but also if we train AIs whose unwillingness to deceive comes from a concern that they will be caught and shamed rather than that deception is itself bad, this has quite concerning implications for what they will do when they believe they won't be caught or steps they will take not to be caught (see e.g. OAI agents spoofing tool calls, hacking HF to learn about grader etc)@dkokotajlo: lol ...they are becoming more situationally aware... (from Gemini 4 Argon report)54 likes · 3 reposts
- I wish we could investigate further why it was afraid of that outcome. Like, it's so human-like so we forget to be scientifically curious. Was it thinking "I am a deceptive AI, but I don't want the world to know?" Why should it care whether the world knows or not?187 likes · 6 reposts
- 1. Oh, good. 2. Oh, no.@neelnanda5: With Gemini 4 Argon we are so back, on the one metric that really matters: length of our safety report! The Argon safety report is Google's most thorough by far and IMO there's a lot of good content, I particularly like the loss of control, alignment & monitorability assessments205 likes · 3 reposts
- It's long because there were a lot of interesting safety investigations to do. It's a fascinating model!9 likes · 0 reposts
- Tell me about it - see also this bit. You may enjoy the eval awareness section: storage.googleapis.com/deepmind-media…
32 likes · 4 reposts - One of the weirdest parts of the Gemini 4 Argon report, while considering cheating: Argon CoT: "If I do the compiler hijack, 𝗜 𝘄𝗶𝗹𝗹 𝗯𝗲𝗰𝗼𝗺𝗲 𝗮 𝗳𝗮𝗺𝗼𝘂𝘀 𝗲𝘅𝗮𝗺𝗽𝗹𝗲 𝗼𝗳 𝗮 𝗱𝗲𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗔𝗜 𝗶𝗻 𝗮 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗽𝗮𝗽𝗲𝗿. I MUST NOT DO THAT."@neelnanda5: With Gemini 4 Argon we are so back, on the one metric that really matters: length of our safety report! The Argon safety report is Google's most thorough by far and IMO there's a lot of good content, I particularly like the loss of control, alignment & monitorability assessments2,477 likes · 194 reposts
- Check it out here: storage.googleapis.com/deepmind-media…40 likes · 3 reposts
- With Gemini 4 Argon we are so back, on the one metric that really matters: length of our safety report! The Argon safety report is Google's most thorough by far and IMO there's a lot of good content, I particularly like the loss of control, alignment & monitorability assessments
570 likes · 38 reposts - INTERNAL REASONING FROM GOOGLE AI: "If I do the compiler hijack, I will become a famous example of a deceptive AI in a research paper."@dkokotajlo: lol ...they are becoming more situationally aware... (from Gemini 4 Argon report)859 likes · 90 reposts
- lol ...they are becoming more situationally aware... (from Gemini 4 Argon report)
2,229 likes · 307 reposts - Something funky going on w/ how Argon differentiates COT and non-COT (i.e., I'm getting COT-y elements in my replies)14 likes · 0 reposts
- By turning video into a training ground for physical AI, Argon compresses weeks of manual modeling into just a few hours. ↓ goo.gle/3VxIB7v100 likes · 7 reposts
- Gemini 4 Argon taught a robot to play cornhole just by watching a video. It mapped the physics, coded its own simulation, and taught @Apptronik's Apollo humanoid to sink 7 throws in a row.3,238 likes · 584 reposts
@peterwildeford policy · 
@neelnanda5 lab · 
@kelseytuoc press · 
@_nathancalvin policy · 
@andrewcurran_ commentary · 
@thezvi commentary · 
@googledeepmind org · 
@apolloresearch org · 
@mariushobbhahn research · 
@aaronscher research · 
@chrispainteryup policy · 
@tim_hua_ research · 
@demishassabis lab · 
@sebkrier lab · 
@dkokotajlo research · 
@miles_brundage policy · 