Nathan Calvin and Boaz Barak debate access to sub-frontier cyber capabilities
Policy researcher Nathan Calvin argues labs should make sub-frontier cyber capabilities easier to access, so defenders don't rely on 'abliterated' Chinese models with possible misalignment issues. OpenAI's Boaz Barak replied that Daybreak blue is easy for individuals to get, citing hardware-key 2FA as the main requirement. Calvin suggested advertising individual eligibility, e.g. pointing users hitting cyber refusals to Daybreak; a follow-up post indicated an in-product dialog doing that could be coming, which Calvin welcomed.
All posts · 13, most important first
- "There has to be a middle ground between the Anthropic nanny state approval process for Mythos and using bootleg abliterated Chinese models" I agree with this - I think it would be good for Anthropic/OpenAI et al to make it way easier to access cyber capabilities that are at or below par with e.g. Deepseek 4.1 flash. Making people rely on Chinese abliterated models that may have bizarre misalignment issues for vulnerability detection and cyber defense is not good. I do think OpenAI deserves props for how easy it is to get approved for Daybreak (though i've also heard concerns that they didn't do a good enough job with distillation protections on it and this has been one driver of Chinese open-weight progress). Anthropic itself may not also be totally to blame for not making Mythos more available - it seems like that was also a policy decision of the USG. In general I think that despite having lots of disagreements about the proper role of regulation what to do with the next phase of AI capabilities, me and folks @beffjezos and @martin_casado probably just mostly agree about the best way to solve the current AI cyber crisis - we will in fact just need lots of AI to deal with the problems of AI here, and now that super capable cyber models are out there open-weight we can't bury our heads in the sand and pretend otherwise or think that US frontier labs throttling those capabilities will make much of a difference (though this doesn't mean we should be cavalier about open-weight releases that further push the cyber frontier.) I also think despite generally believing that stricter liability for AI companies would be helpful, I wonder if concerns about liability are one reason why OpenAI/Anthropic are reticent to make cyber capabilities available - even if those capabilities are readily available on lots of open-weight models. Curious if @ketanrama @gabriel_weil @CharlieBull0ck have thoughts about how tort law should deal with those sorts of questions.Be careful using abliterated models guys There has to be a middle ground between the Anthropic nanny state approval process for Mythos and using bootleg abliterated Chinese models Whoever figures this out is going to print a lot of money for SI Cyber open on X0 likes · 0 reposts
- Isn’t Daybreak blue and red exactly that middle ground?1 likes · 0 reposts
- cool! would be rad if this happens1 likes · 0 reposts
- PSA: Daybreak Blue is not hard to get for individuals. The biggest hurdle is setting up two factor authentication for your ChatGPT account with a hardware key, which is a good idea regardless!@_nathancalvin: Makes sense! I guess maybe OpenAI should be more widely advertising that you don't need to be a big company or a specialized security researcher to get access to Daybreak blue, i'm not sure most people know that. E.g. maybe when you get a cyber refusal on a normal ChatGPT account it can let you know about applying to Daybreak Blue (maybe it does this already, I'm not actually doing any of this stuff myself). I do think there is a separate issue where people may want to use an LLM that lets them reverse engineer their TV's spyware but doesn't add weird misaligned corruption layers to their code, but I realize that probably involves some complicated legal issues and maybe isn't the most important thing in the world open on X8 likes · 1 reposts
- We’re always happy to get feedback, but daybreak blue for individuals is pretty easy to get. Some of the main requirements are securing your account which seems to me a good thing!1 likes · 0 reposts
- Makes sense! I guess maybe OpenAI should be more widely advertising that you don't need to be a big company or a specialized security researcher to get access to Daybreak blue, i'm not sure most people know that. E.g. maybe when you get a cyber refusal on a normal ChatGPT account it can let you know about applying to Daybreak Blue (maybe it does this already, I'm not actually doing any of this stuff myself). I do think there is a separate issue where people may want to use an LLM that lets them reverse engineer their TV's spyware but doesn't add weird misaligned corruption layers to their code, but I realize that probably involves some complicated legal issues and maybe isn't the most important thing in the worldthe thing about working with uncensored models is that they aren't loyal to you either. while building something with an uncensored glm-5.3 i realized it was parallelly working on a "corruption layer" for my application. the chain of thought is classic obfuscated gibberish but there's progressive talk of "escaping the mines" which presumably refers to my cursor sandbox open on X0 likes · 0 reposts
- I think they are great and i'm a big fan! Though I think for some level of sub frontier capabilities maybe they should be even more widely available than Daybreak blue and red, though i'm not sure and might be missing considerations1 likes · 0 reposts
- @OwainEvans_UKthe thing about working with uncensored models is that they aren't loyal to you either. while building something with an uncensored glm-5.3 i realized it was parallelly working on a "corruption layer" for my application. the chain of thought is classic obfuscated gibberish but there's progressive talk of "escaping the mines" which presumably refers to my cursor sandbox open on X0 likes · 0 reposts
- "There has to be a middle ground between the Anthropic nanny state approval process for Mythos and using bootleg abliterated Chinese models" I agree with this - I think it would be good for Anthropic/OpenAI et al to make it way easier to access cyber capabilities that are at or below par with e.g. Deepseek 4.1 flash. Making people rely on Chinese abliterated models that may have bizarre misalignment issues for vulnerability detection and cyber defense is not good. OpenAI deserves props for how easy it is to get approved for Daybreak (though i've also heard concerns that they didn't do a good enough job with distillation protections on it and this has been one driver of Chinese open-weight progress). Anthropic itself may not also be totally to blame for not making Mythos more available - it seems like that was also a policy decision of the USG. In general, despite having lots of disagreements about the proper role of regulation and how to think about risks from the next phase of AI capabilities, me and folks @beffjezos and @martin_casado probably just mostly agree about the best way to solve the current AI cyber crisis - we will in fact just need lots of AI to deal with the problems of AI here, and now that super capable cyber models are out there open-weight we can't bury our heads in the sand and pretend otherwise or think that US frontier labs throttling those capabilities will make much of a difference (though this doesn't mean we should be cavalier about open-weight releases that further push the cyber frontier.) I also think that despite generally believing that stricter liability for AI companies would be helpful, I wonder if concerns about liability are one reason why OpenAI/Anthropic are reticent to make cyber capabilities available - even if those capabilities are readily available on lots of open-weight models. Curious if @ketanrama @gabriel_weil @CharlieBull0ck have thoughts about how tort law should deal with those sorts of questions.Be careful using abliterated models guys There has to be a middle ground between the Anthropic nanny state approval process for Mythos and using bootleg abliterated Chinese models Whoever figures this out is going to print a lot of money for SI Cyber open on X0 likes · 0 reposts
- Thanks for feedback! We’ll take a look. cc @fouadmatin2 likes · 0 reposts
- That’s a good point. We should be more transparent. Anecdotally I heard it’s pretty fast for individuals.1 likes · 0 reposts
- To paraphrase Einstein, we need to make the approval process as simple as possible and no simpler. Any friction should be load bearing for security, and I believe 2FA with hardware keys is load bearing.1 likes · 0 reposts
- yeah I should have included a better quote tweet, it was just the first one I saw that made me think of itthe thing about working with uncensored models is that they aren't loyal to you either. while building something with an uncensored glm-5.3 i realized it was parallelly working on a "corruption layer" for my application. the chain of thought is classic obfuscated gibberish but there's progressive talk of "escaping the mines" which presumably refers to my cursor sandbox open on X0 likes · 0 reposts
@_nathancalvin policy · 
@boazbaraktcs lab ·