Benchmark reportedly shows DeepSeek V4.1 Flash beating GPT 6 Sol on multi-step offensive cyber task
An unnamed benchmark team reports DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on a multi-step offensive cybersecurity task using simulated corporate networks, with Sol tested via the Daybreak trusted access program. Nathan Lambert notes Sol was tested under trusted access, so the comparison is complicated, and adds the model is "probably great at defensive cyber too." Peter Wildeford reads it as open-weight models approaching "Mythos-tier" cyber exploit development ability.
All posts · 2, most important first
- Probably great at defensive cyber too, but no one will post that and get clicks!We found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on our multi-step offensive cybersecurity task. Sol was tested via the Daybreak trusted access program. We have been building a cybersecurity benchmark based on simulated corporate networks. This is a valuable indicator for the cyber-risk a model poses if in the hands of a bad actor. We plan to make the full evaluation public in the coming weeks but wanted to share our first immediate finding here. open on X92 likes · 7 reposts
- Sure seems like more open weight AI models are becoming Mythos-tier in their ability to develop cyber exploits 👀We found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on our multi-step offensive cybersecurity task. Sol was tested via the Daybreak trusted access program. We have been building a cybersecurity benchmark based on simulated corporate networks. This is a valuable indicator for the cyber-risk a model poses if in the hands of a bad actor. We plan to make the full evaluation public in the coming weeks but wanted to share our first immediate finding here. open on X5 likes · 2 reposts
@natolambert research · 
@peterwildeford policy ·