DeepSeek V4.1 Flash beats GPT-4 Sol on multi-step offensive cyber benchmark
Research reports that DeepSeek V4.1 Flash 1017 outperformed GPT-4 Sol on a multi-step offensive cybersecurity task within a simulated corporate network. Nathan Lambert (research) highlighted this finding, suggesting open-weight models are rapidly developing dangerous offensive capabilities in this area. Peter Wildeford (policy) also noted this trend, suggesting open models may reach high-tier performance in cyber exploitation.
All posts · 2, most important first
- Probably great at defensive cyber too, but no one will post that and get clicks!We found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on our multi-step offensive cybersecurity task. Sol was tested via the Daybreak trusted access program. We have been building a cybersecurity benchmark based on simulated corporate networks. This is a valuable indicator for the cyber-risk a model poses if in the hands of a bad actor. We plan to make the full evaluation public in the coming weeks but wanted to share our first immediate finding here. open on X92 likes · 7 reposts
- Sure seems like more open weight AI models are becoming Mythos-tier in their ability to develop cyber exploits 👀We found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on our multi-step offensive cybersecurity task. Sol was tested via the Daybreak trusted access program. We have been building a cybersecurity benchmark based on simulated corporate networks. This is a valuable indicator for the cyber-risk a model poses if in the hands of a bad actor. We plan to make the full evaluation public in the coming weeks but wanted to share our first immediate finding here. open on X5 likes · 2 reposts
@natolambert research · 
@peterwildeford policy ·