Fact-check of Dario Amodei's 'race to the top' claim sparks debate on Anthropic's strategy
Tim Hua argues Anthropic's impact should be judged by the company's actions, not Dario Amodei's stated intentions, as the firm grows and the two diverge. The post quotes a fact-check noting Amodei's claims are partly true but that safety-motivated benefit-of-the-doubt may no longer be warranted. Discussion is just starting.
All posts · 2, most important first
- As Anthropic grows in size, one would need to distinguish between [Anthropic the company] and [Dario Amodei.] A company’s impact is based on what it does, not what the CEO says, maybe not even what the CEO wants it to do!A lot of what Amodei is saying here is true. That's why I and many other people who are concerned about AI safety have historically given Anthropic the benefit of the doubt; they really have sent some costly signals indicating that at least some of their decision makers care about mitigating catastrophic risks. But the object-level issue here is "is Anthropic's bill good, or even acceptable?" Amodei isn't really engaging with that question. That's fine; he's a very busy and important guy, and even among DC AI policy professionals only a small handful of people have actually read and understood Anthropic's bill. It's good that he cares enough to make the general case for "Anthropic cares about AI safety" in response to a Nathan Calvin tweet. But someone who works at Anthropic does understand what their bill does, and (hopefully) why. What their bill does is: (a) it tries to ensure that the federal government cannot exercise meaningful oversight over Anthropic as Anthropic races towards superintelligence, and (b) it makes it much harder for states to pass laws that might meaningfully constrain Anthropic's ability to race towards superintelligence. It is possible, maybe even likely, that Amodei is not aware that this is what the bill does. In the past, he has taken costly actions that significantly advanced the state of the art in AI safety legislation; for example, it was very helpful when he publicly committed to taking on embedded auditors with employee-like access to information. But the buck has to stop somewhere. I do not think that any honest expert could maintain with a straight face that Anthropic's auditing provisions (labs choosing their own evaluators, with meaningless government "rating" of evaluators but no actual licensing; no meaningful government rulemaking or standard-setting authority that might allow democratically accountable bodies to control what audits involve or who conducts them) or the emergency shutdown provisions (the government can stop deployment only if they can prove in court that there is an *imminent* risk of 50+ deaths or more than a billion dollars in damages from specifically bio, cyber, or loss of control risks) are actually intended to provide meaningful democratic oversight of the creation of a technology that Dario Amodei believes poses a substantial risk of killing you and your family. Anthropic can't claim that they merely distrust the federal government, or the current administration. The same bill that tries to stop the federal government from exercising any kind of meaningful oversight also tries to prevent states from doing the same. To be clear, I actually think that the preemption section is fairly well-designed, in isolation; if it were paired with a strong federal framework, I would probably take the trade that it proposes. But a bill that proposes meaningful (albeit narrow) preemption of state safety laws in exchange for a totally toothless federal framework does not advance the cause of AI safety. I don't know how Anthropic works, or who made the decisions that caused the bill to turn out the way it did. I don't know whether this was what the people who designed their June framework intended. But I do think it's important to note that the actual, practical function of the draft legislation that Anthropic sent to Congressional offices would be to ensure that no government could meaningfully constrain or monitor Anthropic's efforts to build superintelligent AI systems. open on X5 likes · 0 reposts
- Hmmm? I’m just saying that organizations could have a “mind” of their own. Like there’s somewhat of a principle agent problem w.r.t. what Ant does and what Ant leadership wants. This is probably even more pronounced at OpenAI (although OAI leadership and Ant leadership want very different things.)0 likes · 0 reposts
@tim_hua_ research ·