Over the past year, concerns have swirled over the potential dangers posed by artificial intelligence, leading to intense debates about the appropriate path for AI safety. Anthropic, an AI lab, has recently put forth its vision for government regulation in the form of a draft bill on AI safety, which it hopes Congress will consider adopting.
Proposed Legislation
The legislation outlines specific scenarios in which the release of an AI model could be blocked from distribution. These scenarios include:
- A "foreseeable, imminent risk" where over 50 people are at risk of dying.
- A situation where over a billion dollars in property damage could result.
According to Politico, the catastrophic risks must stem from:
- The creation or release of a biological weapon.
- The model evading its developer’s controls.
- A cyberattack.
The metrics of 50 people and a billion dollars are similar to those defined in California's recently passed Senate Bill 53, which also categorizes these scenarios as tantamount to catastrophic risk.
Evaluation Process
Anthropic's bill suggests that the assessment of a model's potential for disaster could be conducted by independent evaluators registered with the U.S. Commerce Department. These evaluators might be compensated by the AI labs, provided that their payment is not contingent on the evaluation results. However, it remains unclear how this system would prevent payment from influencing evaluations.
