Anthropic is spearheading a new program aimed at revolutionizing AI benchmarks, focusing on evaluating the performance and impact of AI models, including its own generative model, Claude. Unveiled on Monday, the initiative will fund third-party organizations to develop advanced benchmarks that effectively measure AI capabilities.
The current landscape of AI benchmarks often fails to reflect real-world applications and may not accurately assess modern generative AI models. Anthropic aims to address these shortcomings by fostering the creation of challenging benchmarks centered on AI security and societal implications. The company seeks benchmarks that evaluate a model’s ability to perform tasks such as cyberattacks, enhancing weapons of mass destruction, and manipulating individuals through deepfakes or misinformation.
Enhancing AI Safety and Societal Impact
Anthropic emphasizes the importance of developing benchmarks that explore AI’s potential for scientific research, multilingual communication, bias mitigation, and self-censorship of toxic content. The company envisions new platforms that enable subject-matter experts to create their own evaluations and conduct large-scale trials involving thousands of users.
Anthropic offers a variety of funding options tailored to the needs and stages of different projects, although specific details remain undisclosed. The program includes interactions with Anthropic’s domain experts from various teams, including the frontier red team, fine-tuning, trust and safety, and more. A full-time coordinator has been hired to oversee the program, and Anthropic may acquire or expand promising projects.
Balancing Commercial Interests and Safety
While Anthropic’s initiative to support new AI benchmarks is commendable, it raises concerns about potential biases due to the company’s commercial ambitions. Anthropic acknowledges in its blog post that it seeks alignment with the AI safety classifications it has developed, which may influence the types of benchmarks supported.
Some AI experts express skepticism about Anthropic’s references to catastrophic and deceptive AI risks, such as those involving nuclear weapons, arguing that these concerns may detract from pressing regulatory issues like AI’s hallucinatory tendencies. Nevertheless, Anthropic aims for its program to become a catalyst for industry-wide adoption of comprehensive AI evaluation standards.
Anthropic’s initiative to fund the development of new AI benchmarks represents a significant step towards improving the evaluation of AI models’ performance and societal impact. As the program unfolds, its success will depend on balancing commercial interests with the broader goal of advancing AI safety and effectiveness across the industry.
See also: Anthropic Launches Claude 3.5 Sonnet: A Step Forward in Generative AI
