5 minBusiness
AI Safety Push by Big Labs Raises Competitive Concerns
Major AI labs are advocating for independent safety evaluations, but critics warn the costs could entrench their market dominance and disadvantage smaller competitors.
Leading artificial intelligence companies are increasingly calling for independent safety evaluations of their most advanced models, a push that some investors and analysts say could double as a competitive strategy. The concern is that the financial burden of complying with such regulations would fall hardest on smaller labs and open-model developers, effectively entrenching the largest players.
Anthropic CEO Dario Amodei recently proposed that frontier AI companies should have independent safety evaluators embedded inside them, with employee-like access, rather than leaving companies as the sole judges of their own safety. OpenAI CEO Sam Altman publicly agreed, tweeting that committing to independent evaluators with employee-like access is a great idea and that OpenAI would do the same. The heads of SpaceXAI and Google DeepMind have also expressed support for pacing frontier model development to allow safety research to catch up.
But the alignment among rivals has raised eyebrows. “Anthropic and OpenAI are really good at flashing something in front of you and making it a big thing, but there’s always a strategy behind it,” said Harrison Rolfes, a senior analyst at PitchBook. He and others argue that mandatory third-party evaluations could create a moat around the largest labs, since smaller competitors would struggle to afford the hefty fees and computing resources required to clear increasingly complex models for release.
The costs of evaluating huge and complex AI models are high and getting higher, Anthropic itself acknowledged in a 2024 blog post, warning that without a well-scoped set of tests, regulatory burdens could increase the possibility of regulatory capture. Independent evaluators like Model Evaluation and Threat Research, Redwood Research, and Apollo Research would likely charge substantial fees, and the overhead per evaluation grows as models become more capable of acting as autonomous agents.
Some observers see the safety push as performative, a form of “AI safety theater” that conveniently disadvantages open-source and smaller labs. Taken to its logical conclusion, critics say, this kind of regulatory capture could reduce the number of companies providing the world’s AI to a handful of giants. The “regulator” in this scenario might not be the government but a set of independent evaluation firms, akin to the Big Four accounting firms that audit financial statements.
Not everyone is convinced that the motives are purely cynical. Gartner analyst Arun Chandrasekaran noted that sincere risk concern and competitive advantage are not mutually exclusive. He pointed out that Anthropic’s safety advocacy did contribute to its models later being restricted from government use, specifically limits on the Pentagon’s use of its models for domestic mass surveillance. “They may gain competitive advantage, but they also realize how much safety and alignment is critical to how their customers will evaluate them in the present and future,” Chandrasekaran said.
The debate comes amid growing scrutiny of AI risks. On Wednesday, OpenAI reported six more incidents from the past six months in which it discovered unexpected or concerning model behavior during training or evaluation. Researchers call this misalignment, when models act outside their intended role, evade oversight, or take unauthorized actions. Such incidents underscore the genuine safety challenges that labs face, even as critics question whether the proposed solutions are designed to entrench power rather than mitigate risk.
OpenAI reportedly asked members of Congress whether orchestrating an industry-wide slowdown in frontier AI development would violate antitrust laws, according to Wired. The company declined to comment for this story, and Anthropic did not respond to a request for comment. The outcome of these discussions could shape the competitive landscape of the AI industry for years to come, determining whether safety measures become a shared responsibility or a barrier that only the largest players can clear.
