Big Tech's AI auditors will only see what they're shown

Last week, six of the most powerful AI companies in the world promised the White House they would allow outsiders to check their work. On Sept. 29, Nvidia, OpenAI, Anthropic, Google, Meta and xAI signed an accord on advanced AI models that U.S. President Donald Trump called 'morally binding.' Each company has committed to 'partner with an independent external auditor or evaluator' and have independent board members review auditors' reports and ensure problems get fixed.

While outside audits are a positive step, the accord remains silent on crucial details: who chooses the auditors, what they can examine, and whether any findings will be made public. There's also no penalty for non-compliance. Currently, each company selects its own auditor and determines what that auditor gets to see.

Historical Precedents

The dangers of self-selected auditors were demonstrated in 2000 when Enron paid accounting firm Arthur Andersen $25 million to audit its books and another $27 million for tax and consulting work. Within two years, both firms had collapsed. Congress responded with the Sarbanes-Oxley Act, which removed auditor selection from management control, restricted auditors from selling other services to audit clients, and created a public board to inspect auditors themselves.

Voluntary pledges in the AI industry have similarly poor track records. In 2023, leading AI companies made voluntary commitments to the White House, including allowing outside experts to test their systems before release. When researchers at Stanford, Harvard and Brown evaluated 16 companies against these commitments, the average compliance score was just 53%. On the promise to help outsiders find and report problems, eight of the 16 companies scored zero.

Testing Limitations

Even when outside testing occurs, developers often set the restrictive terms. METR, a nonprofit that tests AI models, noted that its evaluation of OpenAI's o3 model before release was 'conducted over only three weeks,' with 'limited access to information about the model' and no view of the model's step-by-step reasoning. While OpenAI allowed the testing, the example illustrates how much depends on what a company chooses to share.

A trained AI model represents a valuable asset, often costing over $100 million to produce, and whoever possesses a copy effectively owns the model. Handing copies to outsiders creates security risks, historically forcing testers into a dilemma: either submit their questions to the company (revealing the test in advance) or allow the company to share its model.

Potential Solutions

In August, Google DeepMind demonstrated a potential solution by conducting what it called the world's first double-blind test of an AI model, following the principle of double-blind drug trials. Working with Singapore's AI Safety Institute and three other outside groups, the test ran on sealed computer hardware, preventing testers from seeing the model and Google from viewing their questions. Google claims both sides can mathematically verify their material remained private.

This approach shifts trust to the hardware, though researchers previously breached similar sealed systems with less than $1,000 worth of equipment, requiring physical server access. While this vulnerability is serious, it's more manageable than complete exclusion of testers and can be addressed through careful control of the machines.

In a recent report titled 'Alternatives to Concentrated AI,' researchers interviewed 36 people and mapped nearly 30 working alternatives to concentrated AI control. Their key finding was that outsiders can now scrutinize advanced AI without taking possession of the model or its data. For the accord's audits to be meaningful, auditors should choose what they test, companies should disclose their auditors and audit scope, and board committees overseeing these reports should answer to independent parties outside the company.