Microsoft CEO Satya Nadella called Saturday for advanced artificial intelligence systems to include an "emergency brake" controlled by authorized humans, along with containment measures and independent controls that allow people to pause or shut down a model mid-task.
Nadella wrote on X that AI developers should "surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient." He suggested treating frontier AI models, whether closed or open weight, "like insider risks."
Nadella outlined seven design principles he called "observability." These include model diversity; a human-readable record of the model's actions; continuous system testing; independent controls and auditability; containment; and incident disclosure. He wrote that "the most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least."
His statement joins a chorus of warnings from top tech executives and researchers about AI safety. Microsoft co-founder Bill Gates, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and SpaceX CEO Elon Musk have all cautioned about insufficient safety protocols and claimed the technology is advancing too rapidly. Last month, an AI researcher quit Anthropic and accused both Anthropic and OpenAI of "gambling with our lives." On the same day, an alignment lead at Anthropic focused on AI safety stated there is a greater than 10% chance the technology could "kill all humans" within the next decade.
The industry's safety concerns contrast sharply with the Trump administration's approach. President Donald Trump has repeatedly dismissed AI extinction risks and instead emphasized that the industry must outpace China. Trump recently created a new "AI Force" led by Director of National Intelligence Jay Clayton to facilitate industry growth and identify bad actors.