OpenAI Reworks Pentagon Deal After Customer, Staff Backlash Over Surveillance

This was more of a surprise than perhaps it should have been - and raises further questions about AI's future. After Trump's belligerent Defense Secretary banned Anthropic's AI because the company would not submit to his demands regarding mass surveillance and AI-managed weapons, OpenAI opportunistically leaped in, thinking it had scored a coup, especially given the recent word amongst AI users that Anthropic's AI was clearly superior, especially for professional purposes. But then the backlash hit, with downloads of Anthropic's Claude surpassing OpenAI's ChatGPT and AI prgrammers and engineers across the tech industry brutally criticizing OpenAI's capitulation. OpenAI was forced to rework its Pentagon agreement and its CEO apologized to staff for "subjecting you to this pain."

Over the past year, there has been despair felt by many that Silicon Valley had wholly abandoned its early "don't be evil" mantra and fervently embraced Trump's authoritarianism, with tech leaders competing to see who could debase themselves further - even Apple's CEO showing up at the premier of Amazon's 'Melania' bribe-umentary. And while unapologetic firms like Palantir continue to trumpet their aggressiveness, the OpenAI episode suggests there are far more Silicon Valley workers uncomfortable with the implications of AI development and that they are prepared to speak out about it, which may now result in changed European and, perhaps, US AI policy. JL 

Berber Jin and colleagues report in the Wall Street Journal, Jared Perlo and colleagues report in NBC News:

OpenAI announced its Defense Department deal hours after Defense Secretary Hegseth designated Anthropic a supply-chain risk. After backlash against OpenAI’s deal to allow for mass surveillance, Altman unveiled a reworked agreement Monday governing the Defense Department’s use of its AI services which provides stronger guarantees the military won’t use OpenAI’s systems for domestic surveillance. The new agreement states “the AI system shall not be intentionally used for domestic surveillance of U.S. persons and nationals." A groundswell across Silicon Valley criticized Altman and the company for what they saw as capitulation to the Pentagon. Altman (regretted) it looked “opportunistic” and “not united with the field, with extremely difficult brand consequences and very negative PR for us."

OpenAI CEO Sam Altman defended his decision to allow the Pentagon to use its tools for classified work, fielding questions from staff at an all-hands meeting Tuesday about the principles behind the decision. 

 

Altman unveiled a reworked agreement with the Pentagon on Monday night governing the Defense Department’s use of its AI services, which he says provides stronger guarantees that the military won’t use OpenAI’s systems for domestic surveillance.

 

The new agreement states that “the AI system shall not be intentionally used for domestic surveillance of U.S. persons and nationals,” according to a post on OpenAI’s website. OpenAI had faced some backlash as news of an initial agreement between the leading AI company and the Pentagon emerged Friday. Many observers claimed the original language shared on OpenAI’s website provided ample loopholes for the government to surveil Americans.

OpenAI announced its Defense Department deal on Friday, hours after Defense Secretary Pete Hegseth designated rival Anthropic a supply-chain risk, a move rarely used against a U.S. company. After backlash against the potential for OpenAI’s deal with the Pentagon to allow for mass surveillance, the company changed its agreement to explicitly state that domestic surveillance isn’t allowed.

Altman said that while he didn’t regret signing a deal with the Defense Department, he wished he hadn’t announced the decision so quickly, telling staff that it looked “opportunistic” and “not united with the field,” according to remarks viewed by The Wall Street Journal. His remarks echoed a memo he shared with staff and on X Monday in which he said the deal “looked opportunistic and sloppy.”

A groundswell of AI researchers at the company and across Silicon Valley spent the past few days criticizing Altman and the company for what they saw as capitulation to the Pentagon by essentially agreeing to a deal that allowed AI to be used in all lawful cases. Some publicly peppered the CEO with questions on X, while employees made their case on OpenAI’s Slack channel. Anthropic’s Claude overtook ChatGPT as the most popular app in Apple’s app store for the first time.

Many OpenAI employees had called for the company to sign a deal that explicitly banned the use of its technology for mass surveillance and fully autonomous weapons. Anthropic previously rejected the agreement that OpenAI accepted and has been unusual in opposing some of the Trump administration’s policies. 

Altman said in the all-hands meeting that it had been an unpleasant few days. “To try so hard to do the right thing and get so absolutely like, personally crushed for it—and I know this is happening to all of you too, so I feel terrible for subjecting you all to this—is really painful.”

“I think this was an example of a complex but the right decision with extremely difficult brand consequences and very negative PR for us in the short term,” Altman said.

News Corp, owner of The Wall Street Journal, has a content-licensing partnership with OpenAI.

OpenAI’s Friday announcement blew open a long-simmering debate among AI researchers—many of whom come from academic backgrounds—about whether they can trust business leaders to responsibly deploy their technology. OpenAI employees peppered Altman during the Tuesday meeting with questions about the boundaries of the agreement and the implications of working with the Defense Department. 

Altman said the government was willing to give OpenAI influence over how its technology was deployed, and that he wanted his company to have a seat at the table in shaping future decisions. “We have built a technology that is going to be the fundamentally most important tool for the government and governments around the world,” he said. 

A strong U.S. military had been a “great benefit to all of humanity over the last 250 years,” Altman said, adding that he respects that the military gets to make decisions. “Clearly, the military has done things that I extremely disagree with, and am sure will do more in the future.”

In the meeting, Altman said that OpenAI was also looking at a contract to deploy on all North Atlantic Treaty Organization classified networks. An OpenAI spokeswoman later said Altman misspoke and that the NATO opportunity was for unclassified networks. Representatives for NATO didn’t respond to requests for comment.

OpenAI has largely embraced President Trump’s domestic AI strategy. Since Trump took office, the company has announced large investments with the administration to build data centers across the country. OpenAI President Greg Brockman and his wife donated $25 million to a Trump-aligned super political-action committee last year and are spending millions more to promote his AI agenda in the midterm elections.

Anthropic’s weekslong feud with the Pentagon over the startup’s effort to limit the agency’s ability to use its technology for mass domestic surveillance and autonomous weapons created an industry firestorm over how AI is used in the military and who should call the shots. 

After Claude played a role in the January military operation that captured Venezuelan President Nicolás Maduro, an Anthropic employee asked a Palantir counterpart how Claude was used, the Journal previously reported.

Defense Department officials found out and were upset, people familiar with the matter said. Anthropic has said it was a routine call between partners. 

Emil Michael, the undersecretary of defense for research and engineering, said Tuesday at an event hosted by venture firm Andreessen Horowitz in Washington that a “chill goes up your spine” when a company asks a question like that. 

“The soul of their model, their constitution, which is not the U.S. Constitution, can’t be dictating our command and control environment, telling generals and war fighters what to do and not do,” Michael said. “If you don’t believe in the system, as imperfect as it is, then, what do you believe in?” Michael said. Strikes on Iran over the weekend were also planned with the involvement of Anthropic’s Claude models, the Journal reported.