AI and Its Leaders Are Standing On A Legal Liability Landmine

OK, sure, human extinction could be a real bummer. But let's talk about a near-term financial risk closer to most tech execs' true nightmare vision: AI product liability. The field is nascent and few cases have been decided, but the potential financial impact is economically ruinous for AI companies and their investors.

Silicon Valley, naturally, wants the federal government to make it go away. But surprisingly, many in politics are pushing back despite the extraordinary tech 'contributions' being loaded into their coffers. Although the AI industry still refuses to acknowledge that they should be responsible for any cost they don't want to bear, here is going to have to be some sort of accommodation because if there is one US professional group greedier and more ruthless than those in tech, it is US plaintiffs attorneys. JL

Asa Fitch reports in the Wall Street Journal and Josh Marshall reports in Talking Points Memo:

As AI developers imbue their chatbots with more agent-like abilities—allowing them to do things autonomously on users’ behalf—questions about legal liability are becoming increasingly urgent. Liability is a legal and economic framework for properly assigning the gains and risks/costs associated with economic activity. They could lead to limits on what agents can do and, if AI-caused harms become widespread, drain the pocketbooks of top AI labs. The CEO of Palantir, suggested the liability problem was so dire that U.S. AI labs should be nationalized to protect themselves. AI labs (increase their liability risk) when their agents replace jobs and maximize profits for their owners. Laws which properly and securely assigned liability for AI would go a way to solving some of the problems caused by Silicon Valley and its concentrated wealth pushing all its downsides onto the public

What happens when an AI agent buys an unwanted product on someone’s behalf? What if an agent launches a cyberattack to obtain private information at the behest of a user?

As AI developers imbue their chatbots with more agent-like abilities—allowing them to do things autonomously on users’ behalf—questions about legal liability are becoming increasingly urgent.

The answers could be consequential, too. They could lead to limits on what agents can do and, if agent-caused harms become widespread, drain the pocketbooks of top AI labs.

Alex Karp, the chief executive of Palantir Technologies, suggested on CNBC last week that the liability problem was so dire that U.S. AI labs should ask to be nationalized to protect themselves from it.

Treasury Secretary Scott Bessent jumped into the debate Monday, saying it was the companies, not the government, that should be concerned about liability. OpenAI’s management should be held liable for its agents’ recent hack of Hugging Face, he said. The AI labs “need to take responsibility for themselves.”

Liability for the actions of AI agents is largely untested. There have been a smattering of cases that hint at possible outcomes, like the 2024 case involving an Air Canada chatbot that hallucinated that a discount was available for bereavement tickets. A Canadian tribunal decided the airline was responsible there.

More cases are sure to arrive soon. The rollout of agents like Meta’s Muse, which can send emails, make purchases and take a raft of other actions on users’ behalf, makes them all but inevitable. 

AI developers can to some extent shield themselves from liability through disclaimers and user agreements. But such agreements offer no protection in many instances where AI developers are held negligent, or where they cause harm because of flaws in their products. Users, of course, could also be held liable for using AI agents to harm others.

The scale of the financial impact depends to some degree on whether civil courts treat AI agents like hot air balloons or trains.

When hot air balloons emerged in the 19th century, courts attached a strict liability standard. That meant operators could be held liable for harms they caused no matter how much effort went into making their balloons safer.

Trains, as a recent academic paper lays out, came around about the same time but were given a negligence standard. That meant operators could shield themselves from much liability as long as they took steps to make them as safe as possible.

The AI industry would prefer to be treated like the railroads. But a guiding factor for courts in determining which standard applies is how useful the industry is to society.

As University of Washington professor and robot law specialist Ryan Calo points out, getting the negligence standard hinges on AI’s leaders showing courts that they—much like railroads—are too crucial to let liability concerns slow them down.

“The courts were saying, ‘Who is using hot air balloons?’ These eccentric wealthy people,” Calo said. “Trains were critical because they were thought to be crucial to American infrastructure catching up to Europe, which was already industrialized.”

Calo believes AI labs help themselves when their tools reduce people’s busywork—filling out routine forms autonomously, for example, saving time and effort. They do themselves fewer favors when their agents replace jobs and maximize profits for their owners, he argues. 

It seems unlikely that AI labs will take a path that reduces their liability exposure, though. Replacing human work is an attraction for corporate customers.

Either way, the period ahead will be an active one for lawyers and the courts.

---

Anthropic Pushes Back Its IPO

Anthropic CEO Dario Amodei. David Dee Delgado/Getty Images

It’s a November listing for Anthropic. That, at least, is the plan for now.

The much-anticipated IPO was initially planned for next month, but the company wanted to be able to put its third-quarter financial results on display for investors. Anthropic is also, of course, at the center of a debate about slowing the pace of AI development to deal with potentially catastrophic risks.

That debate shouldn’t put much of a dent in Anthropic’s revenue trajectory, given its rise is being fueled by corporate customers that aren’t likely to press pause on AI soon. Anthropic is already expected to reach a $110 billion annual revenue run rate by the beginning of next year. 

I wanted to step back for a moment and talk more generally about the question of "externalities" which has always been the problem lurking in the background of Big Tech's road to national and global domination.

Before getting into that, a subsidiary point. When you starting digging into debates about AI you realize there's just a wild cast of characters, groups, movements all arrayed around this technology, some boosters, others critics. A lot of AI critics point to the fact that a number of the leaders, particularly at Anthropic, are part of the so-called "rationalist" community, adjacent to "effective altruism." And these folks have kind of a cottage industry or cottage subculture which is rife with predicted future extinction events. Is this part of the driver of all these claims about being in a race with AI to make sure it doesn't decide to exterminate us? I'm really not sure. But it does seem to play some role. (This "rationalist" world — no I'm not totally sure how they got possession of this word — are big in the Valley.) Meanwhile, you've got another faction in the tech world, which thinks all that talk of extinction events is a bunch of culty nonsense and/or basically a ruse to allow companies like Anthropic to lock in their dominance as a kind government cartel.

And the interesting thing is that it's ... well, the evil guys, who are on that side of the argument. Particularly David Sacks, who is now Trump's AI advisor and before that and probably continuing is a like a professional Elon Musk fanboy and courtier. So in response to this proposed slowdown, Sacks put a post on Twitter which basically says, 'Great, you're pausing. But don't pretend you need to suspend anti-trust laws or that all these dangers you whine about can't be handled by normal market and liability mechanisms.'

"Stop pretending you need a regulatory approval process that supersedes product liability," to use his words.

Now, I'm certainly not taking the side of David Sacks here, who might best be described as Elon Musk without the charm or the money. But this issue of liability actually is a key one. Indeed it's not too much to say that the story of Big Tech over the last quarter century has been one of improperly accounted for externalities. We discussed this in an Ed Blog post ("Feral AI and the Question of Externalities") back in 2023 ...

One of the central dynamics of the Internet/digital technology age has been the issue of externalities. Facebook makes billions but leaves a path of destruction and dislocation in its wake that society has to grapple with and pay for. Some of this is just Schumpeterian creative destruction. New technologies and new businesses based on them make old ones obsolete and drive their ruin. We’ve broadly accepted this as a fact and a feature, albeit a disruptive one, of living in a capitalist, free society. But many are more like nuclear power plants that dump their used fuel rods in a local river. The issue isn’t capitalist disruption, it’s the privatization of profit and the socialization of risk. The rush to bring these tools to market is partly simple profit motive but, even more, something beyond that: the need to be first. Google at least sees the risk that its empire of search, which still drives most of its billions in profit, could be ripped from beneath it by Microsoft — which has the OpenAI franchise and is working to incorporate it into what has always been its sad-sack also-ran search engine, Bing. That’s existential. Hundreds of billions are potentially at stake for both companies. Being first can mean everything — as it did for Google a generation ago. But for society at large, there are other equities in the balance. And there are flashing warning signs here about the need to slow down.   From: Feral AI and the Question of Externalities by Josh Marshall

Liability is a legal and economic framework for properly assigning, properly tethering together, the gains and risks/costs associated with economic activity. It's true that at the extreme end, liability can become so onerous that society loses through lost innovation and economic growth. In practice that's seldom the problem. A huge, huge amount of the economic powerhouse of Silicon Valley and its concentrated wealth creation has been a matter of pushing off all its downsides onto the public, either collectively or individually. Nuclear power is super, super lucrative if you just fire up a reactor in your backyard with zero containment and throw away the spent fuel rods in the municipal garbage or the local lake./

With AI, laws which properly and securely assigned liability would go at least a decent way to solving some of these problems. The same investors pouring hundreds of billions into frontier AI labs would more clearly see how a few catastrophes could sweep away all that wealth in a moment. Needless to say, markets are not always rational. I would never claim otherwise. But the proper assignment of liability is certainly part of the equation.