Why Prediction Markets Are Bad At Predicting Who'll Be President

Current prediction markets are too small to scale - and there is not yet enough of a financial reward to encourage greater participation - which makes them prone to randomness and inaccuracy. JL


Kelsey Piper reports in Vox:

Prediction markets were supposed to be smarter than the pundits. They were supposed to harness the wisdom of crowds and use financial incentives to be as accurate at predicting global events as the stock market is at predicting earnings for public companies. Existing prediction markets are too small in scale, hard to interact with, and hard to make money from, which renders them inaccurate and vulnerable to manipulation.

If you look at predictions from British gambling site Betfair, the Democratic primary has a new top-tier candidate: Mike Bloomberg. His odds of winning the Democratic nomination shot up suddenly in the middle of this week, and on February 14 he briefly passed Bernie Sanders — who has won the popular vote in the two states that have voted so far and is leading in national polls — as the candidate likeliest to win the nomination and likeliest (behind Trump) to be our next president. At the peak on Valentine’s Day, bettors on Betfair gave Bloomberg a 34.5 percent chance of winning the nomination.

FiveThirtyEight’s sophisticated election model, on the other hand, rates Bloomberg’s odds of getting a plurality of delegates at 15 percent (plus some chance he’s chosen at a brokered convention), behind Biden as well as Sanders. Most experts aren’t rating him much higher than that. The consensus is that he does have a shot, but he’s far from the easy frontrunner.

Do the prediction markets know something experts don’t?

Maybe.

But anyone who has been watching the prediction markets for the last year might have an alternative hypothesis: They’re just not very good. Before Iowa, Betfair, and competitor PredictIt gave Pete Buttigieg only an 8 percent chance of winning the most state delegate equivalents (which he did, pending a recanvass). For much of last fall, PredictIt rated Andrew Yang and Hillary Clinton as tied for third in the nomination race (neither stood a chance, and Clinton wasn’t even running). In 2016, Betfair’s final assessments of candidates in elections that year were overall very good. But there can be a lot of noise — and a lot of nonsense — along the way.

The thing is, prediction markets were supposed to be smarter than the pundits. They were supposed to harness the wisdom of crowds and use financial incentives to be as accurate at predicting global events as the stock market is at predicting earnings for public companies.

If they work at that, we’ll have a powerful new tool for making policy. If we had an effective prediction market, we could use it to aggregate wisdom on questions like the revenue effects of tax changes, the mortality effects of new health care policies, or the best response to an emerging threat like coronavirus.

Stock markets are really good at quickly aggregating information, with prices rapidly reflecting new information and lots of highly paid people spending much of their time studying companies in order to take bets on which are overpriced and underpriced. If prediction markets were similarly effective, we’d have a reliable source of information about lots of things. And if you thought it was wrong, you could just bet against it — moving the price and making money if you were right.

So it’s frustrating that the presidential prediction markets — the most visible, most-traded efforts at making prediction markets happen on a large scale — don’t seem so great at their jobs. What went wrong?

Existing prediction markets are too small in scale, hard to interact with, and hard to make money from, which renders them inaccurate and vulnerable to manipulation. If we want to figure out whether the prediction market concept works, we should try it with prediction markets that are more like the stock market. Until we do, prediction markets are potentially another channel for misinformation.

Are presidential prediction markets bad at their jobs?

The presidential prediction market has been a little zany all primary season. Most notably, it consistently overrated Andrew Yang, who had lots of tech-savvy, online followers who were exactly the sort of people to use prediction markets. That’s likely why the markets consistently rated him as having about a 10 percent chance of winning the nomination, even while polls and experts called it correctly: Yang was never going to win a delegate.

One market, PredictIt, also spent much of last fall inexplicably rating Hillary Clinton a plausible nominee with 11 percent odds of winning the nomination. The markets also consistently underrated Pete Buttigieg, giving him only an 8 percent chance of winning the most state delegate equivalents in Iowa.

And now they’ve become obsessed with Bloomberg over the space of a few days, with his odds on Betfair rocketing from 11 percent on February 6 to 34 percent on February 14 and then down somewhat, despite not much new information coming out that should have improved his prospects.

Predictions from Betfair on February 15.

 Electionbettingodds.com

This is, frankly, a weird prediction. Failing to compete in the early states has never worked for a candidate before. Bloomberg has risen in the national polls, but he’s still behind Sanders; aggregates of recent national polling found Sanders at 23 percent and Bloomberg at 15 percent, putting him not only behind Sanders but also behind Joe Biden.

Contrast the Betfair odds model with the model from FiveThirtyEight, the data-driven pundits who’ve had an unusually strong track record over the past several election cycles.

FiveThirtyEight Democratic delegate odds on February 15.

 FiveThirtyEight

The model makes it perfectly clear that Bloomberg has a real shot. His chances have risen recently as his spending blitz drove a rise in the national polls. But he is much less likely to win than Sanders, as you might expect given that Sanders is leading in national polls, leading in state polls, and has won the popular vote in the states that have voted so far.

(Do note that this is based on a FiveThirtyEight model estimating who will get a plurality of pledged delegates. Bloomberg could fail to have a plurality but win a brokered convention, making his overall odds higher than the 15 percent given here. But factoring this in certainly wouldn’t make him the frontrunner; experts think it’s unlikely the DNC will ignore the plurality choice at a brokered convention, and FiveThirtyEight gives Sanders a 36 percent chance of having an outright majority by the convention.)

It remains to be seen who will be right, of course. But the available evidence is more in line with the FiveThirtyEight model — where Bloomberg is a possibility but not the frontrunner — than the model the betting markets seem to prefer, where Bloomberg is much more likely to be the Democratic nominee.

Prediction markets are manipulable

It is hard to say for sure what is driving Bloomberg’s rise. One possibility — and to be clear, there’s no evidence for this — is that someone with a lot of money is betting on him.

Prediction markets are driven by supply and demand. If there’s an endless supply of people — or one very rich person — willing to bet on Bloomberg, that will drive up the price for Bloomberg bets. That, in turn, will cause the market to report excellent odds he’ll win the election. The hope is usually that the good predictions will increase good publicity and eventually be a self-fulfilling prophecy — that all of us will see that Bloomberg is the frontrunner according to Betfair and start treating him as one.

That something like that could happen isn’t just speculation — in 2012, it actually happened. At the time, the leading presidential prediction market was a site called Intrade, and it consistently overrated Mitt Romney, who lost in a landslide. Researchers looking into the prediction markets found evidence of one bettor who spent $4 million to $7 million artificially boosting Romney’s chances. He was so determined that he ended up being one-third of all Intrade trading on Romney — and he kept Romney’s prediction-market odds high right up through the election.

“It is worth knowing that a highly visible market that drove many a media narrative could be manipulated at a cost less than that of a primetime television commercial,” the study’s authors, David Rothschild of Microsoft Research in New York and Rajiv Sethi of Barnard College at Columbia University, wrote at the time.

People try to do this on the stock market, too. But they mostly fail because stock markets are big and heavily traded. It takes a ton of money — much more than $7 million — to meaningfully manipulate the price of a stock, and the manipulation often doesn’t last very long.

By comparison, prediction markets are small and not very lucrative. High fees make it a waste of time to bet on some contracts, especially ones for unlikely outcomes. Your money is sometimes tied up in the market until the contract is resolved, which can take years. Betfair is closed to Americans due to US gambling laws, and PredictIt (which is a university research project and thus has an exemption from normal gambling law) is closed to non-Americans, and PredictIt sharply limits how much you can bet.

As a result, they’re dominated by hobbyists — which needn’t be a problem as election betting is a fairly boring unobjectionable hobby. Usually, the hobbyists are enough — prediction markets actually do pretty well! But since the markets are small, it doesn’t take all that much money or a very sophisticated operation to manipulate them.

The fact prediction markets are unprofitable isn’t itself a big deal. But the combination of unprofitable and manipulable means they can become a tool for disinformation — and that’s bad.

The fix: Prediction markets should be a little more like stock markets

There are a lot of reasons I’d have a harder time manipulating the price of Google stock than manipulating the price of “Buttigieg 2020 nominee” contracts on Betfair. Let’s go through them in turn.

First, so much money moves through the stock market that lots of people are paid to buy and sell stocks as a full-time job. If I try to dump some money into changing Google’s stock price, some of those people will notice and try to figure out what’s up. Some of them have access to billions of dollars themselves, so I will have a very hard time throwing enough money in to swamp them all.

Second, there are low fees associated with trading stocks. I can buy and sell lots of Google stock with only some minor fees (at least as a share of the size of the trade) associated with doing so. Fees on prediction markets, on the other hand, often make up a significant share of the profits you’d get from trading on them. This makes repeatedly trading less worthwhile.

Third, most stock trading is public, so if I bought billions of dollars of Google stock, people could notice that I had personally bought billions of dollars of Google stock. Right now, it’s hard to figure out who, if anyone, is driving the dramatic increase in Bloomberg bets.

Finally, some forms of market manipulation in the stock market are heavily regulated. Try to trigger a flash crash or release a misleading press release so that you can take advantage of price movements, and you’re pretty likely to be investigated by the Securities and Exchange Commission. While no regulatory environment is perfect, some amount of oversight can make the markets function better by making the people you’re trading with more confident that they aren’t being duped.

Right now, the regulatory stance on election prediction markets is that they’re illegal gambling. A move in this direction would require Congress or regulators to loosen up the rules — but they could still regulate the markets, just with an eye toward preventing scams rather than preventing all betting.

A step in any of these directions would make the prediction markets harder to manipulate as well as a better use of time and energy. They would probably also be more accurate. This is a goal worth pursuing. Correctly done prediction markets can improve policymaking, and bad ones can be an avenue for misinformation. We ought to think about how to get it right.