Prediction Markets To Predict Behavior?

In previous elections, prediction markets were relatively accurate and were touted as competitors to public opinion polling. So how did they do this time?

The Iowa Electronic market had two prediction markets concerning the Presidential election. One was for the percentage of the popular two-party vote, which over the course of betting predicted Clinton 50% and Trump 48%. [These were individual contracts, which may be why the numbers add up to more than 100.] According to the most recent actual vote count, the result of the two-party split was Clinton 51% and Trump 49%.

The other was for the winner of the popular vote, which over the course of betting was 97% for Clinton and 1% for Trump. This was correct as current estimates show her getting over two million more votes than him. 

Alas, winning the popular vote wasn’t enough this time and this was where the prediction markets seem to have run into a problem.

In one of the few markets that focused on electoral votes, a German betting market ended up predicting Clinton 300, Trump 237. (The real result was almost the reverse.)

PredictWise’s betting market had Clinton “winning” with an 86% probability. (In their defense, of course, that also means a 14% chance for Trump, which has to happen some time if we’re talking probability, not certainty, after all.)

image

The folks at the Campaign Workshop observed:

“Polls aren’t perfect, but neither are political betting markets. Since these markets have gained credibility in predicting elections, they have started taking changes in public opinion polls less seriously. Overconfidence in betting markets makes the markets look misleadingly stable, and that false sense of stability makes it harder for them to predict events that shake up the status quo — such as the outcome of the Brexit referendum, or Trump’s success in the Republican presidential nomination process. As Rothschild himself has pointed out, ‘prediction markets have arrived at a paradoxical place: Their reliability, the very source of their prestige, is causing them to fail.’ ”

In looking at these markets and, more generally, crowd predictions of events, it’s worth going back to James Surowiecki’s book, “The Wisdom of Crowds”. He described both the rationale for prediction markets — which have been well publicized — and the characteristics of accurate prediction markets — which have received less emphasis.

“The premise is that under the right circumstances, the collective judgment of a large group of people will generally provide a better picture of what the future might look like than anything one expert or even a small group of experts will come up with. … [Prediction markets] work much like a futures market, in which the price of a contract reflects the collective day-to-day judgment either on a straight number—for instance, what level sales will reach over a certain period—or a probability—for example, the likelihood, measured as a percentage, that a product will hit a certain milestone by a certain date.”

“[F]or a crowd to be smart, it needs to satisfy certain criteria. It needs to be diverse, so that people are bringing different pieces of information to the table. It needs to be decentralized, so that no one at the top is dictating the crowd’s answer. It needs to summarize people’s opinions into one collective verdict. And the people in the crowd need to be independent, so that they pay attention mostly to their own information and don’t worry about what everyone around them thinks.”

image

Did the prediction markets in 2016 meet Surowiecki’s criteria? Not really.

One problem with betting markets is that they are not diverse, not representative of a broad spectrum of the population. As a CNBC report noted: “Another issue that may have contributed to the miss [on Brexit and now the US election] is the relatively similar mindset among bettors generally.”

Since all bettors can see what others seeing, it’s hard to argue that their judgments are independent. And while, in a way, the decisions are decentralized, to the extent they mirror the current polling results and news reports from national media, there is less decentralization.

So do we just decide that the results of this year’s election call into question the value of crowd predictions? I think not.

But rather than focusing on predicting who wins the White House or the Super Bowl or the number of coins in a large bottle, there is another use of prediction markets for business and government leaders — testing the likelihood that people will respond positively to a new program or offer.

No matter how much market research (aka polling) is done, it is often difficult to assess how the public will react to a proposed program. I’m suggesting that prediction markets be used to estimate the reaction ahead of time, as long as they match Surowiecki’s criteria and don’t depend on money bets. At the very least, this would require a large and diverse set of people responding and keeping their judgments secret (until “voting” stops).

Over the last year or so, there have been several reports that rates for Affordable Care (aka Obamacare) had to be raised because there are fewer young, healthy people enrolling than expected. Putting aside the merits of the policy and its goals, this is an ideal case where prediction markets could have helped assess the accuracy of an underlying assumption about the implementation of a very consequential piece of public policy.

Some experts are skeptical of prediction markets because the average person doesn’t have professional expertise. But this use of prediction markets draws on the perceptions of people about each other.

Implicit in the diversity of views that Surowiecki notes is that enough people need to care about the planned program or policy. The reason they care may be to win money, in some cases, but that’s not the only reason. They might care because the market deals with something that affects their lives.

And the nice thing about this is that if only a few people care about a planned program that also tells you something about that plan — or, at least, whether the range of outcomes might be something between a yawn and deep trouble.

It may well be that this more experimental basis to predict behavior will illustrate the deeper value of prediction markets. What do you think?.

© 2016 Norman Jacknis, All Rights Reserved

[http://njacknis.tumblr.com/post/153863533555/prediction-markets-to-predict-behavior]

Can Prediction Markets Test Public Policy?

Prediction markets are hot again.  With the Presidential election next week and the relative accuracy of prediction markets about the election in 2008, I decided to finish this article that I first started in March 2009, but put aside for various reasons.

What is a prediction market?  James Surowiecki, the author of the book, “The Wisdom of Crowds” explains:

The premise is that under the right circumstances, the collective judgment of a large group of people will generally provide a better picture of what the future might look like than anything one expert or even a small group of experts will come up with. [Prediction markets] work much like a futures market, in which the price of a contract reflects the collective day-to-day judgment either on a straight numberfor instance, what level sales will reach over a certain periodor a probabilityfor example, the likelihood, measured as a percentage, that a product will hit a certain milestone by a certain date.

[F]or a crowd to be smart, it needs to satisfy certain criteria. It needs to be diverse, so that people are bringing different pieces of information to the table. It needs to be decentralized, so that no one at the top is dictating the crowds answer. It needs to summarize peoples opinions into one collective verdict. And the people in the crowd need to be independent, so that they pay attention mostly to their own information and dont worry about what everyone around them thinks.

These markets have been around for a few years and have a good, if not perfect, track record.  Various studies have found prediction markets to be better than experts or public opinion polling.

But my focus is not on predicting who wins the White House or the SuperBowl or the number of coins in a large bottle.  Rather there is a use of prediction markets for government leaders testing the likelihood that a particular public policy will achieve success, especially if the policy is intended to change the behavior of people.

Often it is difficult to assess how the public will react to a proposed policy will people actually sign up for program X; will people recycle; etc.  I’m suggesting that prediction markets be used to estimate the reaction ahead of time.

Implicit in the diversity of views that Surowiecki notes is that enough people need to care about the policy.  The reason they care may be to win money, in some cases, but thats not the only reason.  They might care because the market deals with something that affects their lives.

And, the nice thing about this is that if only a few people care about a policy that also tells you something about the policy or, at least, whether youll get into deep trouble proposing it.

You can learn from prediction markets and they may be more effective predictors of policy success than traditional tools.

So far I can tell this has not really been tried yet in the way I’m describing.  The closest was in 2003, when DARPA created a Policy Analysis Market to be “a market in the future of the Middle East”.  That market looked, to critics, like a betting parlor on political assassinations by terrorists and was withdrawn.

In a more futuristic vision, Robin D. Hanson, an economist at George Mason University and a fan of alternative institutions, writes about futarchy, “a form of government enhanced by prediction markets. Voters would decide broad goals of national welfare, but betting in speculative markets would determine the policy steps to achieve those goals.”

I’m not proposing anything so revolutionary as he is.  Instead, let’s try to use prediction markets on a more experimental basis.

More background on prediction markets can be found in:

  • Wikipedia

http://en.wikipedia.org/wiki/Prediction_market and

http://en.wikipedia.org/wiki/Policy_Analysis_Market

  • When the Crowd Isnt Wise

http://www.nytimes.com/2012/07/08/sunday-review/when-the-crowd-isnt-wise.html

  • Interpreting the Predictions of Prediction Markets

http://scholar.google.com/scholar_url?hl=en&q=http://citeseerx.ist.psu.edu/viewdoc/download%3Fdoi%3D10.1.1.159.7574%26rep%3Drep1%26type%3Dpdf&sa=X&scisig=AAGBfm3lg0YqMPEHYfN6LogKNPFcL1GZww&oi=scholarr

© 2012 Norman Jacknis

[http://njacknis.tumblr.com/post/34709617211/can-prediction-markets-test-public-policy]