The Scoreboards Ranking Us That We Cannot See

We know that a lot of enterprises and institutions are using our data. By and large, as a society, we understand that and have made peace with it - in return for the ostensible advantages of convenience and the occasional discount.

And despite some latterly attempts to limit - or at least to create the impression that they are limiting that use - the big tech companies have been happily compliant in the business of converting data into knowledge that is then monetized through advertising or marketing or sales.

But despite this generalized knowledge about the sources and uses of data, very few of us understand how this information is compiled and applied. The problem, as the following article explains, is that if we did know, we might demand far more stringent limits on the use of this information.

What is of even greater concern is the fact that assessments are being made that involve data consumer/citizens don't even have to provide or approve, tacitly or actively. Because the reality is that these rankings and the judgements that follow are increasingly done technologically. There appears to be little, if any, oversight. And no recourse should the individual actually gain access and find out that the information is wrong.

The system has worked, so far, because most people are simply ill-equipped or untrained or too cowed or preoccupied to protest. Given the growing role that this data and the decisions that flow from it increasingly play in most lives, no one - consumer or merchant - should assume that this state of affairs will be sustainable, let alone acceptable. JL

Natasha Singer reports in the New York Times:

Most people in real life are not aware of the types and frequency of rankings to which they are subject. “This will happen whether or not you want to participate, and these scores will be used by others to make major decisions about your life, such as whether to hire, insure, or even date you."

The characters in Gary Shteyngart’s novel “Super Sad True Love Story” inhabit a continuously surveilled and scored society.

Consider the protagonist, Lenny Abramov, age 39. A digital dossier about him accumulates his every health condition (high cholesterol, depression), liability (mortgage: $560,330), purchase (“bound, printed, nonstreaming media artifact”), tendency (“heterosexual, nonathletic, nonautomotive, nonreligious”) and probability (“life span estimated at 83”). And that profile is available for perusal by employers, friends and even strangers in bars.

It’s a fictional forecast of a data-deterministic culture in which computer algorithms constantly analyze consumers’ profiles, issuing individuals numeric rankings that may benefit or hinder them.

Observing a street billboard that publicly broadcasts the score of each passer-by, the Abramov character says in the novel, “The old Chinese woman had a decent 1,400, but others, the young Latina mothers, even a profligate teenaged Hasid puffing down the street, were showing blinking red scores below 900, and I worried for them.”

In two nonfiction books, scheduled to be published in January, technology experts examine similar consumer-ranking techniques already in widespread use. Even before the appearance of these books, a report called “The Scoring of America” by the World Privacy Forum showed how analytics companies now offer categorization services like “churn scores,” which aim to predict which customers are likely to forsake their mobile phone carrier or cable TV provider for another company; “job security scores,” which factor a person’s risk of unemployment into calculations of his or her ability to pay back a loan; “charitable donor scores,” which foundations use to identify the households likeliest to make large donations; and “frailty scores,” which are typically used to predict the risk of medical complications and death in elderly patients who have surgery.

Unlike Lenny Abramov, however, most people in real life are not aware of the types and frequency of rankings to which they are subject. While a federal law called the Fair Credit Reporting Act requires consumer reporting agencies to provide individuals with copies of their credit reports on request, many other companies are free to keep their proprietary consumer scores to themselves.

“This will happen whether or not you want to participate, and these scores will be used by others to make major decisions about your life, such as whether to hire, insure, or even date you,” write Michael Fertik and David Thompson in a forthcoming book, “The Reputation Economy: How to Optimize Your Digital Footprint in A World Where Your Reputation Is Your Most Valuable Asset” (Crown Business). Mr. Fertik is the chief executive of Reputation.com, a service that helps individuals and companies manage their online images.

In his new book, Frank Pasquale, a law professor at the University of Maryland, similarly describes the information asymmetry inherent in the scoring industry.

“Important corporate actors have unprecedented knowledge of the minutiae of our daily lives,” he writes in “The Black Box Society: The Secret Algorithms That Control Money and Information” (Harvard University Press), “while we know little to nothing about how they use this knowledge to influence important decisions that we — and they — make.”

Both books outline how consumer scoring works. Data brokers amass dossiers with thousands of details about individual consumers, like age, religion, ethnicity, profession, mortgage size, social networks, estimated income and health concerns such as impotence and irritable bowel syndrome. Then analytics engines can compare patterns in those variables against computer forecasting models. Algorithms are used to assign consumers scores — and to recommend offering, or withholding, particular products, services or fees — based on predictions about their behavior.

But while both books emphasize the notion that consumer reputations are vulnerable to such covert scoring apparatuses, the authors differ markedly in the steps they say ordinary people might take to protect themselves.

Befitting the founder of a firm that markets reputation management, Mr. Fertik contends that individuals have some power to influence commercial scoring systems. He presents nascent technologies, like online education courses that can score people on the specific practical skills or concepts they have mastered, as democratizing forces that could enable workers to better compete for jobs on merit. His book suggests that readers curate, or hack, their digital reputations — for instance, by emphasizing certain keywords on their résumés to position them better for predictive scoring engines, or by posting positive reviews of restaurants or hotels online, in the hope that algorithms will flag them for future V.I.P. treatment.

“Employers’ algorithms will pick your résumé out of the pile of thousands just as instantaneously and robotically as they pass over others,” he and his co-author write. “Banks and lenders will automatically approve you for the better rates and offers. The more appealing dates on apps and sites like Tinder, Match and OkCupid will see your profile before they see any others.”

Think of this technique as reputation engine optimization. If an algorithm incorrectly pegs you as physically unfit, for instance, the book suggests that you can try to mitigate the wrong. You can buy a Fitbit fitness tracker, for instance, and upload the exercise data to a public profile — or even “snap that Fitbit to your dog” and “you’ll quickly be the fittest person in your town.”

Professor Pasquale offers a more downbeat reading. Companies, he says, are using such a wide variety of numerical rating systems that it would be impossible for average people to significantly influence their scores.

“Corporations depend on automated judgments that may be wrong, biased or destructive,” Professor Pasquale writes. “Faulty data, invalid assumptions and defective models can’t be corrected when they are hidden.”

Moreover, trying to influence scoring systems could backfire. If a person attached a fitness device to a dog and tried to claim the resulting exercise log, he suggests, an algorithm might be able to tell the difference and issue that person a high score for propensity toward fraudulent activity.

“People shouldn’t think they can outwit corporations with hundreds of millions of dollars,” Professor Pasquale said in a phone interview.

Consumers would have more control, he argues, if Congress extended the right to see and correct credit reports to other kinds of rankings.

“If credit scores can be regulated,” he says, “why not the scoring systems used by digital advertisers and employers?”