A Blog by Jonathan Low

 

May 22, 2019

Why CEOs Are Embracing 'Net Promoter Score' As A Performance Metric

Fools' gold? False hope? Distraction? Misinformation? Fad: as in another dubious metric like TQM, CRM and others flogged by management consultants? 

Maybe. Or maybe its another signal to both Wall Street (as well as its avatars like the Wall Street Journal, which fears it's own money-making metrics will be rendered irrelevant) and the accounting industry that industrial age GAAP accounting is increasingly meaningless as a management tool and that unless they take action to reform financial and operational management reporting, leaders can, will - and should - embrace measures that more accurately reflect their organization's reality. JL


Khadeeja Safdar and Inti Pacheco report in the Wall Street Journal:

The score is derived from customer responses to a question at checkout, in email or online: On a scale of 0 to 10, how likely are you to recommend the company’s product or service to a friend? NPS is based on the premise that every company’s customers can be divided into. “Promoters,” or loyal enthusiasts who keep buying, “detractors,” or “passives." The score is determined by subtracting the percent of  detractors from promoters.Last year, “NPS” was cited  150 times in earnings calls by S&P 500 companies (and) in 56 proxy filings (including) American Express, Delta, Best Buy, Intuit, Target, United Health Citigroup, IBM, Schwab.
Best Buy and American Express use it to dole out employee bonuses. Target and Intuit point to it to justify investments. Delta Air Lines and UnitedHealth can’t stop talking about it.
Much of Corporate America is obsessed with its net promoter score, or NPS, a measure of customer satisfaction that has developed a cultlike following among CEOs in recent years. Unlike profits or sales, which are measured and audited, NPS is usually calculated from a one-question survey that companies often administer themselves.
Last year, “net promoter” or “NPS” was cited more than 150 times in earnings conference calls by 50 S&P 500 companies, according to a Wall Street Journal analysis of transcripts. That’s more than four times as many mentions, and nearly three times as many companies, compared with five years earlier.
Executives pointed to strong or rising NPS as proof that shoppers preferred to pick up orders at Target Corp. stores or that Google’s newest Pixel smartphone was off to a good start. Out of all the mentions the Journal tracked on earnings calls, no executive has ever said the score declined.
Dozens of public companies are reporting the score in securities filings. Last year, “net promoter” was mentioned in 56 proxy filings. Some companies, including American Express Co., Best Buy Co. and Citigroup Inc., list the metric as a criterion for executive compensation, alongside traditional measures such as revenue growth and earnings per share.
The score was introduced in 2003 in a Harvard Business Review article titled “The One Number You Need to Grow.” The Bain & Co. consultant who wrote the article called NPS the “simplest, most intuitive and best predictor of customer behavior” and a “useful predictor of growth.”
Since then, the metric has taken on a life of its own, so much so that the inventor, Fred Reichheld, said he is astonished companies are using NPS to determine bonuses and as a performance indicator. “That’s completely bogus,” Mr. Reichheld, who still consults for Bain, said in an interview. “I had no idea how people would mess with the score to bend it, to make it serve their selfish objectives.”
The score is typically derived from customer responses to a single question that companies ask at the checkout register of a store or in an email or web pop-up online: On a scale of 0 to 10, how likely are you to recommend the company’s product or service to a friend? The survey usually includes a follow-up question asking customers to explain their ratings.
NPS is based on the premise that every company’s customers can be divided into three groups. People who answer 9 or 10 are “promoters,” or loyal enthusiasts who keep buying. Those who give a score of 0 to 6 are “detractors,” or unhappy customers. Those who answer 7 or 8 are considered “passives,” satisfied but easily wooed by competitors.
The score is determined by subtracting the percentage of customers who are detractors from those who are promoters, with the result falling in a range between -100 to 100. Passives are ignored in the calculation.
Tracked like ‘gospel’
Management consultants are notorious for pushing ideas to CEOs using jargon and claims of improved business performance. Total quality management, or TQM, which advocated installing quality programs at companies, and business re-engineering process, or BRP, which was a way to restructure companies, gained traction in the 1990s and then faded. NPS has outlived such fads, spawning a cottage industry of consultants and software firms that help businesses implement and boost their score.
Some academics have questioned the whole idea, suggesting that NPS has been oversold. Two 2007 studies analyzing thousands of customer interviews said NPS doesn’t correlate with revenue or predict customer behavior any better than other survey-based metric. A 2015 study examining data on 80,000 customers from hundreds of brands said the score doesn’t explain the way people allocate their money.
“The science behind NPS is bad,” said Timothy Keiningham, a marketing professor at St. John’s University in New York, and one of the co-authors of the three studies. He said the creators of NPS haven’t provided peer-reviewed research to support their original claims of a strong correlation to growth. “When people change their net promoter score, that has almost no relationship to how they divide their spending.”
Some data scientists said the way NPS is calculated, in which one survey metric is subtracted from another, increases the margin of error and requires a larger sample size to get useful results.
“It’s common for companies to track NPS data as if it’s gospel—not knowing that it’s super noisy by design,” said Kim Larsen, who has worked as a data scientist at several companies, including Charles Schwab Corp.
Bain, which now refers to NPS as “net promoter system,” said some companies are focusing too heavily on the score, but still defended the approach for some practical benefits. It is simple to communicate to employees, provides an easy way to follow up with customers and can be used to benchmark against rivals. The firm also said third-party analyses, including the 2007 studies, of whether NPS correlates with revenue aren’t as good as the analyses companies conduct internally.
“These are not stupid people. They are running large, successful companies,” said Rob Markey, a Bain partner who helps clients use NPS. “They have demonstrated to their own satisfaction that it’s good.”
Among the first companies to implement NPS were General Electric Co., Intuit Inc. and Schwab, whose leaders were convinced of the benefits after meeting with Mr. Reichheld and other Bain consultants. Now, hundreds of companies are using the score and many have tweaked the methodology, such as making the numerical scale 1 to 5 or including additional survey questions.
International Business Machines Corp. said it switched from a three-question survey to NPS in 2015. Employees in different departments can see the NPS feedback on their phones. “What it’s become here is a shared truth,” said Kathy McGettrick, vice president of market development and insights at IBM.
Some NPS users in the Journal analysis said the score correlates with revenue growth, though no company would disclose data to prove that point. Several companies said NPS is just one of many metrics they use to make decisions and that it helps them improve products or services.Intuit, which makes QuickBooks and TurboTax, began asking its customers the NPS question around 2003. Shortly after, CEO Steve Bennett attributed the company’s growth in self-prepared taxes to its net promoter efforts, since feedback had spurred improvements in some products.
Intuit has referred to NPS more than 60 times on earnings calls since the survey was first implemented. The company has used it to explain investments, including its 2007 acquisition of web-hosting company Homestead Technologies and the expansion of the ad budget for its QuickBooks Online service last year.
“We have not intended to imply that NPS is related to, or a driver of, revenue,” said an Intuit spokeswoman, adding that NPS is one of many metrics the company uses. “It allows our teams to gather customer insights so they can make decisions internally to deliver customer benefits.”
Mr. Bennett, who left Intuit at the end of 2007, later went on to serve as CEO of antivirus software maker Symantec, where he asked employees to stop using NPS, according to people familiar with his tenure. In response to the Journal, Mr. Bennett said he came to learn that the score is less meaningful than the open-ended question that can follow the rating.
“A big challenge with the methodology is that organizations tend to focus on the metric as the objective instead of gaining the insight to learn and act on to improve the customer experience,” he said. “When organizations manage to the metric, they find ways to game the system.”
The results are easy to manipulate, whether intentionally or unintentionally. On Reddit posts, Best Buy employees share tips and tricks to improve NPS, which the company derives from a random sample of customers. They said they can get better results when they explain to customers how the scoring works, or tell them their compensation is connected to the result. Some said they remind only the happiest customers to take the survey.
“When horrible NPS comments would come in, the management would rail at the employees,” said Alan Sabido, a former Best Buy employee who worked at a Las Vegas store for three years until he quit last year. Mr. Sabido recalled an instance when his store team received a bad score because a customer had a poor experience at a different Best Buy location.
NPS took on a greater role after Hubert Joly joined as Best Buy’s chief executive in 2012. The company said it was administering the NPS survey question to customers who bought products as well as those who didn’t. Best Buy also made the metric one of the criteria used to determine bonuses.
Since then, Best Buy has mentioned NPS or net promoter more than 50 times on earnings calls. The company has created a team of employees at its Richfield, Minn., headquarters called “Enterprise NPS” and some of its store workers are also tasked with “driving positive NPS results,” according to job postings.
“We use a number of methods to measure customer satisfaction and NPS is just one of them. We are aware of its limitations but believe it is valuable,” said a Best Buy spokesman. “Our revenue and earnings have increased at the same time we’ve seen NPS improvements.”
Criteria for bonuses
When Mr. Joly announced he was stepping aside as CEO in April, Best Buy listed among his accomplishments that NPS grew by three points in the past fiscal year.
In securities filings, Delta Air Lines Inc. has said it measures “customer service performance” as the percentage point improvement in its average NPS. The methodology, the company said, was approved by a board committee and its progress is reported periodically to the board. NPS is also listed as a criterion for executive bonuses.
Carol Campbell, Delta’s managing director of customer experience, said most of her work entails using NPS to find opportunities to make investments. The airline added fresh-baked cookies on trans-Atlantic flights, chose to put nine seats in each Boeing 777 row instead of the standard 10 and made enhancements to its app because of NPS.
Delta executives describe NPS as the “true North Star,” she said, though the airline uses other customer metrics as well. “We have been able to statistically correlate our NPS performance with our revenue premium,” she said, referring to how much more Delta is able to charge than a competitor because of its brand.
UnitedHealth Group Inc. has mentioned NPS more times on earnings calls than any other S&P 500 company in the Journal’s analysis. In April, CEO David Wichmann said the insurer’s net promoter scores “continued to advance meaningfully in the first quarter 2019 as we march toward an aggressive target of 70 by 2025.”
“UnitedHealth Group uses NPS as an important measure to understand and continually improve upon the quality of the experience for those we serve,” a spokesman for the company said.
It’s hard for investors to interpret the score because companies don’t typically share response rates, margin of error, or whether results are adjusted for cultural and other biases. Research shows NPS tends to be higher when response rates are lower, and Americans tend to give higher scores than consumers in some countries such as Japan and Korea.
Since so many NPS surveys are sent to customers through emails, web pop-ups and phone calls, it has become harder to elicit a response these days, survey firms said. Email response rates are usually less than 5%, according to an estimate by Qualaroo, a software startup that helps companies conduct user research.
Software providers like Medallia and Qualtrics said they help companies get higher response rates. Satmetrix, which originally helped Mr. Reichheld develop the score and is now a division of software-provider NICE, said it sells access to the “world’s best net promoter certification program.” J.D. Power said it signed an agreement with Bain to “become the officially recognized authority for benchmarking the Net Promoter Score.”

Methodology

To find the number of mentions of the terms “net promoter” or “NPS,” The Wall Street Journal used transcripts of company conference calls discussing quarterly earnings, retrieved from Factiva, a business research tool owned by WSJ publisher Dow Jones & Co.
The analysis started with more than 40,000 transcripts for calls held from 2003 through 2018, for 688 existing companies that were members of the S&P 500 index at any point during that period, as indicated by S&P Dow Jones Indices.
The Journal analyzed transcripts from companies that are still publicly traded. It excludes 209 companies that were acquired during the period but includes 188 that left the index yet remain public.
Nearly 400 transcripts contained at least one of the terms. A paragraph was counted as a single mention if it contained one or more uses of the terms. Portions of the transcripts consisting of questions from analysts were ignored.

0 comments:

Post a Comment