The NPS (Net Promoter Score) survey has become very popular across many sectors, and numerous companies are embracing it. The best thing about NPS is that it’s easy to measure and requires little effort from customers, as it tends to ask only one question: How likely are you to recommend our company to a friend or colleague?
I was asked to analyse the company’s NPS survey, and in the process, learned so much and found it so interesting that thought many other companies thinking about introducing NPS would benefit from my findings. Even if you’ve already adopted it, it could explain why you are getting lower than expected scores.NPS works best with a high number of respondents (in the hundreds or thousands), ideally in B2C, and in the region where it was invented, the USA. If these conditions are not present, it’s likely that the results will be less positive.
Certain aspects of NPS are peculiar and have been criticised by some opponents. In my opinion, the four most negative aspects are:
1. 11-point response scale is too wide
Responses collected from a large, 11-point scale (0-10) are extremely noisy, and meaningful changes in ratings are hard to detect. On the NPS scale, the difference between a 6 and a 7 isn’t clear in the survey analysis, and the lack of labels on the intermediate (i.e. non-extreme) choices also make the distinction very subjective to respondents. The NPS scale is poorly calibrated, and so are the responses.
A better scale would use a three-option Yes/Maybe/No system, or even better a scale with five options. For any survey question, the response scale and number of options should be crafted to the individual question, and an 11-option scale is likely always too big.
2. The method of calculating the result is questionable
The method of calculation is one of the stranger aspects of the NPS.
The bucketing methodology that groups respondents into Promoters, Passives and Detractors, ends up hiding some improvements and exaggerating others. Even if respondents were able to make meaningful distinctions between a 5 and a 6, or between a 4 and a 5, the bucketing method categorises all of these into the Detractor group, and these changes aren’t reflected in the score. An extreme example is when a company with all 0 ratings improves to having a 6 rating; this is a huge improvement, but the NPS methodology means that the score doesn’t change at all. Some changes are also exaggerated by the method of calculation: the distinction between a 6 (detractor) and a 7 (passive), or between an 8 (passive) and a 9 (promoter) is exaggerated by the bucketing methodology.
Also, there is a lack of balance for an 11-point scale where only the last two points are positive (i.e. promoter).
3. Cultural differences
Anyone who has ever compared NPS scores in the US and Europe probably knows this very well. Europeans rate company performances very conservatively and are less likely to give 9s or 10s.
In Europe, children are usually graded on a scale of 0 to 10, and it’s almost impossible to get a 10. In Europeans’ minds, 8 is good, 9 is great and 10 is genius. So, when confronted with a classical 0-10 scale in NPS, survey respondents give you 8 even if they are satisfied.
In Japan, customers tend to give lower ratings as well, since it is considered poor etiquette to rate any business too high or too low, regardless of their performance. Americans, on the other hand, give higher scores than just about anyone else. It’s not surprising then, that NPS was developed in the US.
4. The phrasing of the question
The NPS question asks a respondent to rate the likelihood of a hypothetical future; but strong, reliable survey questions ask respondents about their past behaviours, which tend to be more predictive than forward-looking hypotheticals. “Do you plan to begin a diet in the next four weeks?”, for example, is a very different question from “Did you begin a diet in the last four weeks?” The NPS question forces the respondent to predict an ideal, future self, as opposed to reporting on their actualised behaviours.
These four aspects tend to greatly impact the results of NPS, but there are also three other influential factors worth mentioning:
5. Niche competition
Generally, NPS tends to be a better indicator in highly competitive verticals with many players, since it helps you assess relative performance.
If your business does not have many competitors, it could mean that most of your clients can’t compare you with a true competitor, and subconsciously could be benchmarking certain attributes against one of the new and shiny TSPs’ attributes (Technology and Service Provider).
6. Customer Tolerance Levels
Tolerance is another of the factors that affect Net Promoter Score benchmarks, as people are more likely to be opinionated depending on how much value the products and services deliver to them on a day-to-day basis or how much their business or life depends on it.
The tolerance level for a business can be measured by asking a simple question: “On a scale of 1-10, how likely are your customers to get mad if you can’t address their needs on an immediate basis?”
If the number is closer to 10, a business is in a low-tolerance industry towards service interruption, but if the number is closer to 1, you’re in a high-tolerance sector.
The easiest way to increase the tolerance level for a company is to transform the customer experience by providing more customer touchpoints, greater transparency and easier accessibility. Some of the best examples of companies that have managed to achieve high tolerance, despite being in a low-tolerance industry, are Uber, Southwest Airlines and Netflix.
7. Vendor-Switching Barriers
One of the reasons why non-SaaS businesses tend to fetch higher NPS than SaaS ones is because it’s easier to infuse brand loyalty and high tolerance, as they have inherently high switching barriers.
For instance, if you bought a car and you loved the driving experience, you are inclined towards recommending it to your friends, even if the car gave you a little trouble over the period of time. It’s partly confirmatory bias, but mostly high switching barriers.
You cannot afford to switch to a different brand without taking a financial hit. So, in order to stay consistent with your original conviction, you maintain a strong bias and keep referring the brand.
However, what would be the case if you rented the car? Switching barriers would be relatively low, since you can easily rent a different one to see how it performs.
That’s exactly the kind of problem that SaaS businesses face. Usually, SaaS companies have an inherently low entry and exit barrier, thus, making it difficult to retain customers and build loyalty. That’s also one of the major reasons why most SaaS companies have an NPS in the mid-tier range.
On a couple of occasions, after purchasing something at a store, I was told by the sales associate who served me that I would get an online survey to score his/her customer service. Nothing strange here, until they said: “If you’re happy with my service, please give me a 9 or a 10”. Well, now I know why. This is certainly one way of increasing your score!