The one question to ask about Net Promoter Score in the product review

I wrote earlier about the three metrics – net sales, gross margin and net promoter score (NPS) – that, in my opinion, provide the best proxy for determining product success. Out of the three, NPS is the most sensitive to misuse.

Now, it can never be an exact science to reduce the complexity of the consumer world down to one number.  The NPS method has received some reasonable criticism and it is also quite culturally contextual (i.e. you can’t compare numbers between different countries). Yet, in my experience, it’s the closest anybody has come to something useful and actionable, especially when combined with open feedback. It correlates very strongly with “word-of-mouth” that is essential in keeping the product momentum beyond the initial launch marketing investment and the hype from the enthusiasts that tend to love anything new.

The biggest challenge, in my experience, is not the method itself, but the sample control. It does matter whether you ask the question after 30 minutes or 30 days. And the medium matters. For example, earlier this year, I travelled an airline (I can’t recall for sure whether that was Lufthansa or Scandinavian) that sent me a SMS NPS query about the service of the cabin crew, before I had even gotten out of the plane.

So when you are in the product review and your team presents you awesome NPS scores, the question to ask is about the sample, the timing and method of the data collection.

Attached a screenshot of the NPS inquiry of Hover, the company that handles my domain names. I think they did it well. The survey took place after about one month of usage and was well constructed.
This entry was posted in Stuff and tagged , , , , . Bookmark the permalink.

5 Responses to The one question to ask about Net Promoter Score in the product review

  1. Pingback: The three metrics that matter for a product » Real Box Score

  2. Leon says:

    Hi JP, at some stage it would be interesting to hear your view on reliable metrics worth measuring prior to launching a new product category to market… in other words metrics which are predictive of future product performance.

    Cheers,Leon

    • Well, Leon, that post may take some time. Because, despite 10 years of searching, I’ve yet found a good set of metrics.

      Pre-sales computing gadget research tends to suffer from a severe “stimulus problem.”. One might be tempted to think that when you have 95% of the experiences (not just the sellable product) properly implemented, you could get a linear return on the research accuracy. It just most often isn’t so.

      So I continue the search for those elusive metrics. Meanwhile, I would operate with the Jobsian doctrine that prefers qualitative experts opinions/reviews from people who “get it” or have “magic touch”. The difference being that the expert can also (sometimes) be someone else than myself 🙂

    • Well, Leon, that post may take some time. Because, despite 10 years of searching, I’ve yet to found a good set of metrics.

      Pre-sales computing gadget research tends to suffer from a severe “stimulus problem.”. One might be tempted to think that when you have 95% of the experiences (not just the sellable product) properly implemented, you could get a linear return on the research accuracy. It just most often isn’t so.

      So I continue the search for those elusive metrics. Meanwhile, I would operate with the Jobsian doctrine that prefers qualitative experts opinions/reviews from people who “get it” or have “magic touch”. The difference being that the expert can also be someone else than myself

  3. Pingback: Product guy – make one scoreboard » Real Box Score

Comments are closed.