I wrote earlier about the three metrics – net sales, gross margin and net promoter score (NPS) – that, in my opinion, provide the best proxy for determining product success. Out of the three, NPS is the most sensitive to misuse.
Now, it can never be an exact science to reduce the complexity of the consumer world down to one number. The NPS method has received some reasonable criticism and it is also quite culturally contextual (i.e. you can’t compare numbers between different countries). Yet, in my experience, it’s the closest anybody has come to something useful and actionable, especially when combined with open feedback. It correlates very strongly with “word-of-mouth” that is essential in keeping the product momentum beyond the initial launch marketing investment and the hype from the enthusiasts that tend to love anything new.
The biggest challenge, in my experience, is not the method itself, but the sample control. It does matter whether you ask the question after 30 minutes or 30 days. And the medium matters. For example, earlier this year, I travelled an airline (I can’t recall for sure whether that was Lufthansa or Scandinavian) that sent me a SMS NPS query about the service of the cabin crew, before I had even gotten out of the plane.
So when you are in the product review and your team presents you awesome NPS scores, the question to ask is about the sample, the timing and method of the data collection.