How Netflix’s Customer Obsession Created a Customer Obsession

Photo credit: Adaptation of photo by re:publica/Gregor Fischer (CC BY-SA 2.0).
Customer focus is great, obsession is even better.
  • We used existing quantitative data to understand past and current behavior.
  • We did qualitative — focus groups, one-on-ones, usability — to hear how people think and react to our work.
  • We executed surveys to capture who the customer is and how to think about them — by demographics, by competitive product usage, entertainment preferences, etc.
  • Then we AB tested the hypotheses formed via the above inputs to see what worked.
Note the red line, moving up after June — a surprise increase in cancel rate in June of 2005.
The cancel rate in Hawaii drops quickly — why?
The percentage of DVDs delivered the next day, grew to 95% as the total number of hubs grew to 100.
An example of one of the test cells for the Previews Test
The Netflix non-member site (2006)
  • In the early years, the most effective way to position ourselves against Blockbuster was through the phrase, “No Late Fees.”
  • The “ happy family on a couch” communicated compelling emotional benefits.
  • In the early digital delivery days, the word “streaming” did not exist and was misunderstood as “streamlining” — a bad thing. We used the phrase “Watch Instantly” to describe our nascent service until “streaming” was broadly understood.
  • Demographic survey. It’s good to know who your customers are — we were often surprised.
  • Cancel survey. When customers canceled, we asked them why they left. This gave us a trend line — if an issue spiked over time, we would address it quickly.
  • Sample data. As our customer base grew, we used surveys to assess the quality of key attributes like DVD delivery speed and streaming quality. We enlisted our members as testers, enabling us to ship features sooner, then responded quickly if issues came up.
  • Net Promoter Score. This is a proxy for overall product quality, and while Netflix had much more meaningful retention data, I found it to be a reasonable proxy for product quality and it’s easy to execute. In working with other companies, I am hopeful if the NPS score is above 50, and get excited as NPS climbs into the 70’s. (In the past, Netflix had an NPS that was close to 80.)
A reasonable guess at today’s Net Promoter Score for Netflix.
The Netflix site in 2010. Still not simple — more “stuff” communicates more value.
Today’s Netflix non-member site is very simple. It took 15 years, but the brand now communicates substantial value.
Netflix tested thumbs against Stars. Thumbs doubled the number of ratings.
Look closely at the “98% Match” for Casino — there’s no longer a star rating for each movie.
  • The original hypothesis was that the more high-quality movies members watched (as measured by high star average) the better they would retain. This notion was consistent with what customers said in qualitative. Unfortunately, the data did not prove this out. Over time, higher average ratings for members did not lead to higher retention.
  • In shifting to “percentage match,” Netflix acknowledged that while you may rate a “leave your brains at the door” Adam Sandler comedy only three stars, you enjoy watching it. And as much as you feel good about watching “Schindler’s List,” and give it five stars, it doesn’t increase your overall enjoyment. (The movie is a bit of a downer.)
Summary of strengths (green) and weaknesses (red) of each research technique.
  • Does your company have a well-developed set of “e-staff metrics” that helps you form hypotheses, but also educates your team about the balance of delight and margin?
  • Do you have surveys, indicating the demographics of your customers, why they leave, as well as a Net Promoter Score? Do you have the ability to gather “sample data” from customers to build a trend line and identify mistakes quickly?
  • Do you have ongoing focus groups, one-on-ones and usability sessions with customers? Does your team meet and talk with customers to develop the “voice of the customer” within your organization?
  • Do you have an A/B test capability that allows you to measure the trade-off between delight and margin, and develop your team’s intuition to form stronger hypotheses in the future?
  • Based on consumer insights and learnings, do you have a product strategy that defines your hypotheses about what you hope will fulfill the trifecta of delighting customers in hard-to-copy, margin-enhancing ways?
From Amazon’s 2016 Annual Letter



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Gibson Biddle

Gibson Biddle

Former VP/CPO at Netflix/Chegg. Now speaker, teacher, & workshop host. Learn more here: or here: