How to Balance Customer Delight & Profits

Two Netflix cases illustrate how the DHM strategy model helps product leaders balance delight and margin.

Over two decades, Netflix improved its subscribers’ monthly cancel rate from 10% to 2%. They did this by balancing delight against margin (profit) while building a durable, hard-to-copy advantage. I call it the DHM model: Delight customers in Hard-to-copy, Margin-enhancing ways.

Today, Netflix’s hard-to-copy attributes are:

• unique technology (personalization, streaming encoding)
• network effects (a large device ecosystem)
• economies of scale (a $20B annual content spend), and a
• trusted brand

A challenge along the way: How do you evaluate trade-offs between delight and margin? When might you choose to lose money to delight customers?

Below, I answer both of these questions using two Netflix cases, using directional data.

2005: The Perfect New Release Experiment

In 2005, Netflix had one million members in its nascent DVD-by-mail service. The most frequent customer request? Faster new release DVD delivery. The challenge: three months after a movie appeared in theaters, Studios released the DVD version — and as soon as that DVD version dropped, lots of customers wanted it immediately. Within a few months, however, demand decreased as the novelty wore off.

Rather than buying up a surplus of newly released DVDs to handle that short-term spike in demand, we decided to optimize for the long-term. As a result, during the first few months after a DVD launch, some customers had to wait a week or two to get the new release. If the new release wasn’t available when the customer requested it, we shipped the next movie on the customer’s movie list, then sent the new release when it became available.

In response to customer requests for faster DVD delivery, we designed a “perfect new release” A/B test and allocated 10,000 customers to the test cell. These customers received their new release DVDs the next day, while customers in the control cell waited for weeks or months to get their new releases. Our delight metric? Retention.

The questions we hoped to answer:

• Will faster new release DVD delivery improve retention, and, if so, by how much? (At the time, roughly 4.5% of our customers canceled each month).
• If this experience improves retention, is it worth spending more money on DVD inventory? How much more?

The answer: Yes, faster new release delivery improved retention, but by much less than we expected. The monthly cancel rate improved by about 0.05% — from 4.50% to 4.45%. So, was it worth spending more money on the new release DVD inventory?

Here’s how we evaluated the balance between delight and margin:

• If we rolled out this “perfect new release” experience to all members, we’d save 5,000 customers/year. (One million subscribers x .05 retention improvement = 5,000 saved customers).
• We believed happier customers would rave about the service, attracting new members via word of mouth, so we doubled the number of saved customers. We estimated that each “saved” customer would attract one new member to the service.
• The value of this perfect new release experience was $1 million: 5,000 saved customers x $100 (the lifetime value of a customer) X 2 for the WOM factor.

But the cost to deliver that perfect new release experience was $5 million in additional DVD inventory. Given the potential gain of $1 million against a $5M spend, we chose not to roll the feature out to all members.

What we learned:

1. What customers say doesn’t always match their behavior. You need A/B testing to measure behavior change.
2. It’s helpful to understand how much customers value different features. We invested in things our members valued (broader DVD selection, lower prices, next-day DVD delivery). We invested less in features they didn’t value (new release DVDs, social, unique movie-finding tools).

We never established a more precise word-of-mouth factor, though I’ve heard that Amazon sometimes uses an 8X factor. A large WOM multiple encourages more investment in customer delight, but at the time, we couldn’t afford to spend $5 million for a marginal retention improvement.

2016: The Free Trial Reminder Experiment

Netflix’s non-member page is the store window for the service. It invites customers to input their credit card for a free, one-month trial. At the end of this first month, customers convert to paying members — unless they cancel before their one-month free trial ends.

By 2016, Netflix had nearly 100 million members:

• 2% of visitors to the non-member page signed up for a one month trial, and
• 90% of these customers transitioned into paid membership at the end of the month.

The challenge: Many customers called customer support on Day 32 to explain they had forgotten to cancel the service. The customer service agents canceled each customer’s subscription and refunded their money, but this practice cost Netflix around $10M/year.

To decrease these customer service calls, Netflix tested a free trial reminder. On day 28 of the free trial, Netflix sent texts and emails to new customers: “Your one-month free trial is about to end. Click here if you’d like to cancel.” The result? A drop in paid conversion from 90% to 85%, resulting in a $50M loss. (The 5% decrease in conversion translated to a total loss of $60M, but the $10 million in “I forgot to cancel” customer service calls dropped to zero.)

If you were the product manager, what would you do?

Netflix chose to make the free trial reminder part of the default experience. Why? Customers are delighted by Netflix’s effort to make it easy to cancel. The free trial reminder builds trust, which creates a more robust, world-class brand. Although Netflix loses $50 million, it builds a long-term advantage through its hard-to-copy brand. Last, A/B tests can’t measure everything — some customers who canceled may tell a friend about Netflix’s gracious free trial reminders or return to sign up for the service a few months later.

A/B tests can’t measure all of these factors, which is why judgment and strategic frameworks like the DHM model (Delight customers in Hard-to-copy, Margin-enhancing ways) are critical.

The difference between the 2005 and 2016 decisions: The magnitude of investment

In 2005, Netflix couldn’t afford an additional $5 million for new release inventory. By 2016, however, Netflix revenues were nearly $10B. In 2005, spending $5 million on additional new release DVDs was a high-stakes decision. In 2016, the decision to leave $50 million on the table to build long-term value was a low-stakes decision. This loss was only half a percent of total revenue. Plus, the decision would be easy to reverse (this is the second attribute of low-stakes decisions). Netflix could always choose to stop sending end of free trial reminders.

When you make product decisions, ask yourself if it’s a high or low-stakes decision. If the magnitude is high and it’s hard to reverse, consider the decision carefully. Give it plenty of time, and gather as much data as possible. If the stakes are low and the decision is easy to reverse, decide quickly, then move on. Most decisions by product leaders are low stakes (although they almost always believe the opposite). Postponing decisions creates ambiguity, which makes things worse. Be decisive.

As a product leader, your job is to delight customers in hard-to-copy, margin-enhancing ways. The challenge is to measure these factors and assess trade-offs between each. With practice, you’ll learn what customers value and what they don’t. The DHM model, together with A/B testing, provides critical insight to build both a world-class product and brand.

I hope you find this helpful.

Best,

Gib

Gibson Biddle

gibsonbiddle.com

P.S. Click here to provide feedback on this essay — it takes just one minute.

Read my free “Ask Gib” product newsletter on Substack.