Bad Presentation Gone Good.
An inside look at building a presentation for a 400-person audience. Experts thought it was great, but survey data suggested otherwise. Here’s how I turned things around using Net Promoter Score surveys.
Over the last two years, I have given 150 talks worldwide. For me, presentations are a creative outlet. So, unlike most speakers, who build one or two presentations, I have created ten. I love to build new talks.
A few months ago, I offered to create an all-new “Customer Obsession” talk for Nir Eyal’s “Habit Summit.” Looking back, it was a risky move, given the large audience.
My “fuzzy vision” for the talk was to describe the value of delighting inherently unsatisfiable customers through “consumer science” — a blend of quantitative data, focus groups, surveys, and A/B testing. I planned to bring the idea to life through examples from Netflix.
I am no longer a babe in the woods as a speaker. I outlined my thinking for the new talk in three chapters, each with three main points. I developed unique frameworks. I crafted compelling visuals and examples. I fashioned personal stories for the opening and closing. Then I built in “hooks” to engage the audience.
I finished the draft of my presentation a few weeks before the event. Then I recorded the presentation and shared it with Suzanne Dufore (a research expert) and Nir Eyal (the conference organizer). “Looks great,” was the response from both. Mission accomplished!
Not so fast. I appreciate expert feedback, but what works for an audience is unpredictable. The most important part of a presentation is the ability to engage the audience, and the best way to evaluate this is through practice with a friendly crowd.
A week before the “Habit Summit,” I presented my talk to twenty-five product leaders at Amazon. As I presented, I noted the audience’s body language, the slides they chose to photograph, their questions, and their comments at the end. I enjoy this qualitative feedback but have learned to rely on a more comprehensive feedback system: a Net Promoter Score survey. Not only does NPS provide a somewhat objective measure of quality, but because the survey is anonymous, the written comments are more critical, and therefore more helpful.
When the audience finished asking questions, I sent them a SurveyMonkey link and asked them to complete the survey. As the room emptied, I glanced at the NPS results on my phone:
Uh oh. The NPS was 29. For perspective, I like to score higher than 50 on my first try, then tune the presentation to achieve a 70 or higher for the final version. In this case, only 41% of the audience raved about the talk, and 12% panned it. 41% “promoters” less 12% “detractors” is an NPS of 29. We can argue about the predictive power of just seventeen responses or the unique “promoters less detractors” calculation, but an NPS of 29 is not a good start.
The talk sucked.
I am a feedback freak. I consistently ask two simple questions: “What’s good?” and “What could be better?” In this case, I listened to the in-person qualitative(“It was great!”) but dug deeper for insight in a conversation with Amazon folks who stayed in the room. Last, I evaluated the qualitative responses in the NPS survey. (Click here to see the complete results.)
Insights from the NPS survey: ask fewer rhetorical questions, build in more “polling,” share more Netflix examples, and surprise the audience through product decisions where A/B test results contradict other data sources.
It’s now one week from my 400-person talk, so I brush myself off, mutter “creative work is hard,” then build a new presentation.
Two days before the event, I present my revised “Customer Obsession” talk to forty friends at NerdWallet. This time, I share more mini-cases and describe how customer obsession shaped Netflix’s personalization efforts over the last ten years.
The result:
Phew. The NPS of 46 is close to the 50 that I hope for on a first try. And there’s plenty of insight in the survey responses: focus more on research techniques beyond A/B testing and provide clear takeaways at the end. (Click here for the full survey results.)
Two days later, I’m at the “Habit Summit,” waiting for Nir to finish his talk. Nir is a professional speaker — he’s a tough act to follow.
And I’m next.
I had concerns about my talk, but if I made another 17-point improvement in NPS, all would be well. Plus, I suspected that the “friendlies” at the test drives were tougher on me because they knew it was a rehearsal.
Nir finishes his presentation, then introduces me. I begin the talk, then enter the trance-like “speaker zone,” noting facial expressions of the audience, the slides that inspire photos, and their many questions at the end.
My performance felt good, I managed the time well, and I didn’t repeat my annoying filler phrase (“okay?”) too often. But I have learned that how I feel about a talk and the more objective NPS result are often very different.
The result? Drumroll, please:
Success! I missed the “dance on rooftops” score of 70 by one point, but for a large, diverse audience, I feel great about the score, and there’s lots of helpful feedback that will make the next version better. (You can see the feedback here.)
For my NerdWallet pal who wisely asked for “clear takeaways,” here are three for this essay that I think are relevant for many creative pursuits:
- Ask for feedback. Ask, “What was good?” and “What could be better?” Don’t wrap yourself in the protective cocoon that friends provide with their cheery, “It was great!” comments. The audience will give candid, helpful feedback, but you have to ask for it. And you need to listen carefully.
- Use NPS as an objective measure. I use NPS for talks, essays, work sessions, and even quarterly product strategy meetings. I suck at the beginning, but the NPS score provides a relatively objective metric for what “Great” really looks like, and the qualitative feedback helps me to get better each time.
- Just keep getting better. Creative pursuits are a practiced art, and the moment you become overconfident and stop trying to get better, all heck will break loose. For presentations, test drives with a friendly audience enable a “practice makes perfect” approach. But even when you think you have a “perfect” version, keep soliciting feedback to make it even better.
But wait, there’s more! I just gave a revised version of my “Customer Obsession” talk to a 500-person audience in Cleveland at the Industry Product Conference. (Watch it here). My NPS was 89! You can read the feedback here.
You can also read my “Customer Obsession” article by clicking here. This essay is a companion to the presentation and won’t “spoil” the experience in any way.
If you liked this essay, please “clap” so others can find it. And, of course, I’d love your feedback on this essay:
Click here to give feedback — it only takes a minute.
Many thanks!
Gib
Gibson Biddle
P.S. Below are my essays on product management, leadership, and strategy:
- My First Podcast! I interview Todd Yellin, VP Product at Netflix
- How to Run a Quarterly Product Strategy Meeting
- Branding for Builders
- Hacking Your Product Management Career
- How to Find a Great Job
- Leaders Lead
- What Your Next Head of Product Looks Like
- Six Tips for Making Wicked Hard Decisions
Many thanks to Dan Olsen, Ashita Achuthan, Nir Eyal, Julie Eyal, and Kelsey Biddle for their feedback and editing help.
