The truth is our goal and questions are our tools
I recently read The Mom Test: How to Talk to Customers and Learn If Your Business is a Good Idea when Everyone is Lying to You. It’s written by Rob Fitzpatrick based on his experiences building (and screwing up) in the early days of his startup. The goal of the book is to help folks with the ‘how?’ of learning from customer conversations when validating business ideas, which he compares to excavating a delicate archeological site:
- The truth is down there somewhere – but you have to be fragile to get to it
- A lot of teams use a crate of dynamite to excavate – forcing people to say something nice about their idea
- On the other end, some teams are using a toothbrush to excavate a city – not really digging to to see if there’s any value hidden in the conversation
My motivation for reading this book came from my experience building NoMo. I talked to ~20 people before building it, no one really pointed out any red flags, I got a lot of ‘Oh, this is interesting’, ‘Wow, this sounds cool’ which I interpreted as validation and signal to start building. I then proceeded to spend 2 months working nights & weekends working on building the Android app. After launch – whenever I introduced it to friends and family – I got several ‘Oh such an interesting concept’ or ‘Oh, this was built for me’ statements, only for the person to not even install the app. I finally shut it down after ~3 months of trying to gain adoption (super painful!) and never want to repeat the mistakes I made with NoMo again.
Conversations with customers take time and are easy to screw up. The issue with doing them wrong is that, not only can the conversation be a waste of time, but can signal to you to invest resources in a direction which might be completely wrong which will then leave you wondering (like I was) that how did I end up building something that people don’t want – even after talking to users?!?!
A common criticism of talking to customers is that you’re abdicating your creative vision and building product by committee. Given people don’t know what they want – that would be a terrible approach. Deciding what to build is your job – The questions are about understanding the customers lives – problems, cares, constraints and goals. You gather this information and take the visionary leap to a solution. (more on the failure of talking to users here)
The Mom Test
The Mom Test is a simple set of rules to follow which helps us craft good questions – so that even your mother cannot lie to you!
Let’s take an example from the book:
Son: “Mom, mom, I have an idea for a business —can I run it by you?” I am about to expose my ego —please don’t hurt my feelings.
Mom: “Of course, dear.” You are my only son and I am ready to lie to protect you.
Son: “You like your iPad, right? You use it a lot?”
Mom: “Yes.” You led me to this answer, so here you go.
Son: “Okay, so would you ever buy an app which was like a cookbook for your iPad?” I am optimistically asking a hypothetical question and you know what I want you to say.
Mom: “Hmmm.” As if I need another cookbook at my age.
Son: “And it only costs $ 40 —that’s cheaper than those hardcovers on your shelf.” I’m going to skip that lukewarm signal and tell you more about my great idea.
Mom: “Well…” Aren’t apps supposed to cost a dollar?
Son: “And you can share recipes with your friends, and there’s an iPhone app which is your shopping list. And videos of that celebrity chef you love.” Please just say “yes.”I will not leave you alone until you do.
Mom: “Oh, well yes honey, that sounds amazing. And you’re right, $40 is a good deal. Will it have pictures of the recipes?” I have rationalised the price outside of a real purchase decision, made a noncommittal compliment, and offered a feature request to appear engaged.
Son: “Yes, definitely. Thanks mom —love you!” I have completely misinterpreted this conversation and taken it as validation.
Mom: “Won’t you have some lasagna?” I am concerned that you won’t be able to afford food soon. Please eat something.
Our misguided entrepreneur has a few more conversations like this, becomes increasingly convinced he’s right, quits his job, and sinks his savings into the app. Then he wonders why nobody (even his mom) buys it, especially since he had been so rigorous.
Let’s take the next example from the book where Rob shows us what the conversation would have looked like if it had passed The Mom Test:
Son: “Hey mom, how’s that new iPad treating you?”
Mom: “Oh – I love it! I use it every day.”
Son: “What do you usually do on it?” Whoops —we asked a generic question, so answer to this probably won’t be terribly valuable.
Mom: “Oh, you know. Read the news, play sudoku, catch up with my friends. The usual.”
Son: “What’s the last thing you did on it?” Get specific about examples in the past to get real, concrete data.
Mom: “You know your father and I are planning that trip? I was figuring out where we could stay.” She uses it for both entertainment and utility, which didn’t come up during the “usually”answer.
Son: “Did you use an app for that?” A slightly leading question, but sometimes we need to nudge to get to the topic we’re interested in.
Mom: “No, I just used Google. I didn’t know there was an app. What’s it called?” Younger folks use the App Store as a search engine, whereas your mom waits for a specific recommendation. If that’s true more broadly, finding a reliable marketing channel outside the App Store is going to be crucial.
Son: “How did you find out about the other apps you have?” Dig into interesting and unexpected answers to understand the behaviours and motivations behind them.
Mom: “The Sunday paper has a section on the apps of the week.” You can’t remember the last time you opened a paper, but it sounds like traditional PR might be a viable option for reaching customers like mom.
Son: “Makes sense. Hey, by the way, I saw a couple new cookbooks on the shelf —where did those come from?” Business ideas usually have several failure points. Here it’s both the medium of an iPad app and the content of a cookbook.
Mom: “They’re one of those things you just end up getting at Christmas. I think Marcy gave me that one. Haven’t even opened it. As if I need another lasagna recipe at my age!” Aha! This answer is golden for 3 reasons: 1. Old people don’t need another generic set of recipes. 2. The gift market may be strong . 3. Younger cooks may be a better customer segment since they don’t yet know the basics.
Son: “What’s the last cookbook you did buy for yourself ?” Attack generic answers like “I don’t buy cookbooks” by asking for specific examples.
Mom: “Now that you mention it, I bought a vegan cookbook about 3 months ago. Your father is trying to eat healthier and thought my veggies could benefit from a pinch more zazz.” More gold: experienced chefs may still buy specialized or niche cookbooks. The conversation continues. If it’s going well, I would raise the topics of whether she has ever looked for recipes on the iPad or searched for cooking videos on YouTube.
The first conversation gave us rope to hang ourselves while the second gave us insights and leads which we can now pursue. Usually, the big mistake is almost always to mention the idea earlier rather than later. This leads to the simple rules to follow, called the Mom Test:
- Talk about their life instead of your idea
- Ask about specifics in the past rather than generics/hypotheticals in the future
- Talk less and listen more
The measure of usefulness of an early customer conversation is whether it gives us concrete facts about our customers lives and world views. The original idea looked like cookbooks for iPads. There are thousands on possible variations of this premise. With the idea this vague, we can’t answer any of the difficult questions like what recipes to use? How will people hear about it? Until we get specific, it always seems like a good idea.
Based on The Mom Test, there a bunch of questions which could be categorized as good/bad:
- “Do you think this is a good idea?” – Awful question! Only the market can tell whether an idea is good/bad (unless you’re talking to some industry expert). Everything else is noise. Opinions are worthless.
- “Would you buy a product which did X?” – Bad question. You’re asking for opinions/hypothetical – usually to people who want to make you happy. Anything about the future is an over-optimistic lie
- “How much would you pay for X?” Again bad question since it’s based on a future hypothetical + plus the number makes it seem more rigorous. People will lie to you if that’s what they think you want to hear
- “What would your dream product do?” Okay question – you have to dig further to understand WHY are they asking for these features? Critical to understand motivations and constraints behind these requests. People know what their problem is – but don’t know how to solve them
- “Why do you bother?” – good question – get behind the “why” of problem? What is the motivation in solving this problem. You’re shooting blind until you understand their motivations.
- “What is the implication of that?” – Helps you distinguish between “I will pay to solve that problem” v/s “It’s kinda annoying but I can deal with it problem”
- “Talk me through the last time it happened” – Good question – You get a chance to learn through their actions and not their opinions. Watching someone do a task will show you where the inefficiencies are v/s where they think they are.
- “What else have you tried?” – good question – what are they using now? what’s good/bad about it? If they haven’t looked for ways of solving it already, they’re not going to look (or buy) yours
- “Would you pay X for a product that did Y?” – bad question – people are overly optimistic about that they would do in the future and want to make you happy.
- “How are you dealing with it now?” – good question – gives workflow information + price anchor
- “Where does the money come from?” – good question – mostly in B2B contexts
- “Who else should I talk to?” – good question – mostly in B2B contexts
- “Is there anything I should have asked?” – good question – the person by now knows what you’re trying to get at and sometimes they’ll help you with a question or a line of thinking which you hadn’t thought of at all
Avoiding Bad Data
There are 3 types of bad data:
- Fluff: generic claims, hypothetical, and the future
Deflect Compliments: Almost all compliments are lies – not always intentionally spoken. The other party might be trying to be supportive, or just trying to make you happy. You want facts and commitments – not compliments. Ignoring compliments should be easy – but isn’t. We crave validation and trick ourselves into registering compliments as reliable data instead of vacuous fibs. Symptoms you have compliments and not real data:
- “That meeting went well!”
- “We’re getting a lot of positive feedback”
- “Everybody I’ve talked to loves the idea”
If you see this happening – try to get specific – Why did the person like the idea? How much time/money would it save him? How would it fit into their life? What else has he tried? If you don’t have answers to these questions – you’ve got a compliment which ain’t concrete data.
Anchoring Fluff: Fluff comes in 3 varieties:
- Generic claims: I usually, I never
- Future tense promises: I would, I will
- Hypothetical maybes: I might, I could
The mostly deadly fluff is: “I would definitely buy that!”. It just sounds so concrete. But as mentioned before – people are overly optimistic about what they would do in the future. They’re always more positive, excited and willing to pay in the imagined future than they are once that future arrives.
Certain fluff inducing questions include:
- “Do you ever..”
- “Would you ever..”
- “What do you usually..”
- “Do you think you…”
- “Might you..”
- “Could you see yourself..”
When talking in generics – people describe themselves as who they want to be – not as who they actually are. You need to get specific to bring out the edge cases.
Understanding the “Why” Behind Ideas: People might bombard you with ideas or features that you could possibly add to your product. You must understand the motivation behind that idea to get at what is the root problem your customer is trying to solve
The Pathos Problem: When you’re accidentally approval seeking or fishing for compliments – you expose your ego leading people to feel they ought to protect you by saying nice things – you aren’t actually seeking contradictory information – you’re just looking for someone to validate your idea so you can take the leap. the way to deal with this is to keep the conversation focused on the other person and avoid compliment seeking. Keep your idea and ego out of the conversation until you’re ready to ask for commitments.
Avoid Being Pitchy: When the goal is learning, being pitchy might be counter productive. Won’t take no for an answer might work against you since the person might just offer you some compliments/fluff so as to get the conversation over with rather than trying to not hurt you/keep you happy. Symptoms include:
- “No no, I don’t you get it..”
- “Yes, but it also does this…”
Talk Less: You can’t learn anything if you don’t let the other person speak
Asking Important Questions
2 thought experiments:
- Imagine the company has been huge success: Why did it happen? What happened to cause success?
- Imagine the company has completely failed: Why did it happen? What happened to make it a failure?
Questions to address the above 2 factors that you come up with are one way to arrive at the most important question.
Another set of questions are those which completely change or disprove your business.
Another way to look at it is if you got an unexpected response to a question and it doesn’t affect what you’re doing – it wasn’t an important question to begin with.
Asking at least 1 question which could disprove your currently imagined business is what Rob advises.
Love Bad News: Easier said than done. No one likes listening to bad news – especially when it could mean our beloved idea is fundamentally flawed. Learning to love it as a solid piece of learning which gets us closer to our goal of truth is one way to swallow the bitter pill. Learning your beliefs are wrong is frustrating but it’s progress.
We go through the futile piece of asking for opinions because we crave approval. We (falsely) believe that the support and sign-off someone we respect means our venture will succeed and that persons opinion doesn’t really matter – only the market knows.
Some of the best responses you can get are along the lines of “Umm, I’m not so sure about that” and “That’s pretty neat”. Both are lukewarm responses – which tell you they don’t care! You can’t build a business with a lukewarm response. A classic error when one gets a lukewarm response is to “up your game” and pitch them “harder”. The only thing to be gained from this is a false positive. It is worth is to probe further to understand the nature of their apathy – do they not care about your specific implementation? Have they heard too many similar pitches only to be disappointed? Are they different from your ideal customer?
Look Before you Zoom: You could miss the important question – by focusing on ultimately unimportant nuances. Everyone has problems they know about but don’t care enough to fix – but if you lead them to that micro problem too quickly – they’ll happily drown you in all the unimportant details. Zooming in too quickly – on a super specific problem – without understanding the rest of your customers life can irreparably confuse your learning (this is one of the things I did wrong with NoMo).
The premature zoom is a problem since it leads to data which seems like validation, but is actually worthless.
Address the Elephant in the Room: We need to ask questions which actually derisk the business – comforting ourselves by asking questions which don’t derisk the business doesn’t help us with our goal of learning and is a waste of resources.
Prepare Your List of 3: Pre-plan the 3 most important things you want to learn right now. These are the ones you deem most important or ares which seem murkiest. This helps since you have time to make sure they pass the mom test, they aren’t biasing, makes it easier to face questions that hurt since without a plan you might take the conversation in a direction which is comfortable for you.
Keeping it Casual
Learning about customers and their problems works better as a quick casual chat rather than a long formal meeting. Can try to do separate 3 meetings – first about customer + problems, second about your solution and last to sell the product – the first one can be quick and casual.
Commitment and Advancement
Once you have a product – it’s time to start revealing it to your potential customer that you have talked to. This invites compliments – but you can cut through that by asking for commitments or advancements. Commitments look like the customer giving something they value – time, money, reputation. Advancements means they one step closer to purchasing. Commitment and advancement often arrive hand in hand – you need an intro to the manager (reputation risk) to advance to the next stage in the funnel.
Meetings should either succeed or fail at this stage – there’s no such things as a meeting which just went well.
You must press for a commitment or a rejection – else you end up with zombie leads which are just taking time but no commitment. Symptoms:
- Pipeline of zombie leads
- Meetings ending in compliments
- Meetings which went well
- They haven’t given up anything of value
Forms of commitment:
- Money: letter of intent, pre-purchase, deposit
- Time: Clear next meeting with clear goals, sitting down to give feedback on wireframes, using the trial version for a non-trivial period
- Reputation: Intro to peers, to manager, public testimony
Crazy Customer & the First Sale: Keep an eye out for people who get emotional about what you’re doing. When someone isn’t too emotional about what you’re doing, they are unlikely to end up being one of your early fans.
In the B2B space – the best early customers would be people who have the problem, know they have the problem, hve the budget to solve the problem and have already cobbled together a makeshift solution.
In the consumer space, it’s the fan who wants your product to succees so badly that they’ll front you the money when all you’ve got is a duct tape prototype. They’ll tell all their friends and ask them to chip in too!
- Cold calls: Might need to do them early on but aim should be to start getting warm intros via the early cold leads
- Immerse yourself in where they are
- Landing page test: Collect emails via a landing page and have conversations with people who end up leaving their email
- Organize a meetup
- Creating warm intros
- via Industry advisors
- via Investors
Asking for Meetings: Vision, Framing, Weakness, Pedestal, Ask
How Many Conversations?: Keep talking till you’re discovering new information which might change your assumptions/business meaningfully
Choosing Your Customers
When you look at the big successes – they seem to serve everyone – but everyone started out small: Evernote helped moms save and share recipes.
When you have a fuzzy sense of who you’re serving – you end up talking to multiple segments all at once which leads to confusing signals – you get overwhelmed with options and don’t know where to start you receive mixed feedback and can’t make any sense of it. Before you serve everyone, you have to serve someone. If you aren’t finding consistent problems and goals – you don’t have a specific enough customer segment.
If you’re facing a generic set of customer segments – you can slice them further to pick a concrete starting point. Start with any segment – and then keep slicing off better and better subsets until:
- you’ve gotten a tangible sense of who you can go talk to
- and where you can find them
Start with a broad segment and narrow it down by asking: who would want it them most out of them? Why does that subset want it most? What’s the motivation there? Is that motivation unique to them? What other types of people have this motivation? Quoting from the book:
As a quick example, say I’m building some sort of high-end fitness gadget for busy professionals. It’s going to be expensive, so I figure they have to be high-income (finance professionals?), and it’s going to be digital, so I imagine they’ll be young (25-35?). And finance professionals live in big cities, so we’ll tack that on. So my customer segment is “finance professionals, age 25-35, living in a major city”, right? No! This is a totally worthless segment because it doesn’t help me make better decisions and doesn’t help me find them. On the other hand, if I slice it down further to the sub-group who wants it most, then I can get somewhere. We could slice that initial segment into something like: finance professionals in London currently training for a marathon. Better! We know this slice is taking fitness seriously, so we might suspect they’re stronger early adopters. We could slice that even further by saying that we want the sub-sub-subset who go to the gym during their lunch hour. Now we can have all the customer conversations we desire for the price of a membership to a gym in London’s financial district. If there isn’t a clear physical or digital location at which you can find your customer segment, then it’s probably still too broad. Go back up the list and slice it into finer pieces until you know where to find them. A customer segment isn’t very useful if there’s no way you can get in touch. Now that we have a bunch of who-where pairs, we can decide who to start with based on who seems most: 1. Profitable or big 2. Easy to reach 3. Personally rewarding……On a personal note, I really feel that third factor is important; it’s worth choosing customers you admire and enjoy being around. This stuff is hard work and can be a real grind if you’re cynical about the people or the industry you’re trying to understand and serve.
Rule of thumb: Good customer segments are a who-where pair. If you don’t know where to go to find your customers, keep slicing your segment into smaller pieces until you do.
There are a lot more examples and detail in the book which you can buy off Amazon if you found this blog post to be relevant and learnt a lot.
Tweet @ me if you have any thoughts/comments about this post.