Chapter 6: Testing the Leaps of Faith Assumptions

A minimum viable product (MVP) helps entrepreneurs to start their learning process quickly. It’s thus, the fastest way to handle the build-measure-learn feedback loop with less effort. Compared to product development, which involves a long incubation period and targets product perfection, an MVP aims to start the learning process and not finish it and thus test important business hypothesis.

6.0 Why are first products not meant to be perfect?

At IMVU, when Eric Ries and his team were raising money from investors, they were disappointed for two reasons:

• First, their product was still defective and of low quality.
• The business results weren’t mind-blowing.

Although the good news was that they were on a hockey stick-shaped growth curve, it only went up to about 8000 USD per month of revenue. These results were significant for predicting IMVU’s future growth and validated two of their leap of faith assumptions. One was that IMVU was providing value for customers, and they had an engine of growth that was working. The gross numbers were small as they sold the product to the early adopters. These special types of customers prefer an 80% solution and don’t need a 100% solution.

Early adopters use their imagination to fill in what a product is lacking. Early adopters are suspicious of products that are too perfect. For example, if it’s ready to be used by everyone, how much advantage can one gain by being an early customer? Therefore, additional features or perfection beyond what early adopters require wastes resources and time. This is the ugly truth for many entrepreneurs to accept, as their vision is to provide a high-quality mainstream product that would change the world and not produce one that is merely used by a small number of niche customers who are ready to buy the product before it’s completely ready.

As Eric Ries stated, MVPs range in complexity from extremely simple smoke tests to early prototypes filled with issues and missing features. Determining exactly how complicated an MVP should be cannot be done in an established formula. It needs judgment, and it’s not difficult to develop. However, most entrepreneurs and product development individuals vastly overestimate the number of features required for an MVP. to understand this better, consider a service for a 1-month free trial. A customer must sign up for the free trial before using the service. One business model assumption is that customers will sign up for the free trial after acquiring a certain amount of information about the service. An important question is whether customers will sign up for the free trial if a promised set of features is provided. We predict that the percentage of customers who sign up for the free trial is 10%. This is a leap of faith assumption.

Most entrepreneurs will answer such a question by building the product and checking to see how customers will react. Eric Ries considers this wrong as it can cause a lot of waste. Firstly, if you’re building something no one wants, the whole effort will waste time and money. If customers don’t sign up, they’ll miss out on all the features, but many other opportunities are wasted even if they sign up. Such as how many features are needed to please the early adopters? Each extra feature is a waste, and if you delay the test for these features, it comes with a hefty possible cost in terms of learning and cycle time.
The lesson taught by the MVP is that any extra work beyond what was needed to start learning is a waste, regardless of how important it might have seemed at that time. Eric Ries has shared many MVP techniques from actual Lean Startups to demonstrate this.

6.1 The video minimum viable product

Drew Houston is the CEO of Dropbox, a company that makes an effectively convenient file-sharing tool. You must install the Dropbox application, and the folder comes on the computer desktop. Then, anything dragged into the folder is automatically uploaded to the Dropbox service and instantly replicated on all the computers and devices. A group of engineers were the founders of Dropbox, as the product required crucial technical expertise to build.

None of these engineers has worked in marketing before. They had necessary venture capital support and handled their product development differently. The founders wanted customer feedback on what was important to them. Dropbox mainly wanted to test its leap of faith question: if we can provide a superior customer experience, will people try our product? They believed correctly that file synchronization was an issue many people didn’t know they had. This isn’t the type of entrepreneurial question you can ask or expect an answer to in a focus group. Most of the time, customers don’t know what they want, and they always had difficulties understanding Dropbox when its concept was explained. Drew Houston always learnt this the hard way when he tried to raise venture capital as it was hard to show the working software in a prototype form. Thus, he made a video which showed how the software is supposed to work, but it was aimed at a group of early technology adopters.

In this situation, the video was the MVP. It validated his leap of faith assumption that customers required the product he was developing not because they said this in a focus group but because they actually signed up after watching the video.

6.2 The concierge minimum viable product

The concierge MVP is another type of MVP technique. To understand how this works, let’s consider an example of an American startup called ‘Food on the Table’. Food on the Table makes weekly meal plans and grocery lists based on food families enjoy together. It then coordinates with local grocery stores to find the best deals for the ingredients. Behind the scenes, a team of chefs make recipes using items on sale at grocery stores. Then, these recipes are matched through a computer algorithm to each family’s needs and preferences.

Surprisingly, Food on the Table started with just 1 customer. Rather than communicating with thousands of grocery stores in the country, it communicated with just one. The founders didn’t know which grocery store to communicate with and support until they had their first customer. At the same time, they started with no recipes until their first customer was willing to start their meal plan with Food on the Table’s help. The company planned its first customer’s meal plan without software, without signing any business development partnerships, and without chefs. The CEO, Manuel Rosso and the VP of product, Steve Sanderson, went to grocery stores and moms groups in Austin, Texas and observed customers, which was a part of product thinking and other ideation techniques. However, they were on another mission: to discover their first customer.

As they met prospects and conducted interviews, they attempted to make a sale at the end of each interview. They described the advantages of Food on the Table, provided a weekly subscription fee and invited the customer to sign up for it. Although most people are not early adopters and won’t sign up for a new unseen service, they finally had someone to sign up. That early adopter got the concierge treatment. Rather than communicating with food on the table through software, the customer was given a personal visit by the CEO every week. Manuel and Steve studied what was on sale at the customer’s favourite grocery store and carefully selected recipes based on the customer’s preferences. Every week they physically handed the customer a prepared pack including a shopping list and recipes, gained her opinion and made changes if necessary. Most importantly, they collected a check from the customer for 9.95 USD.

This was inefficient as the CEO and the VP of product was only solving one customer’s problem themselves without a team instead of marketing their services to millions. The worst part is that their efforts did not lead to anything important. They didn’t have a product, revenue, no recipes or a well-established organization. However, they were making good progress when viewed through the Lean Startup. Every week they learned more about what was needed to make their product successful. They were ready for another customer a few weeks later. Each customer they brought on board made it easier for them to get to the next customer as they could focus on the same store and get to know its products and the types of people who shopped there. Each customer got the concierge treatment that was in-home visits. However, the expenses of serving each customer personally started to increase when a few more customers came on board.

Only when the founders were busy bringing more customers did Manuel and the team start investing in automation in the form of product development. Every iteration of their MVP allowed them to save a little time and help a few more customers by providing recipes, shopping lists through email instead of in-home visits, starting to parse lists of what was on sale immediately through software instead of manual and eventually started card payments instead of checks.

In less than no time, they had built a considerable service offering starting in Austin and eventually nationwide. However, along the way, their product development team was always focused on scaling something that was working instead of building something that might work in the future. Therefore, their development efforts involved less waste than in a typical company of the same kind. This example needs to be compared with a small business where it’s common to see the CEO, founder, president and owner directly serving customers individually. In a concierge MVP, this personalized service isn’t the product but a learning session built to test the leap of faith assumptions in the company’s growth model. In reality, a typical result of a concierge MVP is to invalidate the company’s proposed growth model and clarify that another approach is needed. This can also happen if the initial MVP is profitable for the company. Without a growth model, most companies are trapped in being happy with a small profitable business when a pivot might lead to better growth. The only way to know if this happens is to test the growth model with actual customers systematically.

6.3 The role of quality and design in an MVP

One of the most annoying things about the MVP is the difficulty it causes to traditional ideas of quality. Modern production procedures depend on high quality to boost efficiency. They function using the famous dictum of W. Edwards Deming that the customer is the most essential part of the production process. Therefore, you must focus on producing results that the customer considers valuable. Allowing messy work into your process unavoidably causes great differences. The topic of quality presupposes that the company already knows the kind of product features that a customer will consider to be valuable. However, this is a dangerous assumption for a startup to make. Most of the time, we are unsure who the customer is. Therefore, Eric Ries believes that startups should always understand the following principle on quality:

“If we do not know who the customer is, we do not know what quality is.”

Sometimes, even a low-quality MVP can help in building a high-quality product. MVPs are sometimes considered “low quality” to customers, and thus you should use this chance to learn what features customers care about and make adjustments to build a high-quality product. However, there have been cases where many famous products were released in a low-quality state, and customers still loved them.

MVPs require you to have the courage to test your assumptions. If customers react the way you want, your assumptions about them are correct. If you produce a poorly designed product and customers and early adopters don’t know how to use it, your assumptions are wrong, and you need to invest in a superior design. But you should always ask this yourself, as per Eric Ries – “what if they don’t care about design in the same way we do?”. Therefore, the Lean Startup model isn’t against building high-quality products. You should avoid traditional professional standards to quickly begin the validated learning process, but this doesn’t mean functioning poorly.

As you build your MVP, remember to remove features, processes or efforts that don’t directly contribute to the learning you’re trying to secure.

6.4 Speed bumps in building an MVP

The most common speedbumps in building an MVP are legal issues, competition fears, branding risks and morality.

When startups rely on patent protection, there are unique challenges with releasing an early product. In some areas, patent filing begins when the product is released to the public, and even if your startup doesn’t belong to any of those areas, you might need international patent protection and may have to follow the strict requirements. On the other hand, in most industries, patents are mainly used for defensive purposes to keep competitors away. In such situations, the patent risks of an MVP are small compared to the learning benefits. Nevertheless, startups should always seek legal advice to understand all the risks carefully in any case.

Although legal risks may be scary, Eric Ries has always heard of the fear of competitors as a common speed bump in building an MVP, especially large companies stealing a startup’s ideas. If a competitor can outsmart a startup by stealing their idea once the idea is known, the startup is in danger. Building a new team to follow a new idea means you believe you can proceed through the feedback loop faster than anyone else. If that’s true, then there’s no difference that the competitor knows your idea. If it’s not true, a startup thus has much bigger problems than a competitor knowing the idea. Eventually, a startup will face competition from fast competitors.

Most startups plan to invest in building a good brand, and an MVP might seem to be a risky branding risk. At the same time, entrepreneurs in existing companies are constantly frustrated by the fear of damaging the parent company’s brand. In both cases, an easy solution is to launch the MVP under a different brand name. Moreover, a long-term reputation is only in danger when companies engage in PR and try to build hype around the product. If a product fails to do this, actual long-term damage can happen to a brand. However, startups can avoid this risk due to the small number of customers and not being exposed to many. Thus, you must use these advantages to experiment secretly and launch public marketing once actual customers have recognized the product.

Finally, it helps to know that MVPs always lead to bad news. Visionaries are afraid of a false negative that is customers won’t like a flawed MVP that is too small or limited. You will see this from customers when companies launch fully complete products that haven’t been tested before. Teams in traditional product development methods are trained to make go/kill decisions daily which is the importance of the stage-gate development model. If an MVP fails, teams lose hope and abandon the whole project; however, this problem can be solved. The solution is a commitment to iteration. You need to agree ahead of time that no matter what the results are in testing the MVP, you will not lose hope. Successful entrepreneurs don’t lose hope at the first sign of risk. Instead, they have determination and flexibility. The MVP is the first step on the learning journey. Down the road, following many iterations, you might learn that some element of your product or strategy is wrong and understand that it’s time to change to a different method to achieve your vision.

Startups are mainly at risk when external stakeholders and investors lose confidence. When the product was accepted, and investment was made, the entrepreneur promised that the product would be successful, but it’s not the case when no customers are attracted to the product. Traditionally, a manager who promises to deliver something and fails to do so is in trouble, and there are only two potential explanations for this failure. One is a failure of execution, and the other is a failure to plan correctly, and both are inexcusable. Entrepreneurial managers face a huge problem: as the plans we make are uncertain, how can we claim success when we fail to deliver what we promised?

Therefore, the solution to this problem lies in the Lean Startup model.

Conclusion

Based on what we’ve explored in this chapter, every startup needs a well-mannered and systematic approach to discover if they’re making progress and achieving validated learning. This is known as innovation accounting, explicitly designed for startups, and it’s different from traditional accounting. We will consider this in the next chapter of ‘The Lean Startup’ series.