Technology Innovation Management Review March 2012

Minimum Viable Product and the Importance

of Experimentation in Technology Startups
Dobrila Rancic Moogk

they drown. ”
“ Startups don’t starve;Shawn Carolan
Managing Director, Menlo Ventures

Entrepreneurs are often faced with limited resources in their quest to commercialize new
technology. This article presents the model of a lean startup, which can be applied to an
organization regardless of its size or environment. It also emphasizes the conditions of ex-
treme uncertainty under which the commercialization of new technology is carried out.
The lean startup philosophy advocates efficient use of resources by introducing a minim-
um viable product to the market as soon as possible in order to test its value and the entre-
preneur’s growth projections. This testing is done by running experiments that examine
the metrics relevant to three distinct types of the growth. These experiments bring about
accelerated learning to help reduce the uncertainty that accompanies commercialization
projects, thereby bringing the resulting new technology to market faster.

Introduction A response to these challenges is presented by Eric Ries,

in his book The Lean Startup (tinyurl.com/7dxddzz), which
Commercializing a brand new product or service is a has its origins in his blog (startuplessonslearned.com), and
complex task with an uncertain outcome. Whether it is his concrete entrepreneurial experiences in startups
undertaken in a big corporation, a startup, or a not-for- such as IMVU (imvu.com), which he co-founded and
profit organization, it requires vision, determination, where he served as a CTO. Much of Ries’s original think-
and resources. Entrepreneurs of all stripes, in a garage, ing draws inspiration from Clayton Christensen’s theor-
a multinational, or in a social enterprise, have brilliant ies presented in The Innovator’s Dilemma
ideas and doggedness to succeed. Resources, however, (tinyurl.com/7onvohk) and The Innovator’s Solution
can be scarce. (tinyurl.com/7n7x5rd); Geoffrey Moore’s Crossing the
Chasm (tinyurl.com/6qfeowt); and the lean production sys-
Money needed to bring a new idea to market is difficult tems derived by Taiichi Ohno and Shigeo Shingo and
to secure. In an established business, metrics of success popularized by Toyota (tinyurl.com/3a8tx3). This article
that apply to the mainstream products or services typic- will focus on the lean startup and the novel idea of
ally do little to justify an investment in a new idea “minimum viable product”; the article will show how
serving different customers in different markets. For these concepts can make technology entrepreneurs
startups, with the ever-diminishing availability of ven- and their startup endeavours more successful.
ture capital over the last decade, bootstrapping is the
order of the day. Those few that have been lucky to se- A Lean Startup
cure venture capital investment must account for
frugal-yet-effective spending to their boards of direct- For the purposes of this article, Ries’s definition of a
ors. For these reasons, all visionaries who want to suc- startup will be used: “A startup is a human institution
ceed need to ensure that the scarce resources not only designed to create a new product or service under con-
last longer but also bring a demonstration of market vi- ditions of extreme uncertainty” (Ries, 2011; tinyurl.com/
ability as early as possible. 7dxddzz). This goes hand in hand with the intention to re-

www.timreview.ca 23
Technology Innovation Management Review March 2012

Minimum Viable Product and the Importance of Experimentation

Dobrila Rancic Moogk

cognize that startups, apart from what is commonly A Minimum Viable Product
considered to be a bootstrapped, or a venture capital
backed small operation, can be found in for-profit or- A startup operates around a vision that its product will
ganizations of all sizes, among the not-for-profits, and uniquely solve the pressing problems of customers in
even within the government. This wide definition of their target market. The founders often expect that their
startups should be kept in mind even though the ex- product will deliver an unprecedented return on their
amples in this article are predominantly drawn from investment. This vision includes two important as-
new technology companies. sumptions: the assumption around providing value
(i.e., the value hypothesis) and the assumption around
Typically, startups arise around a vision that a new growth in the market (i.e., the growth hypothesis).
product or service (henceforth, a product) will be em-
braced by a particular market because it solves the cus- To illustrate these two assumptions, consider a now
tomers’ urgent problem. Following the path of well-known example of the early success Facebook had
established companies, startups often devise a strategy with investors. In the summer of 2004, when Facebook
to develop a full product based on this vision and de- was just six months old, had 150,000 registered users,
ploy it in the target market. In many such cases, there is and had very little revenue, the company was able to se-
not as much traction as anticipated, which brings about cure its first $500,000 investment (Ries, 2011;
the painful realization that either the product was not tinyurl.com/7dxddzz). In April 2005, Facebook raised an ad-
compelling, the market was poorly chosen, or both. ditional $12.2 million (Arrington, 2005; tinyurl.com/
yduzltb). Apart from the business model that was based
Developing a full product before testing a concept in on producing revenue from different types of ads and
the market is a risky proposition due to the extreme un- sponsored groups, what was it that investors found so
certainty associated with startup operations. But how compelling about the company? Remarkable as it was
can the market be tested if the product is not fully de- that, in such a short time, Facebook amassed 3.85 mil-
signed and implemented? This apparent paradox is lion users, equally impressive was the statistic that 60%
based on the notion that startups operate on the same of the users logged on daily (Arrington, 2005).
management principles as the established companies. Moreover, Facebook did not spend money to acquire
Although established companies typically serve known its customers. The organic growth in registered users
customers in deterministic markets, startups have to coupled with their strong engagement validated the
address an environment of extreme uncertainty. company’s value hypothesis. The increase from 150,000
Hence, startups need to operate in a way that will users at 6 months to 3.85 million users at 14 months val-
provide them with the opportunity to learn while valid- idated Facebook’s growth hypothesis.
ating their vision; ambiguity must be replaced by in-
creasing certainty over as short a timeframe as possible. For a startup, it is essential to validate its value and
growth hypotheses as soon as possible. In order to do
The best learning for a startup comes as a result of ex- that, the company has to come up with a version of its
periments that test a version of a product against relev- product that is complete enough to demonstrate the
ant metrics. The result of the experiment can reveal value it brings to the users: a minimum viable product
whether the original idea: i) is valid, in which case de- (MVP). It then needs to design experiments that will
velopment can continue in the same direction or ii) is use the MVP to confirm (or refute) its value and growth
not valid, in which case the strategy has to change. The hypotheses. On the one hand, an MVP may need less
acceleration of this feedback loop is essential to take time to develop and should have just the “bare bones”
full advantage of the learning. set of features. On the other hand, an MVP should in-
clude development of capabilities to measure its trac-
The key principles of the lean startup include: omni- tion in the market. Although many product features
presence of the entrepreneurs, uniqueness of the star- that were “on the drawing board” will be soon reques-
tups management style, and learning from product ted by the users, designers should avoid the tempta-
testing against relevant metrics. This article will drill tion of including these features in the initial
down further into the aspect of accelerated learning development – their time will be much better spent de-
from experiments designed to validate a product veloping the experiments that measure the MVP’s im-
against pertinent metrics. pact.

www.timreview.ca 24
Technology Innovation Management Review March 2012

Minimum Viable Product and the Importance of Experimentation

Dobrila Rancic Moogk

In order to measure the effect of the MVP, baseline data the higher the difference, the higher the rate of growth;
must be captured. Further data will be collected as part however, the cost of acquiring an additional customer
of planned tests on the initial MVP and its subsequent should include all related costs, including such things
revisions. The complexity and number of tests can be as Google AdWords, supporting a sales force, and ef-
staggering. In an online consumer business, the num- forts to bring customers into a physical store. The MVP
ber of different versions of a product running different experiments of companies that rely on the paid engine
tests can be in the thousands, and can change every of growth should track metrics related to lifetime reven-
week, even daily. ue values and customer acquisition costs.

Regardless of the challenge of managing that complex- It is important to emphasize that metrics designed to
ity, the more pressing question is how to determine evaluate the impact of MVPs should measure the real
what needs to be measured in order to evaluate the im- business impact and not simply produce feel-good res-
pact of these tests. To address this issue, Ries suggests ults through “vanity metrics”. An example of the latter
that three types of engines of growth should be con- case would be a company that relies on the sticky en-
sidered: sticky, viral, and paid (Ries, 2011; tinyurl.com/ gine of growth tracking only the number of newly ac-
7dxddzz). quired customers. While steady growth in this metric is
encouraging, the company may not be making any real
The sticky engine of growth relies predominantly on progress if the number of disengaged customers is
the high retention rate of its customers. A mobile tele- growing at the same rate.
phone service provider would be an example of this
type of growth. Another example would be a fabless Conclusion
semiconductor company that sells intellectual property
(such as ARM and its microprocessors) to be designed Startups are organizations that develop new products
into another product (such as a smartphone). The com- under conditions of extreme uncertainty. The startup
pany relying on the sticky growth needs to diligently label can be applied to small new companies, but it can
track the number of customers who disengage from the also be applied to parts of established enterprises that
company’s product. If that number is related to a peri- are trying to break new ground in order to give a boost
od of time and is expressed as a fraction of all the cus- to a slowing growth. In every case, startups can benefit
tomers, it is called the churn rate. The company’s from the lean startup philosophy, especially from the
growth rate is defined by the growth rate of newly ac- ideas and learning generated as a result of testing min-
quired customers minus the churn rate. In contexts imum viable product versions against relevant metrics.
that depend on a sticky engine of growth, this is a relev- By applying this philosophy, startups can develop
ant metric that should be captured in MVP experiments. products that are tailored to target markets.

The viral engine of growth was originally described as The idea of a lean startup focuses on increasing devel-
“network-enhanced word of mouth” by Draper Fisher opment efficiencies and reaching the target market
Jurvetson (Jurvetson, 2000; tinyurl.com/6nneasa), a venture sooner, thus potentially capitalizing on the first-mover
capital firm that was a seed investor in Hotmail.com. It advantage. It emphasizes that startups should try to
is now a legendary story that the growth of Hot- eliminate waste (e.g., wasted development resources)
mail.com accelerated when the company decided to by releasing an MVP as soon as possible. Startups
add a link at the bottom of every outgoing message sent should use MVPs to engage target customers and test
by existing users, inviting the recipients to register for the value and growth hypotheses using metrics that are
its free email service. Hotmail.com went from zero to 12 suited to the type of the engine driving a startup’s
million users in 18 months with a $50,000 advertising growth (i.e., sticky, viral, or paid). Initially, develop-
budget (Jurvetson, 2000). Other examples of this growth ment should focus on experiments that provide an-
are online social networks and “house parties” used to swers to fundamental questions related to the value
sell a slew of different products. and growth hypotheses. Subsequently, the focus should
be maintained on the engine of growth itself. Although
The paid engine of growth simply relies on the differ- it is possible that more than one type of engine can be
ence in the lifetime revenue from each customer minus propelling the growth, the most successful startups are
the cost of acquiring every additional customer. Clearly, those that focus on only one of them at a time.

www.timreview.ca 25
Technology Innovation Management Review March 2012

Minimum Viable Product and the Importance of Experimentation

Dobrila Rancic Moogk

The fundamental idea behind the lean startup philo-

sophy is that the real product of an early-stage startup About the Author
is an experiment, or a slew of experiments, that contrib-
ute to reducing the initial extreme uncertainty. Progress Dobrila Rancic Moogk has over 19 years of execut-
of a startup can be measured by the learning that is ive and leadership experience in R&D, marketing,
gained from these experiments. The more accelerated product management, and people management
the learning, the closer the startup gets to releasing the roles in high-tech companies ranging from startups
right product in the right market and to attaining its vis- to multinationals in North America and Europe. She
ionary goals. is currently working on business strategy, product
development, and corporate development with sev-
eral high-tech startups. Her interests are in the area
of increasing the efficiency of innovation commer-
cialization. Also, Dobrila serves as a vice-chair with
the University of Ottawa Women in Engineering and
Computer Science committee and a vice-chair on
the Volunteer Ottawa Board of Directors. Dobrila
has a BSEE as well as Master of Engineering and
MBA degrees from the University of Ottawa.

Citation: Rancic Moogk, D. 2012. Minimum Viable

Product and the Importance of Experimentation in
Technology Startups. Technology Innovation
Management Review. March 2012: 23-26.

www.timreview.ca 26