How Business Experimentation Fuels Innovation

Experimentation is crucial to the process of innovation and new technologies are making experiments easier than ever. Harvard Business School professor Stefan Thomke discusses the impact of experimentation (successes and failures) and how most companies aren’t investing in experiments nearly as much as they should be.

June 6 2011 by Stefan Thomke


At the heart of every company’s ability to innovate lies a process of experimentation that enables the organization to create and refine its products and services. In fact, no product can be a product without it first having been an idea subsequently shaped through experimentation. Today, a major development project involves literally thousands of experiments, all with the same objective: to learn, through rounds of organized testing, whether the product concept or proposed technical solution holds promise for addressing a need or problem. The information derived from each round is then incorporated into the next set of experiments, until the final product ultimately results. In short, innovations do not arrive fully fledged but are nurtured—through an experimentation process that takes place in laboratories and development organizations.

The Leadership Challenge

All organizations have an – explicit or implicit — experimentation process, but few senior managers organize that process to invite innovation. In fact, the book In Search of Excellence noted years ago:

“The most important and visible outcropping of the action bias in the excellent companies is their willingness to try things out, to experiment. There is absolutely no magic in the experiment. It is simply a tiny completed action, a manageable test that helps you learn something, just as in high-school chemistry. But our experience has been that most big institutions have forgotten how to test and learn. They seem to prefer analysis and debate to trying something out, and they are paralyzed by fear of failure, however, small.”

This holds especially in the development of new products and services, where no idea can become a product without having been shaped, to one degree or another, through the process of experimentation. But experimentation has often been expensive in terms of the time involved and the labor expended, even as it has been essential in terms of innovation. What has changed, particularly given new information-based technologies available today, is that it is now possible to perform more experiments in an economically viable way while accelerating the drive towards innovation. Not only can more experiments be run today, the kind of experiments possible is expanding. Never before has it been so economically feasible to ask “what-if” questions and generate preliminary answers. New technologies such as computer modeling & simulation and rapid prototyping enable organizations to both challenge presumed answers and pose more questions. They amplify how innovators learn from experiments, creating the potential for higher performance and new ways of creating value for firms and their customers. At the same time, many companies that do not fully unlock that potential because how they design, organize, and manage their approach to innovation gets in the way. That is, even deploying new technology for experimentation, these organizations are not organized to capture its potential value—in experimentation, in innovation.

“Experimentation” encompasses success and failure; it is an iterative process of understanding what doesn’t work and what does. Both results are equally important for learning, which is the goal of any experiment and of experimentation overall. Thus, a crash test that results in unacceptable safety for drivers, a software user interface that confuses customers, or a drug that is toxic can all be desirable outcomes of an experiment – provided these results are revealed early in an innovation process and can be subsequently reexamined. Because few resources have been committed in these early stages, decision-making is still flexible, and other approaches can be “experimented with” quickly. In a nutshell, experiments that result in failure are not failed experiments—but they frequently are considered that when anything deviating from what was intended is deemed “failure”.

Herein lies the managerial dilemma that innovators face. A relentless organizational focus on success makes true experimentation all too rare. Because business experiments that reveal what doesn’t work are frequently deemed “failures,” tests may be delayed, rarely carried out, or simply labeled verification, implying that only finding out what works is the primary goal of an experiment. If there is a problem in the experiment, it will, under this logic, be revealed very late in the game. But when feedback on what does not work comes so late, costs can spiral out of control; worse, opportunities for innovation are lost at that point—reinforcing the emphasis on “getting it right the first time.” By contrast, when managers understand that effective business experiments are supposed to reveal what does not work early, they realize that the knowledge gained then can benefit the next round of experiments and lead to more innovative ideas and concepts – early “failures” can lead to more powerful successes faster. IDEO, a leading product development firm, calls this “failing often to succeed sooner.”

Next Steps

I frequently ask management audiences to list all the business experiments that they are aware of in their companies. After all, in the absence of similar experiences or good predictability of outcomes in complex business settings, true experimentation is the only way to manage uncertainty and identify promising innovation in the future. Projects that become experiments after they are finished or during late stages don’t count because they usually provide few opportunities to learn. However, projects that are designed, funded and managed as experiments (i.e., maximize learning from success and failure) do matter and should be an integral part of a firm’s innovation strategy. I have found that few leaders can prepare such a list or present a portfolio of organizational experiments – even though they know that experimentation is the lifeblood of new products, services and business opportunities. Thomas Edison could not have said it more clearly: “The real measure of success is the number of experiments that can be crowded into twenty-four hours.”