Simulation Advantage

Almost all companies use experimentation to develop and test new products and services. But in an increasingly turbulent business environment, strategies—as well as products and services—can become obsolete quickly and unpredictably. Therefore companies must broaden the arena of experimentation to include business models and strategies.

Yet there are challenges in applying traditional approaches to experimentation more broadly. Such approaches can be costly and time consuming, and they can saddle the organization with an unreasonable burden of complexity. Market-facing tests and pilots can jeopardize a company’s brand and reputation. And market research based on consumers’ perceptions can be a remarkably poor predictor of success for novel concepts and strategies.

To overcome these barriers, a growing number of adaptive competitors are using an array of new approaches and technologies to expand the scope and impact of experimentation in their businesses. In doing so, they are creating a “simulation advantage” by achieving superior “economics of experimentation.” In other words, these companies are able to generate, test, and replicate a larger number of innovative ideas more quickly, at lower cost, and with less risk than their rivals can. (See the exhibit.)

Levers for Capturing a Simulation Advantage

A company achieves a simulation advantage by establishing a system and culture that promote experimentation. Consider Intuit, the U.S.-based software company, which has embraced an aggressively experimental approach to business. By doing so, it has raised the number of new-product launches from an average of five per year in the ten years leading up to 2003 to 31 in 2009 alone, thereby generating significant new streams of revenue.

Companies can employ six levers for improving their experimental effectiveness.

Generate and capture ideas at greater speed and lower cost. Some companies are finding new ways to unleash collaboration among employees and customers to generate a continuous and rich supply of new ideas. Royal Dutch Shell’s GameChanger unit uses an online collaboration portal, workshops, and relationships with universities to gather ideas from contributors both inside and outside the company. A team of ten Shell professionals review the ideas and grant the best ones up to $25,000 in initial funding for development into investment proposals. These proposals are then assessed by project-specific panels, which can allocate more substantial funding from set-aside budgets. So far, GameChanger has evaluated more than 3,000 ideas from its submission teams. Roughly 70 percent of the ideas chosen for development involve at least one individual from outside Shell. GameChanger projects, which have generated a significant volume of patents, are most often picked up for further development in R&D but are also spun off into successful businesses or licenses to other companies. Royal Dutch Shell estimates that it has invested $250 million in GameChanger projects to date.

Make the process of experimentation faster and less expensive. Some companies have decreased the costs of physical prototyping and testing by creating “virtual worlds” in which they test products and ideas. Such environments make it possible to set up, modify, expand, and rapidly execute experiments at much lower cost. Kimberly-Clark, for example, has reduced the cost of testing new ideas before they are scaled up by using a 3-D virtual store to gauge consumer and customer reactions to new products, shelf layouts, and packaging.

Increase the volume of experiments. Virtual worlds have no capacity limit and therefore allow for a thorough exploration of possibilities. They enable companies to gain richer insights by increasing the number of variables tested or by running more tests in parallel. Amazon.com is able to test numerous page variants on different groups of consumers and track their resulting behavior. If the impact of a potential new feature is statistically significant, Amazon promptly incorporates it into its mainstream model. For example, the company ran a large number of experiments examining how the order of search results affected purchases. It then optimized its search results so that the items most likely to be of interest to consumers were listed first. The payoff was a 3 percent increase in sales. Despite the fact that half its experiments “fail,” Amazon promotes continuous experimentation to ensure that it doesn’t miss any break-through opportunities.

Lower the cost of failure. Real-world experiments can damage a brand or threaten supplier or customer relationships when they fail. Some companies have reduced this risk by switching to online testing or soft launches in friendly communities. The European telecommunications company Orange, for example, uses its online platform to interact with 30,000 Lab Explorers—customers who contribute to the company’s innovations. Orange mines the input of this community at multiple points in the development process using a suite of methodologies, including cocreation workshops, surveys, and usability testing. The results enable offerings to be refined before they are broadly released. Through hundreds of such experiments, Orange has consistently improved its “simplicity score”—an aggregate indicator of how likely consumers are to use and appreciate a product—resulting in improved marketplace performance and customer satisfaction.

Enhance predictive power. Companies can use virtual experimentation to capture a broader range of behavioral data with greater accuracy and without the “rationalization biases” associated with traditional perception-based market research. Such experiments improve the accuracy of predictions by providing a view of customer behavior not just at the point of purchase but before and after as well. Companies can also improve the predictive value of experimentation through the use of prediction markets, which collect, aggregate, and track many individual predictions of success.

Consider Best Buy’s online prediction market, TagTrade. More than 2,000 people participate in this virtual stock market by trading stocks on variables ranging from launch dates to sales estimates. Since its inception, TagTrade has consistently beaten the company’s internal estimates by a significant margin.

Accelerate learning and scale-up. Companies with a simulation advantage are able to scale up and deploy successful experiments more rapidly and comprehensively than their competitors can. Some do it by sharing information on successes through online knowledge platforms or virtual training. Others use the business itself as a vehicle for parallel piloting of new ideas. Wal-Mart, for example, tests different layouts in its own stores as a regular part of its operating model. Traffic patterns and sales impact are measured for each layout, and the most successful designs are rolled out across the country.

An Integrated Approach to Experimentation

Establishing a simulation advantage requires more than the application of individual techniques and tools. Rather, an array of levers and behaviors must be integrated, optimized, and coordinated.

Procter & Gamble has become a pioneer and leader in ubiquitous experimentation by means of a suite of innovative tools addressing all six levers described above. P&G’s leaders encourage an outward-looking, collaborative orientation—the goal being that half the company’s innovations come from external sources. For example, P&G leverages open innovation networks, such as InnoCentive, to solve technical design problems. And it uses a walk-in, 3-D virtual store (enabled by the CAVE virtual-environment system) to run experiments faster and at lower cost than traditional market tests. Finally, by employing online user panels, such as Vocalpoint, it is able to introduce products to friendly audiences before the full launch.

P&G has significantly increased its productivity as a result of this integrated system of experimentation. In 2008 alone, ten highly skilled employees were able to generate between 7,000 and 10,000 design simulations, enabling design mockups to be completed in hours instead of weeks. More than 80 percent of P&G’s new-business initiatives now make use of its growing virtual toolbox.

Creating a Culture of Experimentation

Even an integrated set of techniques and approaches will not succeed unless it is deployed within a culture of experimentation. Simulation-advantaged companies share a number of cultural characteristics:

• They appreciate the necessity of an experimental approach
• They embrace creative dissatisfaction with the status quo
• They provide incentives that encourage experimentation and remove barriers that discourage it
• They measure the effectiveness and economics of experimentation rigorously
• They tolerate failure as a necessary part of learning
• They soften internal and external boundaries to facilitate collaboration
• They cultivate statistical literacy and a hunger for information

Harrah’s Entertainment is one company that has successfully adopted a culture of experimentation. Once a minor player in the gambling industry, it now ranks among the most profitable in the world. Its data-rich environment allows it to run dozens of parallel experiments around the clock. As CEO Gary Loveman has joked, “There are two ways to get fired from Harrah’s: stealing from the company or failing to include a proper control group in your business experiment.”

Developing a simulation advantage is a significant undertaking. Everyone in the organization must understand the value of ubiquitous experimentation and learning. Leaders can help their organizations embark on this journey by challenging them to benchmark their experimental performance against competitors and best-practice players. By answering the eight questions below, they’ll put themselves on the path to superior economics of experimentation:

• How can we better leverage the collective intelligence of our employees and customers to generate more ideas at greater speed?
• How can we make our experiments faster and less expensive?
• How can we increase the number of experiments we are able to run at any given time?
• How can we lower the cost of failure for unsuccessful experiments?
• How can we increase the “hit rate” of our experiments?
• How can we identify, scale up, and roll out successful experiments faster and more effectively?
• How can we establish an integrated system of experimentation, including appropriate processes, roles, tools, and metrics?
• How can we establish a culture that embraces creative dissatisfaction, encourages experimentation, and tolerates failure?

The conjunction of a turbulent business environment with the availability of new approaches to experimentation creates both the necessity and the means to build a simulation advantage. Organizations that embrace these opportunities early will be tomorrow’s winners.