Making Big Data Work: Retailing

Traditional retailers generate and capture a deluge of data—most notably, customer transaction histories that can reveal detailed product affinities and promotional and marketing response rates. Now the emergence of big data and advanced analytical tools and techniques can connect data with a larger context. Big data can explain the who, what, when, where, why, and how of retailing.

Although some leading companies have gained a reputation for deft data handling, most retailers have not yet built the analytical capabilities and internal processes necessary to take advantage of the deep well of information they can access. Merchants and marketers often rely on tactics that worked last year, with only slight modifications. Sometimes their promotions end up discounting the wrong items and hurting rather than helping sales. Too frequently, they simply rely on consumer goods companies or suppliers, with their different incentives and motivations, to tell them what to do.

In the end, many retailers have not figured out where and why they are winning and where and why they are losing. They struggle to discover which prices, promotions, and store locations are working best. They have a hard time taking advantage of all the contextual information around transactions that could make a difference in sales. In effect, they know the outcomes of millions of real-time experiments, but they are not able to look at and learn from them.

All this leads to missed opportunities. Ultimately, it opens doors to online and direct sellers, which often have better data and more sophisticated analytics.

Three High-Potential Opportunities

Big data can help turn this situation around. But in such a fluid technology landscape, it can be hard for retailers to tell where to focus their big-data efforts: which projects are a flash in the pan, and which ones will create growth opportunities and competitive advantage?

In our work with retailers across a range of market segments, we see three opportunities that offer high potential in the near term. Exploiting them can generate a significant increase in revenues and profits for retailers.

Boosting the Effectiveness of Promotions. Promotions are a common source of success and frustration for retailers. Overall, promotions tend to have a dramatic effect on retail sales. But once retailers start discounting, they are often hesitant to reduce promotional activity, because of the steady pressure to increase top-line growth. Making matters worse, most retailers do not have a handle on how their promotions actually perform.

Finally, many retailers rely heavily on suppliers for the selection of promotions. However, vendors often confuse retailers by providing conflicting consumer research in support of their brands. And they tend not to care if a promotion boosts sales for their product line while having the unintended consequence of lowering sales in the overall product category.

As a result, 30 to 50 percent of promotions have no positive impact on sales and margins. Even worse, many of them reduce profits without leading to additional sales.

Understanding the performance of promotions is difficult, because the necessary analysis is complex and because processes are interdependent. As a first step, retailers need to establish an accurate baseline of sales without promotions to understand the direct lift from a promotion and the cost of the discount. Retailers also need to tease out the secondary effects of promotions. Cannibalization can result when the promoted item reduces sales of other, substitutable items. A “halo effect” can result from the sale of additional products when shoppers come in for the discounted product. And the “pull forward effect” can depress item and category sales following a promotion, since consumers have already stocked up on the products. (See Exhibit 1.)

In addition, retailers must determine how much support vendors are really providing for promotions. That includes finding out how much of the funding that manufacturers provide is in fact associated with promotional activity. Retailers also need to capture the operational implications of promotions, including incremental marketing, supply-chain, and store-labor costs. Doing this kind of analysis on a single promotion requires good data and techniques but is not all that complicated for large retailers. However, executing this analysis across the thousands of item level promotions running concurrently and isolating external effects such as season and weather can be extremely complex, even for sophisticated retailers. It can be hard enough to determine what happened as a result of a promotion, let alone what might have happened if the company did not run the promotion. Even more challenging is to translate this analysis into tangible actions that buyers can take to improve performance.

In short, the analysis requires a huge amount of varied and detailed big data and advanced analytical techniques to sort out the interdependencies. But the payoff is worth the effort. We consistently see potential improvements in profit margins of around 1 percent of sales, depending on a retailer’s starting position and the share of income it derives from items on promotion. That can represent 20 percent of a retailer’s entire net income, which can be invested in lower prices, better store layouts, and improved shareholder returns.

One grocery retailer achieved these kinds of results after seeing poor performance from promotions. As a result of the kinds of advanced analytical techniques described above, the grocer was able to tailor promotions to products, with some brands receiving additional promotions, others a price cut, and some no promotion.

Targeting Pricing Precisely. Retailers often struggle to set the right prices while trying to provide a consistent pricing experience to customers across stores. We find that pricing zones, or areas with similar pricing, are typically determined on the basis of how a retailer organizes its activities geographically. Setting prices in this way often causes retailers to place a great deal of emphasis on city, state, and country borders. When people cross these sometimes arbitrary boundaries to shop, however, they see different prices from the same retailer.

As a result, customers might cherry-pick the best prices from nearby locations. Also, less price-sensitive customers might pay less than they may have been willing to, and more price-sensitive customers might not buy at all. Frequently, retailers create too few pricing zones, which limits their ability to respond in a more nuanced way to competitive pressures.

A better approach is to use individual customer behavior as the foundation for establishing pricing zones. Rich transactional data from loyalty programs and credit cards across thousands of stores can be used to understand where individual customers tend to shop. That data can be layered onto an analysis of current store locations and pricing zones. All this information can be combined, analyzed, and mapped to show geographic clusters of price awareness based on observed shopping behavior rather than artificially set boundaries.

This is the kind of complex, messy information from which big-data projects excel at generating insight. As a result of an advanced analytical approach, retailers can determine the likelihood of an individual customer shopping at different stores. They can then justify charging different prices across a region, be confident that pricing performance will not erode, and ensure that customers will not be likely to encounter noticeably different prices within the area where they usually shop.

Consider the case of a national retailer that used state boundaries to establish one set of prices across most of the Greater Atlanta area for a group of products essential to drawing customers to stores. An analysis of loyalty card data revealed considerable complexity in customers’ shopping patterns across Georgia and adjacent states. Even within Atlanta, the retailer discovered a wide diversity of customer demographics and shopping behaviors. For instance, because of Atlanta’s sprawl and difficult commutes, stores serving the same cluster of customers tended to be located along commuting corridors.

By examining where customers shopped, the retailer identified about a dozen unique store clusters in Georgia alone. (See Exhibit 2.) The analysis gave the retailer a strong rationale to establish as much as a 30 percent price difference for the set of products across store clusters, within a region that previously had had only one set of prices. The retailer raised prices in some store clusters while still offering the lowest available price in the area. At the same time, it lowered prices in other store clusters just enough to beat the local competition. Customers throughout the region saw competitive prices in their shopping area, while the retailer generated $5 million in additional profits within a single category from the combination of higher prices and additional store visits. Through a big-data analysis of billions of transactions, the retailer was able to scale the approach to the entire U.S., resulting in a much more refined approach to pricing, without causing customers to notice the multiple price points.

Understanding the Value of a Network. Retailers often view the value of their network of stores—the “last mile” of retail—in terms of the value of the land and the buildings. A network of brick-and-mortar stores is seen as a fixed cost tethered to a physical location. Because of the long-term costs of site location, real estate, and construction, changing a network of hundreds of stores can be time consuming and expensive. These days, executives often look at a network as a burden, rather than an opportunity. That burden is made even heavier as sales migrate online and leave many brick-and-mortar stores with too much space or an insufficient network to meet demand.

Big data reveals another way to think about store networks. One specialty retailer was able to unlock value from its retail footprint with an advanced analysis of its network. Consumer research suggested that a well-known technology brand had not achieved its full potential for sales among a key customer segment, in part because its existing direct-sales network was spotty in many areas. The retailer believed it could help the brand provide the necessary coverage for those businesses, but it needed proof.

The retailer decided to analyze how well 12 million businesses and 117 million households in the United States were covered by its network, by its competitors’
networks, and by additional retailers that could be partners with the technology brand. It used geoanalytical techniques to examine 3.4 billion point-to-point customer trips in order to identify gaps in the technology brand’s retail and partner network.

The results were clear: The specialty retailer was 37 percent closer on average to U.S. households and businesses than its competitors were. In fact, the retailer had the best coverage of key customer segments when compared with competitors and other potential partners in almost all U.S. regions. The company determined the true value of its last mile by looking at it through the lens of a complementary player.

As a result, the retailer was able to partner with the technology brand to generate $40 million in incremental revenues from its existing network at little additional cost. It also gained a new understanding of the value of its footprint. We believe that other retailers can achieve results from their network on the same relative order of magnitude by using a wide variety of tactics, aided greatly by big data and geoanalytics.

How to Begin

As retailers explore these opportunities aided by big data, they should take the following initial steps:

• Focus on the most pressing opportunities. Retail executives are highly pragmatic: companies win by improving sales or margins every day. Executives should determine how to fuel growth in specific ways. They should not try to build a “complete solution.”
• Start with the data you truly need. Executives should certainly solidify their infrastructure. Sales, costs, promotions, space, store locations, and customer data should be connected, but only insofar as that will drive value for the business. Companies will likely have to connect more data as efforts evolve. To achieve results quickly, they should begin with just the data that is required in the near term.
• Bring the organization along. The people who make daily decisions in retail—buyers, trade planners, and others—are hungry for useful information. Help them trust the output of the analysis by including them in the process and being transparent about it. This will improve your results and make them much more relevant.
• Translate the analysis into tangible actions for the broader organization to validate. Ultimately, big data in retailing needs to help people make practical decisions faster and easier: Should we promote a product for an extra week? Should we offer a two-for-one deal? Which promotions should we continue and which should we stop? Make sure that recommendations resonate with those making the daily decisions. If they can’t respond to new information easily, they will ignore it.
• Maintain trust. One-to-one marketing and individualized pricing can sometimes be taken so far that customers lose trust in a company’s basic fairness in pricing or its ability to protect their privacy or data. To create trust and gain access to even greater amounts of personal data for big-data applications, retailers must communicate transparently how they use the data and must demonstrate the important benefits to consumers from the new analytical techniques. (See The Trust Advantage: How to Win with Big Data, BCG Focus, November 2013.)

Retailers cannot afford to pass over these kinds of tangible big-data opportunities. Companies that excel at tapping into them can achieve measurable growth. The value they create will be long lasting.