Retailers today operate in an environment of relentless competitive and financial pressure, in which they must manage multiple formats, ever-changing consumer preferences, and tremendous volumes of data. To succeed, they must capitalize on advanced data and analytics in order to transform their merchandising.

Making analytics work effectively means retailers should take an integrated approach to five strategy, organization, and technology levers: 1) set a clear and consistent strategy, 2) prioritize key initiatives according to their strategic value, 3) redesign the organization and upgrade the talent, 4) put the right enabling technology in place, and 5) work in new ways.

It’s a tough challenge, but retailers that rewire their organization to capitalize on advanced analytics in this way are seeing revenue growth and improved margins of up to 2 percentage points each year from the first wave of implementation. And these gains can be reinvested to lock in stronger value creation over the long term.

Increased Complexity in a Changing Market

Until recently, many retailers could win with a clear value proposition, as long as merchandising articulated a decent pricing strategy, understood its promotion effectiveness, and had a process in place to review its chainwide assortment. Today, that’s no longer enough. Customers now want retailers to make things easy for them, with localized assortments, personalized offers, a faster shopping experience, new product segments, and sustainable sourcing. To win, retailers need to deliver all of this—along with real-time price adjustments and integration with third-party online marketplaces.

Many merchandisers are ill equipped to handle this complexity. Some still rely largely on manual processes, Excel spreadsheets, and gut instinct. They house data in multiple silos and run ad hoc processes resulting in disconnected, lagging reports, which means they’re unable to run tests or make data-driven assortment decisions. In addition, frequent turnover among category managers limits institutional knowledge and short-circuits the opportunity for continuous improvement.

Most retailers recognize that they need to do better, and a number of them have already made investments in advanced analytics. But all too often, gains are incremental, unsustainable, or both. Some organizations even lose ground. One grocery chain that relied too heavily on analytics actually saw a slowdown in sales growth, which management attributed to algorithms killing off too many promotions in an attempt to enhance gross margins. (To be fair, most industries aren’t much better. In a recent BCG survey, seven out of ten companies reported minimal or no gains thus far from their AI initiatives.)

The underlying problem is that analytics is often treated as a black box: something that retailers simply turn on and then wait for the right answer to emerge. In fact, analytics is a tool—an extremely powerful tool—and like all tools, it can be more or less effective depending on how it’s handled.

Needed: An Integrated Approach

A key stumbling block is that retailers often lack an integrated approach to advanced analytics in which ways of working, technology, and the underlying algorithms all come together to reinforce one another and give the company a differentiating advantage. Retailers that try to apply advanced analytics as a plug-and-play solution, instead of taking an integrated approach, often experience “organ rejection” as the merchant team struggles to unlock value from the torrent of data now available to them. Out-of-the-box solutions force merchants to make too many low-value changes to their processes, while customized options require too much change and compromise too quickly.

Our work with retailers shows that about 70% of the effort and focus in implementing advanced analytics should be devoted to organizational factors: the strategy, ways of working, processes, and skills and capabilities—as well as how changes to those elements can create an advantage. Another 20% should address enabling technology: fast, scalable solutions that deliver the right decision analytics and recommendations, on-demand and often in real time, in a simple, easy-to-use format. And the remaining 10% should be spent on tuning the algorithms.

These pieces all need to fit together seamlessly. Top retailers apply this integrated process with a clear objective in mind: to give merchants more information and accountability so that they can deliver better customer outcomes and make more scalable decisions faster across a set of practical, actionable use cases.

The Merchandising Function of the Future

What does success look like? We believe that the merchandising function of the future will have three features, with advanced data and analytics underpinning all three:

• It will be truly customer led. Top retailers will have a relentless drive to localize, personalize, and curate the offer in order to keep customers happy. They will spend substantial time understanding consumer needs and broader trends in their segment. Every product category will be optimized to what customers value most, across all dimensions: product value, availability, consumer choice, novelty, and other attributes. Brands and SKUs that deliver on customer needs and financial metrics will be prioritized; those that don’t deliver will lose their place in the assortment.
• Merchandisers will be able to do more, better. Tools and insights will be integrated into processes, empowering teams to make data-driven decisions rather than over-relying on instinct and experience. A tight link will exist between strategy, decisions, and outcomes (and there will be consistent follow-through to address gaps where this connection doesn’t exist). Automated tools will take over the bulk of routine activities such as promotion and ad planning or small changes to the assortment, with greater efficiency and effectiveness as well as better performance. That will free teams to spend more time on higher-value activities, such as curating assortments, localizing and personalizing products and services, and collaborating with supplier partners to deliver better products at lower costs. Teams will also be more responsive to changes in the market, able to identify and implement changes more rapidly.
• Performance will improve every day. The final feature will be a culture of continuous improvement on two levels. First, the retailer will capture feedback from customers and supply chain partners, along with sales performance, and rapidly adjust to continually improve its offer. Second, the organization itself will always be improving, in areas such as faster innovation, more convenience for customers, and keeping costs low. New tools and processes will be kept simple for the initial rollout, with further capabilities added over time.

When merchandising improves in these three ways, it can create substantial value for the enterprise. For example, a department store retailer implemented advanced analytics to boost growth, improve margins, and strengthen partnerships. The retailer started by building category strategies to segment demand and developing a single source of truth about the profitability of its brands. A focus on training and upskilling merchants to take a more active role in negotiations helped generate insights about how and where to improve. The result was a boost in margins of 2% to 4%, clearly defined category strategies, and a better experience for shoppers.

A Framework for Success

Achieving this vision requires putting the right foundation in place to capitalize on analytics, by aligning roles, use cases, and technology. Retailers should use a framework consisting of five integrated levers, which need to be implemented in parallel.

1. Set a clear and consistent strategy. First and foremost, retailers need to develop and communicate an enterprise strategy and explicit value proposition—a combination of offers that can deliver fair prices, convenience, exclusivity, or other points of differentiation like sustainability—for a clear and specific set of target customers. Critically, the strategy needs to cascade down to the level of individual categories or divisions so that merchants have clear direction on how to manage each category in order to help meet the company’s broader objectives.

2. Prioritize key initiatives according to their strategic value. With a clear strategy in place, companies can identify and prioritize the merchandising initiatives that will most effectively drive that strategy. This means launching some immediate value creation measures to “fund the journey”—typically, this entails optimizing the assortment and promotions across all retail segments. From those early successes, retailers can then launch longer-term initiatives to transform the organization and create a sustainable advantage. Specific longer-term measures vary by segment; for example, although all retailers will see value from hyper-personalization, food retailers may look to create an advantage through localization, while apparel retailers may generate more value from markdowns.

3. Redesign the organization and upgrade talent. The current organizational model at most retailers was built for an analog world. The analytics-enabled merchant organization will need new capabilities to capture, analyze, and visualize data. Merchants and colleagues in other key functions will need training on how to use data-driven insights in their decision making. Further, retailers may need to reduce midlevel management layers and empower merchant teams and support roles to become more analytical. Retailers will also have to recruit, retain, and promote newer types of talent, such as data scientists and engineers as well as consumer insight experts.

In our experience, retailers that do this well establish a virtuous circle. A handful of talented data scientists and data engineers deliver a good use case, creating additional demand from merchants once they see how much value this approach can create. That demand for new use cases attracts new talent and helps to build a real, competitive advantage over time.

4. Put the right enabling technology in place. As noted previously, algorithms are not enough. Retailers need modular, flexible technology to enable new processes. Rather than running state-of-the-art algorithms on legacy IT—which may lead to time-consuming workarounds and diminished effectiveness—retailers should build and migrate modular data and application programming interface (API) infrastructure to the cloud in order to enable the advanced analytics needed for strategic initiatives. By looking at specific use cases, they can make smart decisions about what data sets to restructure and add to the cloud, and in what order. Similarly, retailers should expect to continually build and tune algorithms according to various use cases—for example, to factor in product categories with long lead times or outlier events such as holiday sales periods.

5. Work in new ways. Finally, analytics-driven organizations need to work in new ways. As mentioned earlier, this is where the bulk of the effort is spent—according to the 70-20-10 breakdown—and where a majority of the value gets generated. For example, merchant scorecards and planning processes can be restructured to capitalize on data, drawing on advanced tools and reporting to provide a common starting point for all categories as merchants begin their annual planning processes.

Incentives also require restructuring to reward collaboration across functions, both to provide sufficient support to merchants and other decision makers and to end the traditional approach of each category and division developing its own sets of analytics, tools, and processes.

More broadly, change management is critical if organizations are to transform the way work gets done and make those improvements last. Leaders need to create clarity for their teams about what is changing, and why, and must clearly define new roles, processes, and expectations. Change management also requires transparency about where the organization is succeeding—or falling short—and which teams may need more support in terms of resources, attention, or other limiting factors.

Putting It All Together

To see how this approach works in practice, consider an Asia-Pacific food retailer that used analytics to reset its offer. The company started by defining customer purchasing behaviors and preferences, along with customer, operational, and financial guardrails and targets for each category. It deployed deep analytics to spot growth opportunities, with range, pricing, and planogram changes to create a more effective, engaging, easy-to-shop customer offer.

In addition, the business embedded analytics and insights into a set of enhanced category management processes. Sales and margins both increased by the mid to high single digits, market share grew, and the retailer improved its perception among customers and vendors. Better yet, the merchandising team has not only continued to tap into new sources of value from more advanced analytics but has also deployed its data science capability to support other business functions in unlocking new value through analytically driven ways of working.

Retailers are right to see advanced analytics as a transformational opportunity, but success won’t come from focusing on algorithms alone. Instead, retailers need an integrated approach that emphasizes organizational aspects (70% of the time and focus) and enabling technology (20%), along with the requisite algorithms (the remaining 10%). It’s a difficult challenge, but the rewards are worth the effort. In fact, because this kind of comprehensive, integrated approach can be tough, retail CEOs and lead merchants need to start now. By doing so, they can gain a critical advantage over the competition, position themselves to excel despite a complex retail environment, and build the merchandising function of the future.