Unlocking Growth in CPG with AI and Advanced Analytics

In the battle for market share in the consumer packaged goods (CPG) market, large players have been challenged recently by nimble niche competitors, as well as by traditional and online retailers that use their data advantage and direct consumer connections to push private-label or alternative brands. But these companies have multiple ways to counterattack, one of which is to use artificial intelligence (AI) and advanced analytics to transform their own data into valuable insights. (See “How Big Consumer Companies Can Fight Back,” BCG article, September 2017.)

To understand the value, impact, and challenges of adopting AI and advanced analytics for CPG companies, we conducted a joint study with Google in which we interviewed executives at 25 medium- and large-size fast-moving-consumer-goods players and 5 niche brands, as well as approximately 100 industry experts worldwide. (See the sidebar.) We found that by using AI and advanced analytics at scale, CPG companies can generate more than 10% revenue growth through more predictive demand forecasting, more relevant ­local assortments, personalized consumer services and experiences, optimized ­marketing and promotion ROI, and faster innovation cycles.

But this value still eludes most large CPG companies. Although nearly all those we interviewed had started experimenting with AI and advanced analytics in their core business, none had scaled even one application. They cited multiple organizational roadblocks, from timid support on the part of senior executives to poor data governance and a lack of taxonomy (that is, an agreed-upon data framework), as well as fragmented teams and a failure to ­anticipate fully the impact on jobs and ways of working that AI and advanced ­analytics will have.

Indeed, reaping the benefits of these tools requires sustained, coordinated effort to overcome such barriers. It also requires CPG companies to direct their focus on three to five high-priority areas as opposed to dozens.

Putting AI and Advanced Analytics to Work

CPG companies can increasingly access vast amounts of information, from traditional enterprise data (via their finance and operations departments) to consumer data (especially online behavior) to partner data (typically by way of panels, retailers, insight partners, and others)—even data generated from sensors and Internet of Things (IoT) applications. So far, however, they have neither treated this data as a strategic asset to be protected and nurtured nor applied it in ways that would have a concrete impact on their business.

By using AI and advanced analytics techniques, brands can generate actionable ­insights from such data. The most obvious application for AI and advanced analytics techniques is making predictions, such as the level of demand for a new product, the estimated impact of a marketing campaign, or the emergence of a new consumer trend.

We have identified about 30 applications that brands can use to harness AI and advanced analytics to boost their business. They touch all functions in a CPG organization, from marketing and insights to operations, sales, and support. They can also be used to power new, innovative services such as personalized assistants and recommendation engines.

Out of these 30, we singled out 10 primary applications that represent most of the AI and advanced analytics opportunity for CPGs. When deployed at scale, they can result in more than 10% growth in sales. (See Exhibit 1.)

Four of the ten were cited most often by those in our study and are considered “must-haves” in all CPG categories:

1. Demand forecasting for existing and new products by SKU and region

2. ROI measurement for predicting the impact on sales of advertising and promotional spending

3. Data-powered sales activation for identifying the right retail outlets/points of sale at which to activate the applications and the right set of sales actions to take at the point-of-sale level to maximize market share

4. Optimized product assortments at the individual store level

The other six applications are considered sector-dependent, as impact and implementation complexity can differ widely among sectors such as beauty, food and beverage, and consumer health care:

5. Trend predictions for product development

6. R&D and testing acceleration (in silico)

7. Dynamic, localized, personalized pricing and promotions

8. Precision marketing

9. Personalized consumer engagement

10. AI-powered diagnostic and recommendation services

Notably, these applications include trend predictions, which are most relevant to sectors characterized by short time to market, such as beauty, and by dynamic pricing and promotions, which, to implement effectively, require frequent negotiation with retailers (something that is rare in food and beverage, for example).

While determining which AI and advanced analytics applications would be most effective is relatively simple, deploying them across the organization is a task that still eludes most CPGs. Of the 30 CPGs in our study, all had started working on at least one AI and advanced analytics application and half of them on four or more. (See ­Exhibit 2.) But none had taken a single application and administered it throughout the organization.

Barriers to Scaling Up

Most of the CPG executives we interviewed for our study said scaling up AI and ­advanced analytics—and making sure they’re adopted—is now a key topic of ­discussion among senior leaders. But they also cited a number of challenges. Scaling up even one application is difficult, with significant investment and managerial ­efforts required across the organization.

It is typically ­necessary to build a small proof of concept (PoC) for a specific application on a specific brand and in a specific ­country. But ­deploying that PoC at scale will often ­require work on multiple fronts: on the AI and advanced analytics themselves, so that they’re robust enough to be deployed ­globally across the entire organization; on the data, to solidify its quality and standardize its taxonomy across countries and brands; and on existing IT systems, which may be made redundant by new AI and advanced analytics applications or require significant changes before they can either provide or receive data from the applications.

Other areas that need attention include existing business processes, managerial routines, and job descriptions, because AI and advanced analytics will alter any current decision-making processes, automate manual tasks and calculations, and in the process, change the day-to-day duties of a large number of employees and managers. And finally, talent and skills, because building and maintaining applications requires a talent pool (data scientists, engineers, and analysts) that CPGs will need to at least partially develop to avoid relying solely on third-party vendors.

In considering these difficulties, we identified six distinct roadblocks to effectively scaling up AI and advanced analytics at CPG organizations:

• Lack of Vision. The organization makes only a limited effort to evaluate the impact of AI and advanced analytics, along with the associated size of the prize, and to educate senior management accordingly, which limits the willingness to invest.
• Insufficient Prioritization. This leads to a “PoC explosion,” which dilutes efforts. The organization launches multiple small tests with various vendors but performs no follow-­through. It also fails to put the necessary effort into industrializing, scaling up, and rolling out the application.
• Talent Gap. Difficulty identifying, recruiting, and retaining the right talent (data scientists, engineers, analysts) leads to an overreliance on outside vendors, which makes it hard to control the execution. Meanwhile, the organization makes multiple attempts to develop local expertise, but it often lacks critical mass.
• Limited Data Governance. The organization has no processes in place for data management, quality, or ownership, nor does it have common (cross-division, cross-country) data taxonomies to facilitate scaling up.
• Underestimated Impact. The organization misjudges the level of investment required in change management and in shoring up any related skills. It fails to fully anticipate the impact that AI and advanced analytics will have on existing business processes, decision-­making processes, and managerial routines, as well as on employees’ daily jobs and required skills.
• Inadequate Market Specificity. The organization doesn’t consider how digital ecosystems, data availability, channel dynamics, and vendor capabilities vary across markets. It also fails to recognize the differences in requirements, priorities, and constraints among different markets. 

Embarking on the Journey

Scaling up AI and advanced analytics ­applications and building related capabilities is typically a two- to three-year process. (See Exhibit 3.) Before they get started, brands need to take a number of steps to avoid common roadblocks and ensure quick ­implementation.

Narrow Down The Choices

CPGs need to focus on just a handful of applications. Launching 10 to 15 initiatives ­simultaneously makes it more likely that those initiatives will get stuck at the PoC stage—in part because the attention of the senior managers overseeing them will be spread too thin. Companies that focus on a handful of opportunities (those with the greatest feasibility and potential impact) have a better chance of delivering them at scale. For example, a global fashion brand selected two areas to focus on: personalized consumer engagement and store assortment. This allowed the brand to make the necessary investments in data science and IT resources. It also ensured that there would be enough senior-level engagement to make the initiatives succeed.

Get High-Level Buy-In

Before launching any application, companies need to ensure that it will actually get traction. Top executives (typically country or brand general managers or above) should be willing to sponsor and pilot an application in their division or geography, and a senior business leader should be able to dedicate 20% to 30% of his or her time to steering its development. Brands should also work to ensure that there is significant interest among the local team members who will help launch the applications. Delivering an end-to-end AI and advanced analytics solution will add to their workload—and in the process will end up significantly challenging existing habits throughout the organization.

Assess Build-Versus-Buy Choices

Multiple vendors offer ways to address specific issues with off-the-shelf (or semi-­customizable) AI and advanced analytics software. While an off-the-shelf solution offers the advantage of speed, it comes at the expense of intellectual property ownership. It will also lack the functionality of a custom-­built solution, as well as any internal understanding that would come from building it in-house. CPGs should determine early on the areas best served by existing software and those for which it would be best to build their own solutions (potentially with partners, as long as they own the IP). These decisions should be based on the criticality of the process addressed as well as the company’s data advantage over its vendors. (See “The Build-or-Buy Dilemma in AI,” BCG article, January 2018.) For example, a beauty company chose to build a custom solution for predicting market trends that was specific to its categories and hosted in its own environment. This allowed the brand to maintain complete control over a tool that could one day become a competitive advantage.

Address Market-Specific Needs 

Consumer behaviors, digital ecosystems, and access to data assets have evolved in significantly different ways across major markets. Any market-specific constraints or requirements need to be tackled early on, or the organization risks encountering showstoppers late in the design or implementation phase.

Prepare for Impact

Deploying AI and advanced analytics solutions at scale typically requires building new technology environments (to feed, host, and maintain algorithms), adapting to the current ecosystem (for example, by replacing existing expert systems in some key functions, such as operations), and standardizing efforts around data structure and taxonomy. Such actions don’t need to be taken during the prototype phase, but they should be prepared for if a company wants to avoid having to scrap everything and begin anew after a first pilot has proved successful. For example, while it’s not necessary to clean and structure a global SKU database before prototyping a demand-­forecasting solution, it is important to create a first draft of a common taxonomy to describe related products.

Manage The Change

Introducing AI and advanced analytics solutions systematically challenges any existing decision-making processes and, in some cases, drastically reduces or even eliminates the time it takes to complete certain tasks. For example, a luxury goods company found that an effective demand forecasting engine led to a 60% to 80% reduction in the time its supply chain department spent on daily demand planning. To avoid organizational resistance, companies need to effectively communicate to their staff the impact of AI and advanced analytics ahead of time and give teams perspective on how these applications will affect their jobs and ways of working.

Deploying select AI and advanced analytics applications can yield significant benefits in the short term, but the broader opportunity for CPG companies lies in how these applications will help put the consumer at the center of their operating model. CPG companies can use AI and advanced analytics to translate consumer-­related data into insights and then disperse those insights throughout the organization—from product design to supply chain to marketing and sales.

To achieve that goal, investments in AI and advanced analytics will not be enough. It will require significant changes to ways of working throughout the organization, from board-level decision making to shop floor operations. It’s a long journey, but CPG companies can start small by focusing on a few applications and relentlessly pushing them from end to end.