This is the fourth article in a series exploring customer insight (CI) functions in consumer-facing industries.

Thanks to digital advances, companies now know more than ever about the intentions of consumers—whether they are shopping online or in brick-and-mortar stores. Consider consumers researching or shopping for personal goods such as apparel, footwear, or accessories. In the US, 50% to 60% of shoppers report knowing upfront the category (sneakers or jeans, for example) that they want to buy. Men are the most decisive shoppers—50% say they know both the category and the brand they want to buy before they shop, compared with about 35% of women. Only 5% to 10% of shoppers have the exact item in mind before they buy it. This leaves a lot of room for companies to influence purchases. 

But most companies are unable to capitalize on that opportunity. Consumers’ online and mobile activities leave a digital trail, giving companies vital insight about where to reach them; data on things such as their interests, habits, and choices, which may correlate to shopping decisions; and how to predict and influence future behaviors and purchases. Many organizations, though, lack the capabilities to follow that digital trail. And even if they can, this information often remains locked up in the customer insight (CI) function, never making its way into business decisions.

The gap between CI functions and business operations is a problem in many companies. It’s the result of a number of issues, including persistent skepticism among business leaders about the value of the information CI provides—hence a failure to pull CI into business decisions—and an inability or unwillingness to act decisively on that information.

Companies should face this gap head-on. To fully exploit the massive and detailed customer information now available, companies must finally and fully integrate their CI function into their business operations. But this is no quick fix. Achieving effective integration requires concrete action by leaders of both the CI function and the business. Drawing on BCG’s extensive experience in transforming CI operations, we have identified five actions that can allow companies to create real collaboration between CI and the business and build a truly customer-centric organization.

More Mature CI Functions Make Strategic Contributions

To understand how companies can improve the effectiveness of their customer insight efforts, it is important to step back and assess how CI functions evolve and the role they need to play. The Boston Consulting Group’s Center for Customer Insight, in partnership with Cambiar and the Yale School of Management’s Center for Customer Insights, conducted a global benchmarking study of nearly 650 respondents from more than 90 customer-facing enterprises across all segments of the consumer industry in 2009 and then again in late 2015.

That research has revealed four discrete stages of CI maturity (see “Why Companies Can’t Turn Customer Insights into Growth,” BCG article, August 2016):

• Stage 1: Traditional market research provider
• Stage 2: Business contributor
• Stage 3: Strategic insight partner
• Stage 4: Source of competitive advantage

Our most recent study revealed that companies with CI functions at stages 3 and 4 are characterized by robust communication, coordination, and cooperation between CI team members and business managers. For example, in stage 1 companies, 56% of respondents (business managers and CI professionals) believe that business partners pull CI into core operations more than CI pushes its way in, versus 90% among stage 3 companies. Stage 1 companies use CI in about 25% of the more than 30 business decisions and core processes we evaluated, compared with 75% for stage 4 companies.

As CI functions progress through these stages, CI practitioners and business managers become increasingly satisfied with their relationship. As the business increasingly invites CI into decision making, business partners’ satisfaction with CI’s contributions to business performance increases, jumping to 90% overall in stage 4. Business leaders and CI managers who “strongly agreed” that the line pulled CI into decisions had a 95% overall satisfaction with CI effectiveness in business decisions, and 39% said they were very satisfied.

At stages 3 and 4, in which CI is a valued decision-making partner, CI functions are able to execute in four critical areas:

• Sourcing or Harnessing Proprietary Data. CI aligns on business objectives and needs, identifies critical questions, designs research, executes or oversees studies that generate or leverage high-quality customer data, and maintains records of past studies.
• Generating Insights. CI teams experiment with new data sources, technologies, and methodologies, analyze customer and behavioral data, and synthesize data sets from multiple consumer sources to convert findings into insights and inspiration.
• Translating Insights into Recommendations. CI teams synthesize data sets from multiple consumer, competitive, economic, and operational sources to recommend business actions. CI teams visually interpret and display data and use storytelling techniques to spur business partners, executives, boards, and other stakeholders to action. They facilitate direct contact between executives and consumers by orchestrating customer-centered events and forums.
• Supporting Ongoing Business Decisions and Actions. CI teams distribute insights to key decisions makers, provide ongoing support to fine-tune execution stemming from those insights, and quantitatively and qualitatively measure outcomes and return on investment in CI. 

CI Remains Isolated

While there is great potential for CI functions to enhance business decision making, many companies have yet to realize it.

We found in our most recent study that in many companies, CI is no more integrated into key business decisions and core operations than it was in 2009. On average, only about half of core business decisions in the companies in our study are made with the influence of CI, and 35% to 40% are made with input from commercial big data analytics. (Commercial analytics capabilities may be part of CI or may operate as a separate function.) The degree to which CI is involved varies considerably by topic. (See Exhibit 1.)

That’s not to say that progress hasn’t been made: in the years between our two studies, more-advanced CI functions have mastered a broad set of activities and skills, including building influence within the company, effectively managing a range of stakeholders, and enhancing client interaction. In addition, CI and commercial analytics team members have been forced to become more proactive and creative in disseminating relevant and timely customer insights. And our research shows that the influence of CI leaders and practitioners is increasing: the percentage of companies that pull CI into decision making increased to 68%, from 54% in 2009. The biggest gains for CI involvement from 2009 to 2015 were in the areas of pricing and promotional strategy. (See Exhibit 2.)

Companies should further improve integration and cooperation, forging stronger relationships between CI and the business functions, particularly below senior leadership levels. While 64% of business managers are satisfied overall with CI’s effectiveness, a paltry 11% are highly satisfied.

So why do CI functions often fail to have a seat at the decision-making table? One finding that may shed some light is the persistent gap between how business partners perceive the CI function’s capabilities and performance and how CI team members perceive themselves. That gap can range from 25 to 50 percentage points, with CI practitioners typically rating their capabilities twice as high as business managers do. We found that the biggest differences in perception relate to sourcing and storing proprietary data—collecting high-quality data, executing studies, and analyzing and interpreting data—and to generating insights from the data. Interestingly, since 2009 the first set of activities has been outsourced more than all other CI activities.

The View from Inside

To build a productive relationship, the business and the CI function each must understand what the other requires.

The business wants a proactive partner. For their part, business managers indicate that more frequent, relevant, and timely communication with the CI team would improve engagement, strengthen understanding, and create common goals. One senior automotive executive said, “The insights group has to find ways to collaborate and provide insights in a timely and meaningful way to the business group. It’s not just providing the information and analysis at the pace of our business, but the implications before decisions happen and investments are made.”

In fact, our research shows that as CI functions mature, they tend to share and distribute information more proactively. At stage 1, CI distributes research results primarily to the business groups that commissioned the work through traditional media such as reports. At stages 3 and 4, CI disseminates research findings and information on various consumer topics more broadly through regular interactions with the executive and board in which CI makes presentations alongside business counterparts. In addition, more mature CI groups expand their reporting methodologies by, for example, creating self-service visualization tools, pushing alerts, using handhelds in the field, or creating videos and infographics. They use “push” outreach methods more frequently than “pull” approaches such as ad hoc or standing meetings with business leaders.

CI needs customer-centric business partners. In terms of organizational engagement, our 2009 research showed that the CI focus was on being represented at key decision-making meetings. That representation, and even basic coordination, is largely in place now, but that’s not the same thing as integration and cooperation. In the future for CI—as well as for commercial analytics—the focus will need to be on building a collaborative relationship with the business characterized by transparency, openness, true partnership, information sharing, responsiveness, respect, and a communications approach that gives partners the benefit of the doubt and avoids the blame game when things go wrong.

From a CI team’s perspective, the best business line partners have four defining characteristics. First, they have an external, customer-centric orientation and mindset. Second, they do not simply coordinate with customer insight—they are effective collaborators. Third, they have strong leadership skills. And fourth, they are strong in business execution—once they make a decision, they act effectively and decisively, bringing others along with them.

The first characteristic—true customer centricity—is particularly critical. CI practitioners describe the best business managers as those who are guided by customers’ needs and demonstrate that through everyday behaviors, including:

• Defining leadership and commercial roles in part on customer metrics
• Participating first-hand in qualitative research and mystery shopping
• Frequenting target customers’ social and cultural events
• Spending time in customers’ markets in order to understand their needs
• Devoting weekly or monthly executive and leadership meetings to customer topics
• Making simple gestures such as starting meetings with a focus on customer needs and on strategy

Finally, CI practitioners want a clear explanation of desired business outcomes and the key business metrics that define success for their projects and research efforts. They want to understand the level of investment that is appropriate for the outcome business managers are aiming for so that they can recommend the right approach: no CI support, initiatives that draw on existing data, or new research projects. In addition, mutual clarity and agreement at the outset about the desired outcome allow direct as well as soft, indirect measurement of returns from CI and commercial analytics investments.

Five Steps to Maximizing CI’s Impact

At most companies, CI and business functions agree on the financial and strategic opportunity to strengthen CI capabilities, transforming insights into business actions. They also largely agree on the opportunity for CI to support business decisions on an ongoing basis rather than just for the life of a project. But how can CI and the business build a truly effective partnership? We have identified five steps to drive the kind of change that can facilitate a robust collaboration.

Organize for collaboration. Rather than work on its own, CI should be integrated into business teams, regardless of reporting structures. Our research found that commercial teams think that senior CI manager continuity is critical for the quality of the work across projects. They want someone who knows their brand, market, economics, and strategic priorities—one person with whom they can plan projects and whom they can contact when projects are performing poorly. At the same time, the CI function must make sure its insights and analytics are defensibly top quality, which will further strengthen business teams’ confidence in CI’s value. But while linking CI professionals to specific business teams makes sense, pools of centralized resources should also be available to handle large or unpredictable projects, or when specialists, such as data scientists, are required.

Companies with stage 4 CI functions are out in front when it comes to incentivizing effective collaboration between the business and CI. They revisit their performance management and compensation metrics, incorporating consumer metrics such as customer satisfaction for a broad set of senior business leaders in the organization beyond CI.

Cultivate the right leadership skills. To thrive in today’s unpredictable business environment, companies must ensure that CI leaders and practitioners can navigate ambiguity, adopt more creative methods, and leverage new data sources. Perhaps unsurprisingly, stage 4 companies focus on developing these capabilities as well as strategic-thinking skills among CI leaders.

While the CI function will always need researchers and analytics experts, companies should create executive tracks outside of CI for high-potential members of the function. The opportunity to move from CI to a business role sets an aspirational tone for CI and its role within the organization. And executives from the business should be able to move into CI in order to gain experience in customer-centric roles before perhaps moving on to brand, regional, BU, and corporate general management roles.

To support the evolution of CI leadership, CI functions should consider creating an advisory board of high-level executives, academics, and close external partners such as key market research suppliers that meets once or twice a year. They also should consider forming a cross-discipline committee or sounding board of customer- centric, innovation-oriented internal executives who demonstrate the desired engagement model between CI and the business. This group should meet two to four times a year in person and more frequently by phone or online.

These boards can provide critical mentorship to CI leaders. They can give the CI function input on priorities; on technological, methodological, and academic trends; and on the engagement model. They can also provide access to a group of early adopters of new CI tools and approaches. In addition, the boards can help identify where new technologies and approaches are most effective and help CI managers better understand how business units operate and make decisions. For their part, the members of these boards gain a deeper understanding of functional priorities and of innovations in customer insights and commercial analytics.

At the same time, heads of the CI and commercial analytics groups should be in the habit of conducting their own meetings with the leaders of the brands, regions, and functions they serve. The frequency of these meetings, and the management levels at which they occur, will depend on the size, complexity, and scope of the company as well as the scope of the CI function.

Give CI control over its budget. For organizations in which relationships among business owners, managers, and CI practitioners are being rewritten, giving CI control of the insights and commercial analytics budget can drive meaningful, structural change. It helps create a culture of constructive checks and balances among equally powerful functions. As a result, CI and business teams are encouraged to continue to strengthen their collaboration rather than retreat to their silos.

Leverage CI’s enormous knowledge base. There is a vast trove of valuable analysis, context, and perspective within CI in the form of existing reports and projects. The trick is to make all that accessible and useful to the business.

Stage 4 CI functions capture and provide access to data from many sources—including previous studies and analyses, point-of-sale records, and other digital and passive sources—and integrate it into one comprehensive source. The goal is to combine structured data with rich unstructured, qualitative data, including information on consumers’ emotional needs, at the household or customer level. Increasingly, CI teams in large, global enterprises are able to support business decisions without incremental studies across the portfolio of brands or regions. They closely match the cost of CI involvement to the upside it will provide, and they leverage “close enough” studies, facilitating the flow of information across the organization and relying on suppliers as ready sources of information.

Revisit the outsourcing model. Because CI functions may outsource many activities, such as the generation of high-quality data, qualitative research projects, and predictive modeling, they must ensure they have a sound model for doing so. The CI function should evaluate four areas of its operating model related to outsourcing:

• Supplier identification, qualification, and selection
• Negotiations and supplier management
• Scope of the work of suppliers and the degree of integration suppliers have with CI as well as with the business
• The final work product, data, and tools suppliers will deliver

At stage 3, CI qualifies suppliers, employs both master and project-based supplier agreements with negotiated rates, maintains internal feedback on suppliers, and meets annually with suppliers to discuss innovation, capabilities, and joint planning or objectives for the year. CI functions at stage 4 typically decide which suppliers to use and negotiate discounts or services in performance disputes. At both stages, companies strive to develop deeper relationships that increase the effectiveness and the benefits of outsourcing. They issue RFPs on projects over certain thresholds, even in an expedited form. In some cases, supplier team members are dedicated to and integrated with their customers’ CI and commercial analytics teams.

Stage 3 and 4 CI functions are shifting what they are willing to outsource. After our 2009 study, many companies outsourced more CI activities, particularly related to proprietary data and data analytics. This created several unintended consequences, including a perceived decline in the quality and distinctiveness of the insights generated, loss of control over important proprietary data, and an erosion of CI integration with the business. So while stage 4 CI teams are outsourcing basic analytics and even entire projects that are tactical, backward-looking, descriptive, or real time, they are increasingly bringing innovative, forward-looking, prescriptive studies back in-house.

Companies should ensure that suppliers have appropriate safeguards, such as nondisclosure agreements, when working with competitors. Terms should be clear and consistent on ownership of things such as algorithms, customized tools, databases, and custom analyses, and on what existing data, algorithms, reporting, and visualization tools are excluded from protection. Companies should also examine service providers’ standards for data security and policies for addressing breaches. Increasingly, companies with stage 4 CI functions are prohibiting the use of their data in normative databases or limiting the level at which it can be reported separately.

By taking the steps outlined above, companies can improve the integration of their CI functions into the business and thereby improve innovation. By strengthening this relationship, consumer-facing companies can integrate customer insights into their business operations and ensure that the valuable trail of customers’ digital activities and behaviors doesn’t grow cold. Failure to rapidly translate this data into business strategy will leave an opening to smaller digitally native entrants to do so. And that will make maintaining share and finding growth impossible for incumbents.