The CIO’s Role in AI Value Creation

Across industries, generative AI (GenAI) has generated excitement—and there is little wonder why. By creating content, on demand and on point, GenAI accelerates many tasks, functions, and experiences that move businesses forward. But GenAI is still in its early stages, and relatively few organizations thus far have managed to unlock its potential. Research conducted for our report “Where’s the Value in AI?” reveals that just 26% of companies are creating real value from AI. Many more are struggling to move beyond the proof of concept (POC) stage. And more than 50% say that legacy IT architectures are hindering by limiting scaling, reusability, and data availability. (See the sidebar, “About the Global Study.”)

For chief information officers (CIOs) or other C-suite leaders in charge of technology, GenAI is exerting pressure to deliver concrete benefits from the technology. But it is also creating two opportunities.

First, GenAI has significant implications for the tech function, helping CIOs optimize their company’s IT productivity. A key finding from BCG’s study is that about a third of the value potential of scaling AI lies in enabling functions such as customer service, procurement, and IT. Companies that have implemented high-value GenAI use cases—and there are some compelling ones for tech—are already achieving savings of up to 10% in IT. But so far, relatively few CIOs are tapping that vein.

Second, GenAI offers CIOs an opportunity to expand their circle of influence within the organization. To succeed with GenAI—and AI more broadly—companies need to tackle and communicate risks, build a foundation for scaling, and create clear policies governing AI tools, uses, and regulatory compliance. None of this is easy, but CIOs are uniquely positioned to lead on these critical topics. By doing so, they will not only adopt GenAI appropriately, but also help establish IT as an innovation partner of the business, instead of just a service provider. That’s crucial, as it’s the defining trait of a world-class tech function and of a company that leads.

Optimizing IT Productivity

One difficulty that many companies have encountered with GenAI involves identifying where the technology’s value lies: the use cases that provide essential capabilities and drive critical efficiencies. Within the tech function, that question is becoming easier to answer. (See the sidebar “GenAI in Tech.”)

Paths to Value

Three types of GenAI use cases are particularly compelling. First, programming assistants, such as GitHub Copilot, can boost software development productivity north of 30% when deployed smartly. Second, IT service desk tools enable self-service for common issues and automate tasks such as ticket resolution and incident management; for instance, a global insurance provider that we’ve worked with estimates that GenAI can handle 75% of its service desk tickets by managing repetitive, low-complexity issues. Third, GenAI excels in knowledge management. Natural language interfaces—which, in effect, let users talk to data—provide intuitive access to content such as IT system documentation, development status reports, and product data and descriptions—especially in combination with a retrieval-augmented generation (RAG) system. RAG supplements GenAI’s training by referencing specific content, such as updated information or documents, outside the training data.

Another area where GenAI can provide valuable enhancements is cybersecurity . As it sifts through security logs, email, and other unstructured data, the technology can spot threats that more traditional systems might miss. Unfortunately, GenAI brings efficiencies to the bad guys, too, and with respect to cyber budgets the two trends may cancel out each other. A further benefit of GenAI lies in accelerating agile development by assisting in tasks such as sprint planning, backlog management, and status reporting.

Currently, however, relatively few organizations are reaping these benefits. BCG’s study found that cross-industry adoption rates rarely broke out of the single digits. (See Exhibit 1.)

This leaves a lot of value on the table. By our analysis, GenAI use cases can address up to 50% of tech function costs and deliver efficiency gains of up to 30%. Of course, GenAI itself has costs, centered largely on the computing requirements of model training and inference, as well as model maintenance and licensing. All told, organizations should anticipate a 5% increase in expenses. But it’s a sound investment because—factoring in addressable costs, efficiency gains, and additional expenses—it can yield net cost savings of up to 10% in the tech function. (See Exhibit 2.)

As good as that sounds, cost reduction is just part of GenAI’s value proposition. Efficiency gains enable CIOs to devote more talent and resources to key initiatives in areas such as innovation, customer experience, and digital transformation. And by accelerating or even automating repetitive, time-consuming tasks, GenAI lets teams focus more on strategic work to drive revenue, enhance competitiveness, and increase the overall impact of the tech function.

What the CIO Should Do

By activating GenAI within the tech function, CIOs can spark significant and visible value creation. This not only drives impact within IT, but also helps secure organization-wide buy-in and adoption of GenAI.

Success requires a holistic approach—one that goes beyond technology. Indeed, organizations that successfully deploy AI at scale devote the bulk of their effort to process and people. We call it the 10/20/70 rule: allocate 10% of the effort to data science capabilities for developing and implementing algorithms, 20% to a modernized and scalable tech stack, and 70% to effective processes supported by talent and robust change management. That crucial 70% is often overlooked or undercooked.

Our playbook for realizing the full benefits of GenAI in the tech function includes five key action items.

Focus on value. CIOs should prioritize use cases on the basis of value. This means looking at addressable costs, but also at expected efficiency gains. A GenAI use case that addresses a large cost but offers a modest boost in efficiency may rank lower than a use case that addresses a smaller cost but promises a dramatic improvement in productivity. But CIOs shouldn’t shy away from taking a bold approach, tackling the big rocks as well as the smaller and easier areas. CIOs should consider how a particular use case aligns with their company’s business strategy. An application that furthers a key business goal can deliver value that matters most.

Establishing a procedure to evaluate, prioritize, and monitor use cases is crucial, as it ensures optimally effective deployment of the company’s efforts and resources. A Fortune 50 US company created a central team to focus on value from GenAI initiatives. It identified more than 200 ideas and segmented them into three groups on the basis of business impact. The central team prioritized and executed the highest-impact ideas, but channeled the medium-impact ideas through existing delivery processes and designated the lowest-impact ideas for experimentation within defined guardrails. The company tracked each initiative in terms of the value it generated and discarded those that failed to meet expectations.

Rethink processes. GenAI provides an opportunity to reimagine processes—a so-called zero-based approach. GenAI automation will deliver only so much value if baked into a legacy process that retains its legacy inefficiencies, duplication, and non-value-adding steps. The better way: rethink the process from scratch. CIOs should aim to minimize complexity and build only necessary steps and capabilities, then automate where possible with GenAI technologies. In the end, value comes from eliminating what you don’t need and automating what you do.

Adopt new strategies for negotiating complex vendor contracts. GenAI use cases can address many internal IT costs. But the technology also lets vendors automate processes and allocate resources more effectively—opening the door to benefits for providers and customers alike. Although contract structures typically offer limited flexibility to share in the gains, CIOs can adopt new negotiation strategies that create a “win-win.” For fixed-price contracts, aligning payments with performance or outcomes can be an effective solution. For managed service agreements, one option is to consider usage-based models or scalability options that introduce greater flexibility in AI-driven environments. By adopting these approaches, CIOs can indirectly address nonaddressable costs.

Invest in change management. GenAI lives and dies on adoption rates. Merely providing new tools, regardless of how cutting-edge they may be, isn’t enough. AI future-built companies—the 4% of study participants that are already creating substantial value with AI—realize the amount of change management required. Two-thirds of these organizations invest heavily in upskilling. Another best practice: prioritize tools that address real worker problems and reduce toil. Such tools drive efficiency while also boosting morale and satisfaction, which in turn further increases productivity. And get the word out. Communicating the benefits of GenAI—and better yet, showing the benefits through workshops, coaching, and centers of excellence—helps spark adoption.

Have the right talent in place. By keeping key strategic capabilities in house, organizations maintain control of and agility in GenAI innovation. To that end, CIOs should build internal talent, forming teams that have the skills and knowledge necessary to drive increasingly advanced GenAI initiatives. But balance is crucial. We find that a hybrid approach— strong internal teams collaborating with selected vendors —works better than opting for fully internal or fully outsourced teams. That said, the home team should retain capabilities that are mission critical or drive competitive advantage.

GenAI brings an opportunity and a need for CIOs to introduce new roles and reimagine existing ones. Among the most critical emerging positions are chief AI officers to oversee initiatives, ethics and compliance officers to design and implement policies, and prompt engineers to refine and analyze prompts for assessing model performance. Existing roles that should evolve include software developers and architects, who will need to integrate solutions with AI frameworks; cybersecurity specialists, who must address new security and data privacy risks; and product managers, who need to gain a level of proficiency in understanding GenAI’s capabilities in order to leverage them to solve business problems.

Creating new roles and showing, not telling, that the company is leading the way on innovative, responsible use of GenAI are crucial for recruiting and developing essential talent. Consider the case of a Fortune 100 company building proprietary GenAI tooling. Like other businesses staking out ground in the GenAI space, it had no inherent “right to win” in the GenAI talent market. So it took a new and differentiating approach. The company established an AI center of excellence with more than 30 new full-time specialized employees, created roles such as AI engineer and responsible AI lead, drafted new interview protocols, and—to assure candidates that it wasn’t just “talking the talk” on GenAI—developed an employee value proposition and recruiting message that emphasized its commitment to responsible AI (RAI) and to real GenAI implementation, not just strategy. Together, these steps helped the company bring best-in-class talent to the table.

Positioning IT as an Innovation Partner of the Business

GenAI investments are expected to grow some 60% over the next three years, accounting, on average, for 7.6% of IT budgets by 2027. So far, however, as just a quarter of companies have moved beyond POCs and are realizing actual value, returns remain limited. By quickly and visibly reaping benefits from GenAI, the tech function can become a showcase for the technology, helping build momentum for it across the organization. But momentum isn’t enough. CIOs must lead.

BCG’s research on AI adoption reveals a key differentiator between AI leaders and laggards: CIOs in a leadership role. At 86% of AI future-built companies, IT leads or co-leads with the business on GenAI. For AI stagnating companies (those ranking lowest in AI maturity), the corresponding figure is 54%. (See Exhibit 3.)

Three Key Challenges

But what form should that leadership take, exactly? In our view, the CIO is the ideal point person to take on three major challenges that hinder AI and GenAI adoption and scaling:

• Complying with Emerging Regulations. GenAI’s ability to learn from, use, and generate content creates risks: hallucinations (false or misleading responses), bias, and misuse of data (for example, revealing sensitive information). Regulations are quickly emerging, making compliance more complex, especially for companies that operate internationally. In the European Union, the AI Act requires companies to classify AI applications by risk level, register each high-risk AI system in a public database, implement risk management procedures for such systems, ensure transparency, and maintain data governance. In the Asia-Pacific region, emerging regulations mandate transparency, accountability, and ethical use of AI systems. In the US, as of August 2024, 31 states had passed some form of legislation, with widely varying provisions, according to the Cato Institute. And more is coming. Companies need to determine which regulations apply to them, ensure compliance with those rules, and confirm that they are properly serving their customers, shareholders, and values. 
• Ensuring Scalability, Reusability, and Data Availability. Many organizations don’t harvest the value they expected from AI and GenAI, or they struggle to move from POCs to implementation at scale. The most common culprits are technical debt, difficulty accessing data, platforms that don’t play well with AI technologies, and slow and costly isolated development of individual use cases. Siloed, hard-to-access data is a major problem: AI and GenAI models are only as good as the data on which they train or act. And organizations often build use cases on different tech stacks, which reduces overall implementation efficiency. Without an application and data architecture that enables scaling, organizations may find that, wherever the value of AI lies, it lies out of reach. 
• Avoiding New GenAI-Related Shadow IT. While many companies struggle to move beyond POCs and minimum viable products, their employees may take the initiative to try out new capabilities on their own through tools such as ChatGPT. This is a double-edged sword. On the one hand, these first movers may serve as an excellent source for identifying use cases that drive efficiency and innovation. But on the other hand, adoption outside IT’s knowledge and control inevitably creates shadow IT. Unsanctioned use of GenAI tools can expose an organization to risks related to data privacy, security, and compliance. It also leads to uncoordinated tech investments. And as companies have no way to know what is actually working—and hence what tools and uses they should be promoting—success stories have limited impact. At one software company, employees widely adopted GitHub Copilot in order to seize the benefits of GenAI-powered programming assistants. But they did so through private accounts, which hindered the company’s ability to ensure responsible and safe usage, and made it difficult for leadership to measure efficiency gains, as no baseline had been established. 

All three of these challenges are in a CIO’s wheelhouse. AI in general and GenAI in particular operate in a rapidly changing technical landscape. And the technology is strongly linked to an evolving regulatory environment, as is another area where the CIO leads: cybersecurity. By owning these challenges and their solutions, CIOs can strengthen the organization’s ability to scale GenAI use cases and, more broadly, to spark AI transformation. In the process, CIOs strengthen their own role.

What the CIO Should Do

CIOs can seize the opportunity to play an active role in helping accelerate AI transformation by undertaking three critical initiatives.

Lead on RAI. A strategic imperative, RAI helps companies minimize the risks of AI and foster transparency, accountability, and ethical use of the technology. It’s a focus area for GenAI leaders: 74% of AI future-built companies continuously monitor compliance with their RAI framework. By leading RAI in their organizations, CIOs can ensure that compliance isn’t thrown over the fence. And they can reduce concerns that often hamper buy-in for GenAI initiatives.

Leading doesn’t mean acting in isolation, however. In designing and implementing RAI programs, CIOs should work with a cross-functional team that includes representatives from business units as well as legal, HR, and other functions. This will ensure that RAI practices fulfill legal requirements and promote corporate goals and values—and just as crucially, it will help build support and a mandate for driving RAI across the organization.

CIOs should stress clarity by establishing and articulating rules that complement existing controls. They should also help create a standard definition of AI across the organization. By aligning the organization on what AI means, CIOs can ensure that everyone understands it the same way.

More broadly, CIOs need to transition from managing software development and operations to leading AI-driven innovation, where compliance becomes as much a focus as agility and customization. This means building teams that not only are proficient in AI and technology but also can navigate the regulatory landscape and work within robust risk management frameworks.

Establish a modular, scalable architecture that supports reusability. CIOs can help organizations move away from a patchwork of silos to an enterprise AI foundation —one that lets companies quickly and effectively scale AI and GenAI use cases, and supports the full organization—not just individual units or adopters. (See the sidebar “Built to Scale.”) This foundation has four key components: foundation models that power GenAI and can serve as the basis for more customized large language models (LLMs); GenAI platforms that orchestrate, manage, and provide access to LLMs; a data layer that pools information from across the organization; and a self-monitoring capability to promote continuous improvement.

But CIOs can introduce another enabler, as well: archetypes. We’ve found that use cases, regardless of industry, tend to align with one of eight archetypical AI applications—such as a knowledge assistant with retrieval augmented generation that uses company-specific data to respond to prompt-based queries. The key is to build the foundation for each archetype first. Since the use cases that map to the archetype will share a similar structure and requirements, the organization can readily tweak and reuse it for each use case. In effect, the archetype becomes a blueprint for efficient deployment.

Using this approach, a large European company mapped more than 300 use cases to archetypes. Instead of building a multitude of use cases one by one, it built the archetypes and then easily reused them. Because the company didn’t have to develop each use case separately, implementation was faster and cheaper. And the business functions often executed implementation directly. The blueprint helped ensure consistency in model usage, cost management, and operational efficiency—all while accelerating the path to scalable AI solutions.

Develop governance that keeps shadow IT at bay. CIOs can help companies reap the benefits of experimentation without creating a zoo of tools. By establishing a clear governance framework and consolidating tools, CIOs can channel innovative efforts productively and safely. This arrangement encourages early adopters to experiment with approved technologies and identify value-creating use cases. But all such efforts occur within a secure, controlled environment that ensures oversight and accountability. CIOs should also capture and share successful implementations, enabling the organization to learn from these experiments and scale solutions effectively and responsibly.

GenAI is getting a lot of attention in the press, the boardroom, and the market—but it needs even more attention from CIOs. Besides sparking transformative change across an organization, the technology can transform the CIO’s role, influence, and impact. CIOs who lead the charge on GenAI will capture value within the tech function, enable the organization to do the same, and help steer and shape its future.

The authors would like to thank Sabine Doeschl, Marc Roman Franke, Alisa Ilina, Iivari Karrila, Kim Kirchbach, and Mariana Zehender for their valuable contributions to this article.