This brief is based on the article GenAI Doesn’t Just Increase Productivity. It Expands Capabilities.
As the world prepares to mark the two-year anniversary of ChatGPT’s worldwide introduction, many CEOs are focused on how generative AI can make knowledge workers more productive within their existing skill set. But a new BCG study reveals that GenAI can also help employees complete complex tasks beyond their current capabilities.
BCG’s Henderson Institute and scholars from Boston University and OpenAI’s Economic Impacts Research Team conducted an experiment involving roughly 480 BCG consultants who volunteered to perform two of three tasks: coding in Python, building a predictive model, and validating statistical-analysis outputs from ChatGPT.
The experiment found that even consultants with no coding or data science background could complete these tasks successfully with the assistance of AI. When performing tasks particularly well suited to GenAI, consultants who used it significantly outperformed those who did not.
The results of the field experiment suggest GenAI can help employees adapt more easily to evolving job requirements. There are, however, important caveats to consider, and CEOs will need to manage the risks to fully reap AI’s rewards.
The greatest aptitude expansion effect that the experiment revealed involved coding—a task well suited to GenAI. When asked to code in Python, AI-augmented consultants performed, on average, roughly 15 percentage points below the benchmark set by BCG’s highly trained data scientists. Even consultants who had never written a line of code previously weren’t far behind. By comparison, consultants who coded without the help of GenAI performed, on average, nearly 50 percentage points below their AI-supported counterparts.
It’s important to note that though the consultants using GenAI had never coded in Python, they were familiar with the basics of the task they were coding for—merging and cleaning two data sets. That baseline knowledge likely enabled them to identify obvious errors in the GenAI output, helping them execute the task successfully.
GenAI proved less helpful for completing the second task—creating and assessing a predictive analytics model. The AI-augmented consultants performed nearly 25 percentage points below the benchmark set by data scientists and were more likely than the control group to be misled by the AI. But the AI tool helped the consultants discover new modeling techniques and identify the correct steps to solve the problem, making it a valuable brainstorming partner.
For the third task—validating, correcting, and assessing statistical analysis outputs by ChatGPT—the AI-enabled consultants performed only 12 percentage points below the benchmark set by the data scientists and 20 percentage points higher than their nonaugmented counterparts.
While the experiment focused on data science, the findings suggest that workers can use GenAI to perform tasks outside of their skill set but within the technology’s capabilities. CEOs may, therefore, want to ensure that GenAI augmentation is incorporated into strategic workforce planning and talent recruitment. For example, companies can think beyond finding a certain number of people with specific knowledge skills, like coding, to whether a current employee could do a specialist job effectively if augmented by AI.
CEOs also need to keep in mind that GenAI does not eliminate the need for employees to learn and develop new skills.
Finally, more than 80% of the consultants who participated in the experiment and regularly use GenAI on the job said it enhances their problem-solving skills and accelerates their performance. This suggests that highly skilled knowledge workers enjoy using GenAI when it helps them feel more confident in their role. The takeaway for CEOs: ensure that GenAI bolsters professional identity rather than undermines it.
GenAI Doesn’t Just Increase Productivity. It Expands Capabilities.
A new experiment shows that GenAI isn’t only a tool for increasing productivity—it can broaden the range of tasks workers can perform.
Daniel Sack is a member of BCG GAMMA, Boston Consulting Group's innovation hub on AI and digital topics. He specializes in building digital products based on machine learning/AI as well as building the teams that build [KD1] [HK2] them, primarily in consumer industries. Dan co-leads GAMMA's PLAN AI efforts integrating advanced analytics into end-to-end planning processes. He also co-leads GAMMA's Delivery Excellence (DevEx) program, a system of processes and governance that ensures each team maintains the highest standards for software code and technical solutions, including Responsible AI.
Lisa Krayer is a core member of Boston Consulting Group's Technology, Media & Telecommunications; Technology & Digital Advantage, and Marketing, Sales & Pricing practices. She has conducted in-depth research into the impact of generative AI on individuals and businesses for the BCG Henderson Institute. She has also supported thought leadership in quantum computing and other emerging/deep technologies.
Emma Wiles
Assistant Professor of Information Systems, Boston University’s Questrom School of Business
Related Content
Explore more insights from The CEO Agenda and BCG’s experts.
The latest generation of space-based technologies can help companies across industries lower costs, improve productivity, drive growth, and meet sustainability goals. Here’s how CEOs can capitalize.
BCGは、コンサルティング業務における生成AIの活用を検証する最新の実験を行いました。それによると、人間は生成AIが大きく価値貢献できる分野では不信感を抱き、逆に生成AIに適性がない分野では期待を寄せすぎているようです。
Some two thirds of transformation efforts fail. Here’s how CEOs can improve their odds.
The international order hasn’t been this uncertain in decades. CEOs must approach geopolitical risk with the same urgency as digitization, AI, and the climate crisis.