At a time when many Americans are thinking about the future of their communities, one important ongoing issue is the reform of the criminal justice system. New methods of research and data analysis offer evidence for improved ways to address long-standing problems in the prison system. These analytical methods can also help design programs that correlate more closely with effective outcomes.

Consider the goal of reducing prison recidivism. Reducing recidivism can contribute to stabilizing the lives of formerly incarcerated people and their families, lowering the costs associated with incarceration, and reducing crime rates.

As part of an analytics-based research effort with the US Department of Justice (DOJ), our team at BCG explored a data-driven approach to evaluate and improve in-prison efforts to reduce recidivism. Recidivism is defined as the rate at which individuals return to prison after release from former incarceration. The study found that improvements in the recidivism rate correlated with some prison training programs more than others.

For example, programs that included cognitive-behavioral therapy components were significantly more effective in reducing recidivism. We found that the rates for federal incarcerated people could be reduced by more than 10%, at a cost of less than 1% of the total Federal Bureau of Prisons (BOP) budget, if the right programming is applied rigorously to the right people in the right way at the right time.

The four analytical methods deployed in the study can help address many intransigent problems facing federal and state corrections departments. The analysis techniques we used can identify what works and which populations need the most help, even without full or perfect data. Most important, these methods are more likely to avoid explicit or implicit biases and to highlight promising solutions previously overlooked.

A Context for Reform

Managing recidivism is a form of prevention, similar in some ways to disaster preparedness. A 2016 study by the National Institute of Building Sciences found that for every $1 spent on hurricane, flood, earthquake, or wildfire preparedness, $6 are saved in disaster relief costs. Similarly, states that have committed to aggressive programs to limit both incarceration and recidivism have seen a reduction in prison staff and prison closures, leading to financial savings and decreasing crime rates. Providing federal funding to state and local prison bureaus for programs that reduce recidivism could thus yield similar gains.

One stumbling block has been a lack of reliable information about effectiveness and quality for the large number of programs designed to smooth reentry and reduce recidivism in the federal and state prison systems. At various times there have been as many as 11,000 BOP programs aimed at improving reentry outcomes, nearly all originating at local facilities and run at local discretion. These have varied in quality, rigor, and access, and only a few programs have shared their results or insights with others. Moreover, only a handful of national programs have been evaluated in terms of their impact. The data about inmate learning choices, long-term outcomes, and recidivism rates is thus incomplete or scattered.

A Prototype Research Initiative

Research in recidivism has often been inconclusive because of these gaps in the collection of data, disparate data sources, and few longitudinal studies that cover change over time. In launching its analytics effort with BCG, the DOJ was looking for a data-driven assessment of what works.

The study began by articulating questions that defined the research goals:

• What programs did the BOP provide? Which were the most effective? Where were there gaps?
• Who needed to receive these programs, when, and in what “dosage”?
• How could these programs be provided in the most effective way?

To answer these questions, the team analyzed and created data sets, reviewed the major academic research, government reports, best practices, and benchmarks for reducing recidivism. It conducted interviews with wardens, released individuals, BOP personnel, criminologists, and other practitioners.

First, the team developed a taxonomy of programs, outlining the full spectrum of what programs should be available in prison systems. The researchers outlined which types of programs were most correlated with reductions in recidivism (for example, cognitive behavioral therapies and jobs programs).

Then, the team used three different data sources to assess the needs of the incarcerated population:

1. Basic attributional data such as age, reason for incarceration, education levels, and basic medical and mental health needs. This offered a preliminary perspective on factors such as the need for alcoholism and substance abuse treatment and further education.
2. To simulate proxy data for the BOP incarcerated population and its greatest potential leverage points for reducing recidivism, the team reweighted an existing state-level data set of incarcerated population programming needs. That way it could approximate the makeup of the federal prison system.
3. The team also leveraged a US Courts-run risk-needs assessment of the released federal prison population as a proxy for unmet needs while incarcerated.

Using these three data sets, a fuller picture emerged of the major programming needs for this population (and critical subpopulations, such as women, the mentally ill, etc.) along with the needs currently not being met by the existing programming system.

Similar methods can be used to assess any criminal justice system, at local or broad scale. As data is gathered over time, patterns of correlation can be used to identify the features that represent success. Qualitative expertise is also used to help draw meaningful assessments from quantitative correlation. For example, a small group can be brought together to make sense of the data: to discuss and interpret the patterns that emerge, to pressure-test and validate their significance, to ensure equity and mitigate bias, and to credibly generalize from the specifics to broader conclusions.

These conclusions can be used to make changes to policy, to adjust the roster of vendors, or to introduce and test new programs. The use of predictive analytics—algorithm-driven conclusions about potential future events—applies data from the past to better plan for the future. A justice system could prioritize the needs of incarcerated people and anticipate which programs will likely be effective and should be increased. These conclusions can then directly influence program design and evaluation.

A Path Forward

Organizations and processes in the criminal justice and corrections systems must also evolve to optimize around emerging analytical or technical capabilities. Beyond the sample findings we already described, as well as a review of what has worked well in the system to date, the study also recommended changes in existing processes that should accompany more advanced analytics. In adapting organizational structures, it’s helpful to apply the metaphor of a bionic organization: blending the strengths of human skill and technology to build dramatic new capabilities. These recommendations fall broadly into the following four categories.

1. Recognize and meet individual needs. When an individual enters the prison system, the intake assessment should gather attributional information: the individual’s recent employment status, educational background, family status, level of family support, key components of physical and mental health, addiction status, and financial stress level. This data is instrumental in matching training and rehabilitation programs to the individual’s needs.

The same level of insight and personalization should also take place for those who exit the system. Beginning within a year of scheduled release, incarcerated people can benefit from specific classes in basic life skills (such as financial management), family and relationship skills (such as managing social networks), and employment preparation.

2. Measure effectiveness and expand the programs that work. The training programs most frequently offered to inmates involve job skills, but research suggests programs that seek to modify behavior may play an equal or more significant role in decreasing recidivism. Risk factors for formerly incarcerated people include issues related to anger management, associates who have been imprisoned, family and marital discord, and substance abuse.

Cognitive-behavioral therapies (CBT) can assist in reducing the risk from some of these factors and help to provide additional coping skills, such as training in emotional regulation and impulse control skills. CBT training is directly correlated with reductions in recidivism, yet too few incarcerated people receive the amount of it they need. Often, this training is delivered through programs targeting substance abuse or sex offenses, which limits the number of participants. Other incarcerated people would also benefit from basic cognitive skills training. More intensive programs may also be needed for people with moderate mental health problems. These individuals could be offered counseling and therapy programs.

To ensure participation, completion of CBT programs should be a prerequisite for enrollment in occupational training programs. Participation could also be linked to increased privileges or perhaps reduced sentences. Moreover, inmates should have ongoing access to “booster shots” of refresher programming. Finally, their response to these critical programs should be tracked to better understand—and improve—the effectiveness of the training portfolio.

3. Raise the quality of the program portfolio by sharing best practices. Prison systems should conduct random audits to continuously improve the quality and accountability of all categories of training programs: educational, occupational, cognitive, and behavioral, and those oriented to life skills and release preparation. The list of offerings should be kept up to date and relevant to incarcerated people’s current needs.

For example, job training design should not be the responsibility of an individual warden. It should be regionalized or nationalized, taking advantage of academic resources outside of the immediate area and distance learning. The data can give guidance on which jobs have the highest potential for steady income for formerly incarcerated people in different regions of the country. It can also help provide direction for further job programming and broader systemic investment. As one warden of a high-security facility said, “I’d love to offer something like carpentry, but I just don't have the kind of equipment, facility, or space to do that.” Accurate, up-to-date data can help guide resources into the right programs.

4. Leverage data to anticipate staffing and other resourcing needs. All of these practices depend on having skilled social workers, educators, and other staff members in place to develop, oversee, and deliver effective programs and services. Both CBT and job training are limited by the number of available staff, both in rural areas (where the talent pool may be more limited) and urban areas (where prisons must compete with higher salaries offered elsewhere).

Social workers, for example, can make a dramatic difference, supporting family ties and other external opportunities, and increasing incarcerated person check-ins beginning two years before release. A dynamic HR strategy is needed to hire, train, and motivate additional psychologists, treatment specialists, clinicians, social workers, and other specialized staff. Data analysis can help to identify how many staff—and with what credentials—are needed to carry out the given programming strategy.

Many facilities need other types of support beyond staffing. The data shows that upfront investment in unit team training, risk assessment tools, and recruiting tends to pay off dramatically in recidivism results.

The World of Data-Driven Services

The four steps we have just described represent the beginning of a path on which data can be leveraged to influence programming decisions and ultimately criminal justice outcomes. This new way of working applies data and analytics-driven strategies to provide better outcomes not only in all kinds of justice systems but also in other areas of government.

In general, analytics will and should be used for a wider variety of government-based interventions in social programs and social justice. The power of the purse is one of the strongest vehicles that governments have, and analytics can make a great difference in fields like housing, infrastructure, health care, and infrastructure. Although analytics do not and cannot replace human judgment, they can be a valuable tool to use alongside it. In the end, analytics offer an important starting point for thinking about how to manage any agency or institution where seemingly small decisions can have significant consequences for individuals and their communities.

The authors would like to acknowledge Olivia Anglade, a project leader in BCG's Washington, DC office, for her contributions to this article.