Conquering the Challenges of Agile at Scale in Government

Improving what governments deliver to citizens and how they deliver it is no easy task. Citizens’ expectations are rising sharply, as savvy startups and global technology companies set high standards for digital services. Meanwhile, complex and often chronic challenges such as homelessness and long-term unemployment make it hard for governments to respond, particularly when public-sector structures are rigid and may not be suited to the collaborative approaches needed to drive rapid change. With their eyes on prizes such as greater customer centricity, speed to market, efficiency, improved adaptability, and a more engaged workforce, governments are starting to adopt agile ways of working. But with obstacles lying ahead, the question is—how will governments make the transition?

As BCG argued in its first article on agile in the public sector, “Agile as the Next Govern­ment Revolution” (August 2018), the public sector can reap many benefits from agile—an organizational paradigm in which small, multidisciplinary, empowered teams work in iterative, fast-paced cycles while constantly prioritizing value. An agile-­at-scale transformation occurs when an organization takes this approach beyond projects or isolated pockets (such as IT) and uses it as the primary means of delivering a core business activity (for example, designing and building new services).

Scaling agile is not straightforward, and doing it poorly can be worse than not doing it at all. In the public sector, complex stakeholder environments, a wide range of objectives, long-established power structures and processes, a risk-averse culture, and labor union influence present significant hurdles to implementing agile at scale. Nevertheless, these obstacles can be overcome. By setting goals and aligning the organization closely with their objectives, governments can empower agile teams and give them the autonomy they need. Creating alternative funding models will support agile teams in developing new solutions through experimentation and iteration. And designing new talent management strategies will enable public-sector workforces to attract, develop, and manage the different capabilities needed to drive rapid improvement in government services.

Empowerment Meets Alignment

For scaled agile transformations to succeed, agile teams must be tightly aligned to organizational objectives and empowered to achieve them. This means setting clear missions for teams, delegating decision rights to foster autonomy, equipping teams with the necessary resources and multidiscipli­nary staff, and establishing measurement and governance frameworks that focus on results. Much of this is challenging to implement in government.

In public-sector contexts, the complexity of policy making can introduce obstacles to empowering and aligning agile teams. Whereas private-sector organizations have clear overall objectives—increasing profita­bility and maximizing shareholder value—public-sector organizations have a larger set of stakeholders to satisfy and more complex social, political, and economic objectives that may be more subject to change. Short-term election cycles and shifts in political priorities can complicate efforts to set enduring missions for agile teams.

It can also be difficult to empower teams in organizations that need to be responsive to senior elected officials. Traditionally, public-sector organizations have had strongly hierarchical management structures, centralized decision making, limited delegation of authority, and chain-of-command reporting lines. As a result, many teams are highly attuned to their political leaders and can be reluctant to delegate decision making or determine independently the best path to take to achieve agreed objectives. Moreover, the focus on responsible stewardship of public resources often combines with a culture of risk aversion to slow down change and require constant consensus building at senior levels of the organization.

Added to these factors are constraints re­lated to public accountability to legislative bodies and the need to meet audit and record-­keeping requirements. Heavy reporting burdens and the need to follow certain procedures can slow or disrupt the work of agile teams, as can norms regarding on­going documentation and reporting (for example, intradepartmental briefings, audits by other government bodies, and risk and impact assessments) and centralized procurement processes.

We recommend that public-sector organiza­tions adopt the following four measures to overcome these challenges:

• Combine tightly aligned objectives with delivery flexibility. Internal and external stakeholders (including politicians) need to be tightly aligned on overall outcomes and success measures. Nevertheless, it is critical to allow agile teams flexibility so that they can prioritize their efforts in line with value and adjust their strategies as they learn lessons and as technology changes. There is an art to managing this “tight-loose” model, and one important role for senior public-sector leaders is to preserve the flexibility that gives agile teams their autonomy. As Jeff Bezos, the CEO of Amazon says, “We are stubborn on the vision. We are flexible on the details.”
• Set and reinforce new cultural norms. Agile methods emphasize continuous iteration and early testing with users—practices that are alien to many public-­sector organizations. Agile teams should therefore receive coaching in how to fail fast and fail small by testing minimum viable products with a contained group before refining them and broadening their distribution to more users. To combat collective risk aversion, leaders must repeatedly reaffirm that mistakes are part of the learning process. For example, at the World Bank, the senior vice president for operations issued a “waiver” to teams involved in a major agile program to free them from the Bank’s existing internal processes and encourage them to experiment, fail, and learn. In the UK, government departments have hosted events to celebrate the “honorable failures” of their agile teams; and in Australia, an agile team at a government department displayed a “social contract” that all team members had signed stating that mistakes are part of the process.
• Use structural alignment to drive change. Government organizations may need to decisively break away from existing structures to signal cultural change and enable the creation of cross-functional teams. Moreover, enabling those teams to work effectively requires wider changes to accountability, hierarchy, job positions, the physical location of teams, and HR policies. Both an extensive communications effort and stakeholder engagement across the organization are necessary to win the hearts and minds of staff. For example, video documenting the journey of staff in the early stages of a transition can help win the support of the broader organization as the team pushes ahead with changes.
• Build trust and create empowerment. Senior leaders must make it clear that agile teams are empowered to do what it takes to achieve their missions. An audit trail is important, but governance and documentation requirements should be kept to a minimum, and steering committee meetings requiring that papers be submitted weeks in advance should be eliminated. Leaders can perform their governance duties by attending showcases of the work a team has produced in each sprint, preferably in the team’s workspace, and adopting a mindset of encouraging the team and removing obstacles to progress. Simple one- or two-page summaries of key discussion points following each showcase can capture the oversight provided, while the real value arises through in-person com­munication between agile teams and senior leaders. Agile coaching for both leaders and agile teams is vital to achieving the goal of shifting their mindsets and behaviors, and providing the feedback needed to secure the new behaviors.

New Funding Models for Agile Teams

Traditionally, government agencies have been constrained by rigid and lengthy fund­ing processes that support linear, predictable, project-based development. Often, central finance authorities must approve detailed business cases and project plans, particularly given expenditure controls and public scrutiny. Yet in many instances these funding approaches are flawed because the plans create false precision and certainty. Solving complex problems requires iteration and refinement with real users, and continual adjustment as political realities, technologies, and citizens’ expectations change.

Successfully scaling agile demands a fundamentally different approach. In a digital world, governments need flexible funding models that can accommodate experimentation, user testing, and iteration and can give teams latitude to shape the direction of a product or service (within legislative bounds). Instead of exercising project-by-­project control, central government authorities should emphasize an organization’s medi­um- to long-term outcomes and metrics. As long as they meet these metrics (linked directly to value to government or citizens), agile teams should have autonomy to push ahead, and the authorities’ regular discussions of their progress should focus on the value that the teams are delivering.

We recommend taking the following measures to overcome public-sector funding constraints:

• Build close relationships based on trust. Fostering close relationships with central finance ministries and digital agencies can help overcome requirements to submit detailed business cases and project plans. For example, agile teams can invite representatives from central government to participate in showcases and sprint planning meetings. One former executive at an Australian agency stressed the importance of engaging finance leaders from the outset so that they understand and gain confidence that the teams are delivering meaningful ongoing change.
• Create new forms of governance. New reporting structures are essential to give agile teams the flexibility and autonomy they need. For example, the UK government has developed a streamlined business case and simplified approval processes for the initial scoping stages of agile projects with budgets under £750,000 and for the implementation phases of projects costing less than £10 million. In Australia, a large service delivery organization has agreed on a new model calling for less documentation and imposing simpler reporting requirements, in exchange for greater transparency through more frequent progress checks aligned to the organization’s agile delivery cycle. Unless a team fails to meet its performance metrics or veers outside its agreed mission, the default is for the agile team to continue delivery.
• Adopt blended capex/opex models. In the long term, as agile becomes an established way of working in government, change will no longer need to be delivered through large one-off projects and separate teams. Instead, enduring agile teams will continuously deliver change alongside their business-as-usual tasks. This arrangement will require central agencies to rethink their funding models to give agile teams responsibility for both capital expenditure (capex) and operational expenditure (opex). Central finance agencies can adopt blended capex/opex models for teams. Alternatively, public-sector accounting officers could be permitted greater latitude to designate funds to capex or opex. These adjustments will ensure that agile teams have the mandate to deliver on both the development side and the operational side of their missions.

Managing Talent for New Ways of Working

Central government bodies often exercise control over job role classifications, compensation packages, and recruiting methods used to hire public servants. This can make it difficult to adopt more innovative and tailored approaches to attracting, developing, and managing different talent needs for a new way of working.

For an organization to scale agile successfully, people management must be transformed at all stages of the employee life cycle, from recruitment to talent development and performance management. Agile typically requires different skill sets, such as product owners and scrum masters ­(agile team facilitators), alongside user-­experience designers, data scientists, and developers. Expertise alone is not enough; agile organizations need people who are also adaptable, motivated, and effective in cross-functional teams.

Managers—particularly those overseeing IT, legal, compliance, or some other functional area—can find it disconcerting to move to an agile way of working. As cross-­functional teams are established, these managers often have less involvement in setting day-to-day priorities and overseeing the work of functional staff, since this falls to the leadership of agile teams. As more members of their staff become embedded in cross-functional teams throughout the organization, functional area leads must focus on playing a broader coaching and mentoring role, facilitating “communities of practice” for their function, and developing a talent pipeline for the organization.

For government organizations, adapting existing HR processes to attract, develop, and promote the talent needed for agile teams can be a challenge. Leading companies in the private sector can, of course, offer generous compensation packages—but for governments, offering less competitive rates of pay is only part of the problem. Centrally established recruitment processes, pay grades, and job positions restrict flexibility in public-sector hiring. To enable governments to build agile teams, this situation needs to change.

Furthermore, the structure of agile teams tends to be flat, which runs counter to many traditional hierarchies in government, where tenure and experience managing teams often carry more weight than adaptability or technical skills. The compre­hensive HR changes needed in a shift to agile can therefore spark concerns on the part of employees and unions.

We recommend four ways of managing talent to support agile at scale:

• Develop new roles and hiring processes. Government must create roles that are essential for agile ways of working. Hiring processes should be more flexible and tailored to particular skills requirements. These practices have proved successful in the private sector. For example, to support its new way of working, a major Australian bank revamped its hiring criteria, downplaying tenure and instead emphasizing a candidate’s behavioral fit, evidence of teamwork, and adaptability.
• Develop expert career tracks. Running parallel to standard managerial career tracks, expert career tracks can enable public-sector organizations to provide promotion opportunities for highly skilled employees and decouple compensation from management status. This practice, known as the Dreyfus model, is becoming increasingly common in the private sector. In this model, compensation increases with the advances that the employee makes in technical skill. This provides a way to codify skills, compensate employees for more than their management skills, and attract the specialized talent that is essential to agile at scale. Adopting such changes requires central government agencies to play a leadership role in redefining career paths and structures across the public sector.
• Engage unions early and often. Regular communication efforts that are led by seasoned HR staff, that involve unions in the design of the organization’s future state, and that provide opportunities for union representatives to observe agile teams can establish trust and build support. If managed properly, initial concerns about the impact of agile methods on employees and about the possibility that existing roles will become redundant can be overcome. Experience shows that unions can become active supporters of agile as the greater empowerment and satisfaction that employees derive from this new way of working become clear.
• Create a sense of purpose. Governments must promote their organizations as attractive, inspiring places to work. They can appeal to the altruistic motives of the talent they need by emphasizing the potential to make a difference to society. Once they adopt agile ways of working, governments can attract top talent by connecting with communities of agile practitioners, promoting the benefits of an agile culture, and demonstrating their ability to deliver impact quickly.

Enabling Faster and Better Delivery

Implementing agile at scale in the public sector is no easy task. The changes to processes, structure, and culture are significant, and challenges unique to the public sector must be overcome. Moreover, progress must occur in the face of long-standing power structures and constraints that make adopting new ways of working difficult.

Yet public-sector organizations have often had to adapt to new circumstances—and like any organization, they can apply the principles of smart change management. As is true in any major transformation, visible and committed leadership is vital to steer the organization toward new ways of working. This includes creating alignment through shared purpose, empowering teams by building trust, promoting candid communication across the organization, and engaging all stakeholders.

Adopting agile at scale can yield significant benefits. Public-sector organizations moving in this direction can build an enduring capability that delivers service to citizens faster, better, and more cost effectively.