Five Secrets to Scaling Up Agile

Companies across many industries are struggling with the transition to Agile, a fast, iterative software-development method. Too many companies have the appearance of Agile—with, for example, hastily converted, brightly colored meeting rooms and daily stand-up meetings—but they achieve little of tangible impact.

At software start-ups in which Agile is commonly used, the development team is at the heart of the business, so buy-in, sustained commitment, and collaboration come fairly naturally. It is not easy, however, to integrate self-directed, cross-functional Agile teams into the existing hierarchy of large companies.

Some large companies, however, are figuring out how to make Agile work. Rather than impose its specific methodologies, they apply its general principles, paying special attention to the integration of Agile teams into the rest of the organization. 

When large companies get Agile right, the results can be stunning. Productivity can improve by a factor of three. Employee engagement, measured in quantitative surveys, increases dramatically too. New product features can be released within weeks or months rather than quarters or years. Rates of innovation rise, while the number of defects and do-overs declines. In the first year after going Agile, one bank’s development team increased the value delivered per dollar spent by 50%, simultaneously cutting development time in half and improving employee engagement by one-third.

As the quality of software rises and the responsiveness of processes improves, some companies are applying Agile principles to activities other than software development. For such companies, Agile can become a journey of continuous improvement.

Why Agile?

Agile grew out of a desire to improve traditional methods of software development. Customarily, software has been developed sequentially, with the waterfall serving as a rough metaphor for its progression. Separate groups conceive, design, build, test, put into operation, and maintain software, each group waiting for the preceding group to complete its work. The method is inefficient. In many cases, participants spend more time sitting in meetings and managing handoffs across organizational boundaries than writing and testing code. According to the often-cited The Chaos Report less than 10% of large software projects come in on time and on budget. ¹ 1 The Standish Group International, The Chaos Report, 1995. Notes: 1 The Standish Group International, The Chaos Report, 1995.

The waterfall method comes from engineering, but writing many types of software is different from building a bridge. A river doesn’t change its course, but software users have frequently changing and unpredictable needs. Consequently, Agile relies on bringing together many different points of view and supporting back-and-forth dialogue between developers and business executives.

Many forms of Agile have been developed, but at its heart, Agile is a set of beliefs. It is iterative, empirical, cross-functional, focused, and continually improving.

• Iterative. Agile is based on doing things repeatedly until you get them right. Short iterations mean that teams can change direction and react quickly. Progress remains visible and predictable because development happens in short sprints. Delivery risk declines progressively.
• Empirical. Agile teams rely less on the plans, estimates, and assumptions common to waterfall methods and more on A/B testing and other real-time metrics generated by end users. One of the many virtues of sprints is that they produce empirical feedback quickly, allowing teams to self-correct. Agile teams also measure and track their activities closely.
• Cross-functional. Agile teams have members of such relevant functions as business, marketing, development, and, in some industries, risk management working closely together in order to facilitate early and frequent feedback from business executives and customers. All the members of the team have specific roles and responsibilities.
• Focused. Agile teams are fully accountable. They do not work on several projects simultaneously; nor do they leave a project once their specific duty is done. In for the duration, they develop a sense of accountability.
• Continually Improving. Agile software is a work in progress, with constant updates and experimentation aimed at satisfying customers.

Putting the Agile set of beliefs into practice can be difficult in large companies, given layers of processes and structures, such as HR, finance, and legal functions. Rather than viewing Agile as yet another new process, companies should integrate Agile values into their own software-development organization and culture, making reasonable modifications when necessary.

There are five secrets of success for large-scale Agile transformations.

It Starts at the Top

Transformative change requires support from the top. Senior leaders need to be actively involved in fundamental decisions about the business purpose of going Agile and the cultural barriers and root causes that might stand in the way of success. Without this commitment, legacy approaches to, for example, capital allocation, HR processes, and portfolio management will doom Agile. That’s why business—not just tech—leaders must be accountable.

Agile transformations are different from other transformations: leaders must mobilize management to march in an unfamiliar new direction. The fast pace and cross-functionality of Agile can put many executives out of their comfort zone. Without strong and steady support from the top, many executives and team members revert to the norm. The CEO of a large European bank told us that he wants his organization to operate as a technology company that deals with financial services products.

To Fly, You Need Pilots

In a large organization, Agile pilots are necessary in order to determine whether Agile will work there and whether the organization will accept Agile principles. Pilots are critical to a company’s making the necessary adaptations to Agile.

For example, in a scrum, a single product owner takes responsibility for managing the relationship and interactions between developers and customers. This role requires a careful mix of technical and business skills. Companies may need to have even two or three people collectively serving in that role until the organization develops people who have the required multifunctional skills.

Likewise, it might be difficult to fully implement iterative development in all instances, but frequent feedback between developers and business executives ought to be the norm.

Staged rollouts in waves create momentum by building relevant capabilities and ensure that Agile principles and culture are embedded across the organization. (See Exhibit 1.)

Managing the Tipping Point

The pilot phase is followed by steps that must be executed with some delicacy to avoid unnecessary tension: it’s time to scale up Agile in an organization that may be theoretically willing to accept it but, practically, is challenged to do so.

HR processes, such as performance management, may not be set up to handle fully dedicated cross-functional teams where team—not individual—results matter most. Agile’s flexibility will almost certainly strain budgeting processes even if Agile is ultimately less costly than traditional development activities. An organization’s IT infrastructure may not be set up to accommodate continual integration and deployment because of lengthy provisioning times. Furthermore, traditional development teams may be resentful, and certain activities may be outsourced.

These are all real technology and organizational concerns that will not resolve themselves on their own. Executives must actively manage the integration, and the enterprise almost certainly will have to invest in training and development to encourage the right culture and behaviors.

Several successful approaches exist for scaling up Agile within organizations. At one extreme, the music-streaming service Spotify has fundamentally changed its organization structure. The company’s product-delivery organization is made up of squads, tribes, chapters, and guilds. The primary unit is the squad, a multidisciplinary team that works toward a shared purpose and is run by a product owner. Tribes are groups of squads that work on related areas. Chapters are groups of people with similar expertise across squads, and they form the line organization. Guilds are interest groups that anyone can join. (See Exhibit 2.) Other companies have simply overlaid cross-functional teams above existing hierarchy.

Measure, but Measure the Right Things

The ultimate goal of Agile is to improve the business. Therefore, the ultimate measurement should relate to business performance. If the goal of a bank’s Agile project is to reduce the dropout rate in credit card applications, then the dropout rate should be the most important metric. But in order to improve the business, companies also need to track software reliability, security, complexity, and size.

That’s where software measurement tools enter the picture. These tools allow companies to demonstrate empirically the productivity and quality improvement of Agile development and the overall performance of Agile teams.

Never Stop

Agile development is an exercise in continuous improvement. It is not a one-off exercise. Agile requires constant monitoring to ensure proper functioning. Companies need to take steps to bake the Agile principles into the organization. There are many ways to ensure that Agile endures. Many companies, for example, create teams consisting of the leaders of each Agile project, and they share best practices.

At its heart, Agile is about creating the right context in which your people—specifically your developers—can do their best work. It is often thought of as a method for writing software, but ultimately, it is a way to run and continually improve your business.