Beyond the anecdote: True drivers of digital-delivery performance

| Article

Organizations across all sectors are under increasing pressure to create value and deliver results. From navigating the global pandemic and hybrid work environment, to managing supply chain disruptions and inflation, the winners of tomorrow will be the organizations that harness the power of technology to adapt to changing environments. Our observations show that organizations that are agile, invest in talent, foster a strong culture of craftsmanship, and have deployed the latest engineering practices drive better digital outcomes—typically seeing up to 27 percent gains in efficiency, customer satisfaction, employee engagement, and operational performance as organizations work to value over just doing work.1 However, understanding exactly what drives performance is difficult, and the industry is full of anecdotal “silver bullets.”

McKinsey has been researching what drives agility and related performance outcomes for more than five years, collecting self-reported data for over 1,700 teams from around 75 organizations2 across the globe. Here our research looks at 48 practices grouped into 19 capability levers3 within strategy, structure, people practices and culture, process, and technology4The five trademarks of agile organizations, McKinsey, January 22, 2018. to help us understand what drives team-level performance beyond raw team talent across digital leaders.

Our analysis shows that 23 percent of performance can be explained based on observable practice, with the rest driven by dimensions that are harder to quantify and control for, such as the talent of the team, the types of projects, and more.5 However, focusing on these controllable variables can make the difference between an institution that struggles and one that outperforms competitors.

1
Improvements in certain organizational practices have a disproportionate impact on specific performance outcomes.

Interestingly, although strong practices help across all dimensions, certain practices have disproportionate impact on specific dimensions. For example, as seen in Exhibit 1, a strong and mature technology infrastructure and architecture has a disproportionate impact on speed but is less relevant for team effectiveness.6 This variation between the impact of practices allows us to identify the levers that are most influential for each performance outcome that can then be coordinated to focus on achieving specific goals depending on priorities, within the context of overall improvement.

How does your organization fare across these dimensions? Our research dives into how isolating specific factors can improve performance as defined by an organization’s priorities.

Speed. Reducing the time it takes to bring ideas and innovations to market and adapting to the changing environment are critical in today’s increasingly volatile market. Speed in software delivery gets products, features, and services to market more quickly and can be measured with a number of metrics, including the following:

  1. Release frequency. How frequently do you perform releases into production within your team?
  2. Lead time to deploy. On average, how long does it take for code to be deployed in production after code commit?
  3. Lead time to delivery. On average, how long does it take to realize a prioritized business idea of average complexity (including approval and funding)?
2
Of the 19 capability levers, four have a statistically significant relationship with team speed.

By running a linear regression from the 1,700 teams’ self-reported data, we were able to determine which capability levers have a statistically significant positive relationship (95 percent confidence) with each measured performance outcome. Throughout the article, these exhibits convey the capabilities with a statistically significant positive relationship for the relevant outcome as well as the difference in scores between the top quartile and bottom quartile of teams for that particular outcome. The example above illustrates the four capabilities that have a statistically significant positive relationship with speed (the average of release frequency, lead time to develop, and lead time to deploy scores) as well as how the fastest teams (top performers, which were in the top quartile for speed) performed on each of these dimensions compared with slower teams (bottom performers, which were in the bottom quartile for speed). Values for capabilities represent self-reported survey responses, with higher values representing better performance. Different capability levers can be pulled, depending on the desired outcome, to achieve improvements.

In this example (Exhibit 2), linear regression analysis highlights the importance of several factors that significantly improve organizational speed.7 In particular, technical support for software teams plays an outsize role, which is not surprising given that delivery teams often spend a third of their time managing environments, deploying code, communicating with others, and tending to related activities that can be successfully automated with the right technical foundations. The top drivers of speed are as follows:

An agile pilot at a financial-services client showed that having the right collaboration technology enabled teams to meaningfully improve service quality and client satisfaction. In this case, the team leveraged the right supporting systems and tools, including group chat applications, digital whiteboards, and screen sharing to speed up issue resolution from weeks to hours.

  1. IT infrastructure and operations: Highly productized and automated IT infrastructure allows teams to self-provision and manage environments automatically (within minutes) and reduces the labor of IT operational activities.
  2. Roles and responsibilities: Roles that are standardized, broad, and empowered avoid delays related to communication and coordination. Specifically, empowered product owners/managers and technical leads supported by an engineering staff with broad full-stack capabilities.
  3. Supporting systems and tools: Easy-to-use and widely adopted supporting systems and tools, simple technology changes to the delivery pipeline, and communication and planning tools can materially improve speed.
  4. Focus on the customer: Placing customers first and interacting with them regularly motivates the team to remove obstacles and focus on simplicity. At large, users want simple systems, and the more organizations interact with those customers, the easier it is to avoid feature complexity, which bloats and delays development efforts.

The bottom line: Companies looking to increase their speed should double down on IT infrastructure/automation, clearly define roles and responsibilities, ensure access to supporting systems and tools, and emphasize the customer throughout their organizations.

Productivity. Increasing the time that teams allocate for work and speeding up the pace of work is another key goal of agile—stop discussing and start doing. We measure productivity in the following ways:

  • Productive time. What is the proportion of productive time within the team? Productive time excludes meetings, rework, fixing incidents/defects, and so on.
  • Velocity. What has been the average velocity per full-time equivalent (FTE) percent increase since the start of agile adoption (that is, the percent increase in current velocity per FTE versus average velocity per FTE of the first three months)?
3
Higher maturity across five capability levers has a statistically significant positive relationship with team productivity.

In Exhibit 3, our regression analysis shows the importance of structural, process, and people factors to remove unnecessary friction and support team productivity.8 The top drivers of productivity are as follows:

At a logistics client, we saw that product owners who built closer relationships through open dialogue with key members of offshore-development teams were able to more effectively communicate the product vision and requirements and complete their features four times faster.

  1. Workforce size and location model: Right-sized, strategically located teams facilitate the flow of information and work while minimizing friction. While the recent past has taught us that remote teams can produce similar (and impressive) outcomes to those of co-located teams, we find that team setup needs to be carefully designed to approximate in-person communication, including same time zones, appropriate tools, and ample opportunities to be in the same room. This setup can minimize handover time, reduce the number of meetings, and cut down on rework, as teams can better understand tasks and avoid errors that stem from miscommunication or misinterpretation.
  2. Team processes: It is intuitive that harmonized team processes (including effective agile ceremonies, a strong product road map, and a single backlog), coupled with an ingrained culture of continuous improvement and sharing best practices, support team productivity. Establishing a clear operating model gives teams a clear view of what is coming next and optimizes the number of meetings. Building a clean backlog also provides clarity over priorities and reduces the need for rework.
  3. Reporting structure: Creating a reporting structure that supports team autonomy, organizes teams around user journeys, ensures teams have the required skills within their group, and fosters strong communities of practice minimizes cross-team dependencies and enables teams to work toward their objectives with less administrative effort.
  4. Talent management: Organizations that prioritize talent management by coaching, building capabilities, and rewarding individuals allow teams to be more productive.
  5. Informal networks and communication: When individuals are encouraged to expand their informal networks and access information, they feel connected to the rest of the business and can easily collaborate with others (no silos). This openness (sometimes referred to as “radical transparency” or “shared consciousness”) reduces the need for time-intensive meetings and increases the time that individuals have available to work on achieving their goals.

The bottom line: To improve productivity, companies will need to create an effective location model and overall structure, institutionalize team and performance processes that unleash talent, and leverage the sweeping power of informal networks.

Quality. Many of our clients focus on increasing the quality of their teams’ output. We measure quality by looking at the following:

  • Change failure rate. What share of releases have either resulted in degraded service or subsequently required remediation over the last quarter?
  • Mean time to recover (MTTR). On average, how long does it take to recover from severity one and two incidents and/or security vulnerabilities?
4
Higher maturity across four capability levers has a statistically significant positive relationship with team quality.

Regression analysis (Exhibit 4) shows that quality depends on getting many dimensions right.9 Specifically, quality is most affected by the following:

At a leading financial-services company, we found separate owners for development and quality, which resulted in an MTTR that was significantly behind the company’s peers before pushing for “T-shaped” engineers who would take charge of development, testing, and operations.

  1. Vision: Setting an organizational vision, which emphasizes quality of deliverables, and ensuring that vision guides day-to-day decisions so individuals feel committed to that vision; these form the organizational value for quality.
  2. Culture: Perhaps counterintuitively, building a culture of psychological safety that tolerates and shares failures generally results in higher levels of quality, as team members trust each other and learn from mistakes. Furthermore, practitioners are more willing to raise the alarm when necessary if they know there will not be negative repercussions.
  3. Architecture evolution: Modular architecture (ideally using microservices) and integration of new services with core systems (for example, designing APIs so services can be reused by other teams) contribute to cleaner releases and faster recovery from setbacks. Evolving the architecture for resiliency and having the right tools are paramount.
  4. Reporting structure: While reporting structure may seem disconnected from quality, autonomous teams that are part of strong communities of practice generally have higher-performing members who do not depend on others to catch mistakes for them and who can act quickly as a united team to solve problems. Reporting structure is also linked when different perspectives are collected—for example, catching a legal requirement while discussing the backlog versus during user-acceptance testing, which likely would delay a release.

The bottom line: For organizations focused on improving quality, it is imperative to set a clear organizational vision, create a culture of experimentation and growth, and modernize the IT architecture.

Effectiveness. Anecdotally, we see teams become more effective and team members more engaged as organizations adopt an agile way of working. Our definition of effectiveness includes the following:

  • Delivery predictability. On average (over the past quarter), what share of the work committed did the team actually deliver (within a sprint or a release cycle)?
  • Value realization. What share of the business value committed (over the past quarter) was delivered?
  • Team engagement. On a scale of one to ten, how likely are team members to recommend this company as a place to work?
5
Higher maturity across seven capability levers has a statistically significant positive relationship with team effectiveness.

The regression analysis10 in Exhibit 5 shows significant positive correlations between effectiveness and seven traits:

At a leading North American retailer that was looking to transform, we saw what happens when there is no “North Star.” The organization had no clarity on why it was transforming, so the execution of technology initiatives was disparate, misaligned, ineffective, and frustrating. Everyone knew they needed to improve online sales, but there was no clear commitment/vision on the path to get there. Teams weren’t delivering what the business needed.

  1. Leadership: Leaders who embrace the service leadership mindset set a clear vision for their teams and then get out of the way to allow those teams to get more done. Great leaders emerge amid uncertainty and provide required direction.
  2. Vision: Teams are more effective when they contribute, commit, and have ownership of a shared vision that guides day-to-day decisions.
  3. Informal networks and communication: When individuals expand informal networks and communicate, they improve dialogue between business and development teams and can deliver the value committed to and expected by the business. In addition, transparency with information and metrics creates a sense of trust, which increases employee satisfaction.
  4. Linkage mechanisms: Jointly defined processes to interact, coordinate, and resolve dependencies within and across teams create linkage mechanisms that facilitate collaboration, value realization, and higher employee network promoter scores. This collaboration also helps identify interdependencies at an earlier stage.
  5. Reporting structure: Reporting structures with autonomous, adequately skilled teams, who have the right capabilities within the team, are also more effective.
  6. Talent management: Organizations that encourage and reward individuals for building expertise through strong capability-building programs, structured leadership coaching, and feedback enable individuals and teams to be more effective. As a consequence, team members are more likely to recommend their organization to others.
  7. Resourcing: Dynamic resourcing supports effective teams not only by ensuring that individuals are aligned with teams based on organizational goals but also by allowing the organization to optimize team composition more easily. We typically see mature companies frequently and radically change their team structure and team compositions.

The bottom line: Improving effectiveness is about connecting the stakeholders in the value chain, including business, engineering, design, etc. Therefore, we see networks and linkage mechanisms as paramount with the right leadership on top to remove silos and empower autonomous teams.

With this deeper understanding of the capabilities that truly drive speed, productivity, quality, and effectiveness, organizations can better prioritize their efforts and succeed in the current ever-changing market environment. The first step to action is to understand where you are today, decide where you want to be, and define the tactical steps to get there.

Explore a career with us