Sep 06, 2019

The 2019 Accelerate State of DevOps report quantifies operational differences between elite and low performance engineering teams

The 2019 Accelerate State of DevOps report from DevOps Research and Assessment (DORA) team seeks to quantify the performance of elite engineering teams. The report builds on six years of research with data from 31,000 professionals from around the world in a variety of software development roles.

The report analyzes how engineering teams can be more productive in developing and deploying software. Productivity, according to the report, is a combination of engineering throughput and software stability. When compared to low performance teams, elite engineering teams have:

  • 208 times more frequent code deployments. Elite teams make multiple deployments per day, while low performers deploy roughly once per month.
  • 106 times faster lead time. Lead time is a proxy for throughput, measuring the time from code changes to deployment. Elite teams experience lead times that are usually less than 24 hours, while low performance teams require one to six months to move code from commit to deployment.
  • 2,604 times faster time to recover from incidents. During outages and service disruptions, elite teams recover in less than one hour, but low performance teams can take anywhere from one week to one month.
  • 7 times lower change failure rate when changes to production result in degraded application performance for users. Elite teams experience change failure rates ranging from 0% to 15%, while low performance teams can experience rates between 46% and 60%.

To build high performance engineering teams, the report suggests that companies should carefully consider how they define their culture and organize their knowledge.

According to DORA’s analysis, “high-performing teams need a culture of trust and psychological safety, meaningful work, and clarity.” The report highlights the importance of fostering a culture that allows team members to take risks with the backing of a supportive team. Optimizing for information flow, trust, innovation, and risk-sharing is predictive of software delivery and operational performance.

Knowledge-sharing, too, is an important component of building a high performance company culture. Teams that actively use and maintain internal knowledge tools are 1.73 times more likely to be productive. Effectively scaling teams requires careful knowledge management, as well. Low performance teams preferred training centers (otherwise known as DOJOs), where employees are removed from their usual routines to learn new tools or technologies, and centers of excellence, where expertise is centralized and consults others. Both methods create silos and isolate expertise, negatively impacting team productivity. Elite teams employ communities of practice, where groups that share common interests in methodologies are encouraged to share knowledge within and across teams, and grassroots scaling, where small teams work together to transform a development process and then informally share their success with the organization.

Toolchains, like knowledge sharing, can be optimized as well. The highest performing engineers are 1.5 times more likely to use easy-to-use tools (e.g. not proprietary software or difficult to maintain software). Moreover, according to the report “elite performers automate and integrate more frequently into their toolchains on almost all dimensions.”

To synthesize the reports main points: the most successful, high performance teams actively work to accelerate their development processes by maintaining an open and team-based culture that effectively balances development throughput, automation, speed.

Want to get more of these in your inbox?

Subscribe for weekly updates from the Software team.