Release throughput is a score based on the average of your team’s global percentile rankings across lead time, delivery frequency, and delivery size.
Release throughput helps you understand your team’s overall DevOps performance by combining both velocity and volume metrics—i.e. how frequently your team delivers new features compared to the size of each unit of work.
It is the north star metric for delivery and DevOps performance. By combining multiple data points, release throughput more accurately measures your team’s flow of work to capture your organization’s DevOps efficiency.
Measuring release throughput
Similar to DORA metrics, release throughput can be measured using source control and GitHub data—the same data source for lead time, commit frequency, and commit size.
On the Software platform, we analyze metadata from your organization’s source through our GitHub integration. When you connect GitHub to an organization, you will be able to see historical data, as well as your organization’s current release throughput score.
Combining key delivery metrics
Release throughput combines several key DevOps performance indicators. Specifically, the release throughput score is based on three factors—lead time, delivery frequency, and delivery size.
By combining these metrics into a single score, we provide a holistic view of your DevOps performance and create a single, north-star metric using multiple data points.
- Lead time is the time between first code—measured by the first file changed in a commit—and merging a commit into the main branch. It is a measure of how long it typically takes teams to deploy new features.
- Delivery frequency measures the median number of commits integrated into the main branch per developer per day. It is used as a proxy for how often your team ships new code.
- Delivery size is a measure of the size of each commit—the sum of additions and deletions—integrated to the main branch. It is an indicator of how well your team can break down code changes into small units of work.
Each metric reflects your organization’s percentile ranking compared to other companies in the market. We consider 90th percentile and above as elite, 75th to 90th as high, 50th to 75th as medium, and 50th and below as low.
Ways to improve release throughput
Combining multiple inputs to measure DevOps performance helps your team balance speed, volume, and quality. Improving your DevOps performance requires understanding how they interact and making changes across the delivery pipeline.
Importantly, optimizing for speed over quality, or vice versa, will negatively affect long-term trends in your team’s release throughput. Moving too quickly without sufficient testing or peer reviews can introduce bugs and rework into the cycle, reducing overall velocity. Similarly, moving too slowly can introduce stale code in branches, resulting in long QA cycles and rework, and increase abandoned work, reducing the amount of code shipped.
Using release throughput as a north-star metric also prevents teams from over-optimizing for specific metrics. Teams can simply commit more frequently, avoid testing, or try other workarounds to artificially improve metrics like lead time and delivery frequency.
To improve release throughput, teams first need to improve visibility into their DevOps performance by getting their data and metrics. With better visibility, teams can then analyze where they rank compared to other companies in the market, with the ultimate goal of finding inefficiencies in the delivery pipeline.
Understanding low vs. high release throughput
High release throughput means your organization experiences a fast flow of work. Developers are able to make and deploy changes to production easily. They break down larger projects into small units of work that can be quickly, easily, and frequently merged into production environments.
Low release throughput indicates your organization may be experiencing uneven or painful flow of work. Developers may be experiencing long wait times, frequent handoffs, or slow merge times. The lower your release throughput, the more difficult the delivery process is for your team.
As previously mentioned, low release throughput can be a result of long lead times, low delivery frequency, and large delivery sizes.
Long lead times means it takes longer for code to move from ideation to delivery. Developers encounter delays and friction when writing, merging, and deploying code.
- Reduce wait time and handoffs. It’s important to estimate the typical number of handoffs within the software delivery pipeline. Each handoff to operations, security, and testing teams can slow down the flow of work.
- Limit work in progress. Too much work in progress leads to burnout and disruptive context switching. It also delays high priority tasks by decreasing team focus. Visibility into work in progress, using dashboards and storyboards, can help teams set limits based on team capacity.
- Improve test automation. Manual testing late in the development cycle can increase review time and decrease delivery frequency. In addition, code issues discovered later in the delivery pipeline are more expensive and time-consuming to fix. Teams should provide fast feedback to developers as early as possible, introducing test suites that follow the test pyramid methodology.
- Create self-service tooling. Teams should invest time into creating self-service environments and tooling that allow developers to complete their work with fewer repetitive requests to other teams.
- Shift-left on security. For companies working in heavily-regulated industries, security reviews often create bottlenecks at the end of the development cycle. Consider pre-approving development tools and introducing security checks even earlier into the development life cycle, particularly with easy-to-integrate tools like secrets detection and static analysis.
Low delivery frequency indicates issues integrating code changes. It can reveal challenges implementing continuous integration.
- Invest in loosely-coupled architecture. Tightly-coupled architecture means components in a codebase are strongly intertwined; changing one part of it often requires introducing changes to many other parts of the codebase. By creating a loosely-coupled architecture, developers can avoid costly merge conflicts and reduce the complexity of their changes.
- Balance manual and automated reviews. Manual code reviews consume more time and resources than automatic code reviews. Performing basic checks before requesting code reviews can provide faster feedback to developers.
- Make change approval lightweight. Lengthy change approval flows and external change approval boards can decrease delivery frequency. High-performing teams create lightweight change approval workflows that automate low-risk changes and standardize any manual reviews.
- Enable continuous deployment. On high-performing teams, developers work to always keep the codebase in a deployable state. They add automated testing, perform continuous code reviews, and work in small batches to avoid code freezes and deployment delays.
Weak delivery size means your team works in large units of work. These are often unwieldy and slow down the delivery pipeline. Decreasing delivery size can improve lead time and delivery frequency by limiting the scope and complexity of code changes.
- Adopt trunk-based development. Long-lived feature branches can often lead to large units of work that are more difficult to merge into the main branch. Conversely, trunk-based development is the practice of merging small and frequent code changes into a single branch, known as the trunk branch. High-performing teams typically use three or fewer active branches per repository.
- Improve continuous integration. Guardrails, such as feature flags, easy rollbacks, and better monitoring, encourage more frequent merging of smaller units of work.
Frequent, small code changes can also indicate constraints in the delivery pipeline. Small delivery size may suggest teams are making frequent hotfixes or bug fixes. It can also suggest teams may be bogged down in planning, meetings, or strict change approvals that make it difficult for developers to ship more meaningful changes to production.
Relation to DORA metrics
Engineering teams often rely on metrics created by Google’s DevOps Research and Assessment (DORA) team to measure DevOps performance. The DORA metrics are a series of four DevOps metrics that help teams understand both their development velocity and product stability.
The four key DORA metrics are:
- Deployment frequency: How often your team releases to production
- Lead time for changes: How long it takes changes to get into production
- Change failure rate: The percentage of deployments causing a failure in production
- Time to restore service: How long it takes your team to recover from a failure in production
Although helpful benchmarks for engineering organizations, the DORA metrics don’t capture the size of changes, otherwise known as delivery size and often measured as the average commit size. The DORA metrics will not provide much insight into how delivery size—the size of each unit of work—impacts team velocity.
Small delivery size can be a sign of hotfixes, bugs, or low code quality. It can also be a sign teams are over-optimizing for specific metrics. To improve their DORA metrics, teams can write and deploy just a few lines of code every day, increasing deployment frequency and decreasing lead time.
Additionally, when using the DORA metrics, deployment frequency is measured as an absolute value. However, it can be helpful to understand deployment frequency per developer for insight into how your team is balancing both team size and velocity.
Continuous visibility with Software
Over time, teams can measure how their release throughput changes with new workflows, tooling, and team members. They can maintain visibility into their DevOps performance with automated reporting and customizable dashboards.
Why real-time data matters
One of the core tenets of high-performing DevOps teams is fast feedback; release throughput is no different. When measuring release throughput, it’s important to use accurate and near real-time data.
Surveys and research provide helpful guidance, but quickly become stale and outdated. Industry reports are refreshed annually—too slow to keep you competitive in the fast-paced world of software development. Real data and real-time metrics better enable continuous improvement.
Surveys are also prone to respondent biases. Surveying engineering teams depends on their ability to understand various metric definitions, accurately recall data over a long period of time, and respond honestly without external factors influencing their viewpoints.