This year, the Google / DORA State of DevOps Report dived deeper into information security. In 2021, the report highlight the importance of Secure Software Supply Chains – building in security throughout the software development cycle and supply chain. The research leveraged the Supply Chain Levels for Secure Artifacts (SLSA) framework in conjunction with NIST’s Secure Software Development Framework (SSDF) to explore technical
practices that support the development of software supply chain security.
You can see from the chart above that this year’s results show a clustering in the medium performer group. The authors consider that this may be a result of not having data from the elite performers this year, or an effect of the pandemic restricting the ability of teams to innovate practices. It’s also worth noting that the floor has risen: this years low performers perform better than last year, so whilst the ceiling is lower, the floor is higher.
This year’s report showed that high-trust, low-blame cultures focused on performance were 1.6x more likely to have above average adoption of emerging security practices than low-trust, high-blame cultures focused on power or rules. This reflects the difference between Safety I and Safety II approaches, originally coined by Erik Hollnagel. Cultures that focus largely on avoidance of risk, and implementing rules or procedures to prevent risk actually perform worse over time than cultures focussed on looking at “what went well”, not just “what went wrong”.
The report showed that generative organisational cultures, as described by Westrum’s Cultural Typologies, tend towards being higher performing. Alongside that, other drivers of high organisational performance were:
- Team stability and longevity
- Transparency and confidence in funding
- Opportunities to work more flexibly
From a technological perspective, the capabilities that contribute most to high performance are version control, continuous integration, continuous delivery, and loosely coupled architecture.
Interestingly, less experienced teams who implemented trunk-based development actually have less positive results around
trunk-based development and see:
- Decreased overall software delivery performance
- Increased amounts of unplanned work
- Increased error-proneness
- Increased change failure rate
However, more experienced teams show significant benefits and the presence of trunk-based development shows a positive impact on overall organisational performance. This is likely both an effect of the J-Curve effect and the additional practices required to successfully implement Trunk-based development. The lesson for teams: keep going!
An interesting key finding is that the State of DevOps 2022 evidence suggests that healthy, high-performing teams also tend to have good security practices broadly established. This is reinforced by evidence from other industries that guardrails of any kind help to reinforce psychological safety on teams. Last year’s (State of DevOps 2021) report showed that Secure Software Supply Chains that integrate security practices into pipelines and processes, enable teams to deliver secure software quickly, safely and reliably.
An important aspect to consider is that technical capabilities have a positive impact on SLSA-related practices and through this positive impact on SLSA related practices have a positive impact on both software delivery performance and organisational performance. That is to say that SLSA practices and continuous integration, version control and continuous delivery capabilities are a virtuous cycle: the capabilities built through CI, CD and version control are the same capabilities that enable teams to adopt and improve SLSA practices.
Finally, a point on SRE and the “error budget” framework: the report shows that software delivery performance alone does not predict organisational success. Excellent software delivery combined with high reliability (high DORA Metrics in this case) correlate with organisational success. Which makes sense: when a service is unreliable, users won’t benefit from pushing code faster into that
fragile context.