Adding Security into the DevOps Culture

The importance of culture cannot go underemphasized in DevOps. In a movement that is transforming the lives of how people build and deliver software, sometimes we forget the value of the people. It is easy to get focused on containers or the latest architecture patterns, but as a core tenet, DevOps is about culture first.

First a little background…

Last week, we released a model on DevOps and the transformation that happens in these four key areas:

• Treating our systems and infrastructure as code, including how we version it to how we build it.
• Changing our engineering culture to orient around the total delivery and usage experience.
• Creating feedback loops in your runtime environment that informs all parts of your engineering team.
• Favoring faster delivery cadence and a reduction in change volume.

Lets explore what this means when we add security to the mix—we will be using the word rugged as a way to introduce security into DevOps. This is not a way to create a new type of DevOps, but describe the functional ways we change our approach as we grow. It’s still DevOps, we are just applying a rugged filter to help callout specifics to new practices.

Now back to our regularly scheduled program.

Why is culture so important?

Just this week, the 2016 State of DevOps report was released and there was a significant finding on culture:

 

This means that employees that worked in high performing teams were twice as likely to recruit friends to come work with them. It means that they are not only happy but also proud to associate their personal brand with the company. This is a direct reflection of the culture and that the team is accomplishing big things. It’s one thing to work somewhere, its a completely different thing to ask your friends to join you.

What culture means to security

Culture is the foundation of DevOps and was a solution to the cultural divide between the dev and ops. We see the same cultural problems with security, often with staffing rations of 100 developers to 1 security engineer and very differing priorities: speed and features vs. security and compliance. This is changing, and there are two cultural shifts that happen when applying rugged to DevOps:

Shift 1: Know thy self

The first shift is one of self realization—often the first step happens when we realize that if security is a blocker, then it will be routed around. Building or fostering a culture of gating functions surrounding security is not a sustainable or forward thinking model.

Shift 2: Recognize the organizational landscape

The second shift is knowing that we can’t hire out of this problem and we need to find a way to deputize security champions in each group. Depending on your organizational culture the process will vary. For ideas on how to accomplish this cultural transition, see our article on Silo Smashing Ideas.

Pragmatic Technical Changes

Its often said you don’t fix cultural problems with technology. This is usually true, but the are some things you can do technically to influence your culture. In the 2016 State of DevOps Report there was one technical pattern called out that actually lead to higher performance and a better culture: trunk based development.

We found that having branches or forks with very short lifetimes (less than a day) before being merged into trunk, and less than three active branches in total, are important aspects of continuous delivery, and all contribute to higher performance. So does merging code into trunk or master on a daily basis. Teams that don’t have code freeze periods (when people can’t merge code or pull requests) also achieve higher performance.

They go on to note that this specific technical practice has a significant impact on performance and culture. For security engineers and adopters of Rugged DevOps, there are two notable technical practices that impact culture.

Practicing Lean Security

In W. Edwards Deming’s 14 points on management he noted:

Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.

The entire security industry is built on doing inspection at the end, on a mass basis. We do penetration testing, annual assessments, end of SDLC code reviews… The list goes on. We are inspecting and validating too late when we need to move security testing closer to when the code is being written. Shifting security engineering efforts earlier in development is not just a technical change, it has plenty of cultural benefits.

Doing ChatOps for Security

The practice that many teams have found as beneficial is integrating their security tooling into their team communication channels. At Signal Sciences, we do this with almost every piece of tooling we use; from deploys to security to alerting, everything goes into our team chat. When we built out our NextGen Web App Firewall we knew we didn’t want the main communication mechanisms to be email or log entries (you can still get those if you want them), but instead we chose to distribute security events to the entire team in the medium they are most comfortable with.

Many of our customers integrate with Slack and HipChat or alerting products like VictorOps and PagerDuty. This takes the silo’ed knowledge of where attacks are happening and federates it across the organization. As you might have guessed, this has changed the way security is perceived at many of these organizations—its a cultural win.

Summary

We have seen that the modern approach to DevOps and culture brings new challenges and new opportunity. As we overlay a rugged approach, there are several ways security can change the culture and add value.

---

Thanks for reading this article. If you enjoyed it please let us know by clicking that little heart below.

At Signal Sciences we are building the industry’s first Next Generation Web Application Firewall (NGWAF). Our NGWAF was built in response to our own frustrations of trying to use legacy WAFs while enabling business initiatives like DevOps, cloud adoption and continuous delivery. The Signal Sciences NGWAF works seamlessly across cloud, physical, and containerized infrastructure, providing security without breaking production traffic.