Here’s some security jeopardy for you:

Answer:

“It’s going to break the app…”

“It’s going to kill performance…”

“It’s going to be too hard to configure…”

Question:

What is the immediate response from DevOps and Product teams any time they hear the words ‘Web Application Firewall’?

As a member of Signal Sciences’ sales team, I hear these objections, and others daily, when talking to potential customers about our Next Generation Web Application Firewall. In fact, the objections I hear are so consistent that it makes you think “What could WAF vendors have possibly done wrong over the years to provoke the same set of objections from such a broad set of people?”

For progressive security leaders who want to shift security from being a business blocker to a business enabler, picking controls that are friendly to the DevOps and product teams is critical. Here are the most common objections I hear from DevOps and product teams about legacy WAFs, why they are valid, and some questions to consider asking when evaluating potential solutions.

“It’s going to break legitimate traffic…”

This is usually the first objection I hear, and it makes total sense. When a security control breaks legitimate production traffic, everybody loses. Your users lose because they can’t use your product. The support team loses because they are often the first responders. The DevOps team loses because they have to triage and locate the source of the issue. The security team loses because they have to respond to the ops team (AND because they have “broken the app”). The business loses because it gets less revenue and/or burns its reputation. And so on.

1. Generate rules to describe what malicious requests look like.
2. Deploy the ruleset in front of all incoming traffic.
3. Compare each incoming request to the ruleset and make a block or allow decision for each request.

This approach may have worked when you had a small set of users who behaved in a predictable way, and when your application didn’t change more than once or twice per year. However, in today’s environment, user behavior is highly dynamic, which makes it extremely difficult to write a ruleset that accurately describes the difference between malicious and not malicious. Applications change so frequently that unless the rules are continuously updated, they quickly become out of date. A legacy WAF now requires hundreds or thousands of rules governing thousands of constantly changing variables including URLs, parameters, cookies, XML elements and form fields. When block/allow decisions are made based on those rules for each and every incoming request, it’s no wonder there’s a perpetually high false positive rate. Furthermore, some legacy WAFs require you to deploy them in such a way that they become single points of failure. In other words, if the WAF were to fail, all user traffic would be broken.

Questions to ask when evaluating potential solutions:

• What percentage of your production deployments are actually in full blocking mode?
• If your product breaks, how do you ensure that all of my traffic won’t break?
• How does your product mitigate the risk and cost of false positive decisions?
• When your product makes a blocking decision, how much transparency do I have into why it was made? (In other words, what do I do when DevOps or Product engineers come to me saying “Your WAF broke good traffic!”)

“It’s going to kill performance…”

This objection is a close second in terms of frequency and for good reason. When the app is slow, it damages user experience and therefore impacts churn or adoption.

When a legacy WAF is deployed ‘in-line’, inspecting each request and comparing it to the ruleset before allowing it through, that inspection has a cost: latency. When you ask legacy WAF vendors how they mitigate latency, the answer is typically either to buy a bigger appliance, or to shift the WAF deployment to a CDN. While these “solutions” may address the latency problem, they tend to introduce a new set of objections. For example, bigger appliances cost more and are difficult to scale, and CDN based WAFs can require major architecture changes which may be difficult politically.

Questions to ask when evaluating potential solutions:

• What infrastructure and architecture decisions are impacted by deploying your product?
• How does your product provide an SLA on latency?
• If DevOps and Product teams blame your product for impacting performance, how would I confirm or refute that?

“It’s going to be too hard to configure…”

Your DevOps and Product teams want to see value quickly. They have their own projects to work on, and don’t want to spend weeks or months configuring a new security solution. As a security leader, when you propose a new control that’s complicated to configure and takes a long time to show value, you hurt yourself in two ways:

• You decrease your chances of achieving full coverage with that control. In other words, you might get it deployed on one app, but once people see how much of a pain it is to deploy, they’re likely to resist deploying it elsewhere.
• You spend valuable political capital to get it deployed at all. Often in practice, this means you end up with partial coverage and no buy in for further deployments.

With any legacy WAF, you’ll need to select or write a core ruleset, flush it out, then test and refine it several times before you even think about deploying it in production. Once it’s there, you may find it behaves differently when exposed to real traffic, which requires more tuning and input from security and DevOps. All told, it’s not unusual for it to take months to get to the point where you’ve got the WAF deployed in blocking mode in production (if you get there at all). If you’re considering an appliance-based, legacy WAF, you’ll need to consider the network and architecture changes required to implement and how long it will take your DevOps team to make them. If you’re considering a CDN based WAF, you may also need to factor in time for giving your private SSL keys to the CDN provider.

Questions to ask when evaluating potential solutions:

• What infrastructure and network changes will be required from my ops team prior to deploying your product?
• What is the average time it takes similar companies to install your product on a single app?
• On average, how much time do similar companies invest in configuration before they are ready to run the product in blocking mode, in front of live production traffic?

“It’s not going to scale…”

With the trends toward the public cloud and infrastructure as code, your DevOps and product teams want everything to be elastic, including security controls. In other words, if the CTO says “Part of our strategy is to improve flexibility of our platform, and we’re going to achieve that using AWS and Chef”, you don’t want to be stuck saying “But what about our WAF?”

The way companies plan, purchase and deploy infrastructure has changed dramatically over the past decade and legacy WAFs that are appliance based (physical or virtual) are relics of a time before public clouds. As such, when it comes to the way you plan for, purchase and deploy them compared to the rest of your infrastructure, it can feel like a square peg in a round hole. The consequences of this can be waste (e.g. if you’re forced to buy more than you need) and bottlenecks (e.g. if it requires a separate, manual deployment process), both of which DevOps and product teams hate.

Questions to ask when evaluating potential solutions:

• Can I automate deployment of your product using configuration management tools?
• How elastic is your pricing model?
• What are the differences between deploying your product in a ‘physical infrastructure’ environment and deploying it in a public cloud?

“It’s going to be too hard to maintain…”

I typically hear this objection from people who have worked with legacy WAFs in the past. As I mentioned earlier in this post, legacy WAF rulesets become incredibly complicated and difficult to maintain. This imposes cost on your team in two ways: keeping the ruleset up to date, and responding to issues caused by the fact that the ruleset is out of date (see: false positives). Total cost of ownership for a legacy WAF can be incredibly high over time. But don’t take my word for it. This is an actual quote from a legacy WAF vendor’s whitepaper titled ‘Pragmatic WAF Management’:

“Every aspect of managing WAFs is an ongoing process. This is the antithesis of set it and forget it technology. That is the real point of this research. To maximize value from your WAF you need to go in with everyone’s eyes open to the effort required to get and keep the WAF running productively.”

Based on what I hear from our customers, when you present this objection to a legacy WAF vendor you’re likely to get one of two responses: “Don’t worry, Machine Learning!”, which you can believe at your own risk, or “Don’t worry, Managed Services!” which doesn’t eliminate the cost, just shifts it.

Questions to ask when evaluating potential solutions:

• On average, what is the total cost of ownership for your product at companies similar to mine over n years?
• How do you mitigate ongoing maintenance and tuning costs?

Ultimately, your security controls are more effective when they have the support of DevOps and Product teams. If you’re considering a legacy Web Application Firewall, you can improve your odds of success by making sure that whatever solution you choose has good answers for these objections.

---

Signal Sciences’ industry first Next Generation Web Application Firewall was built in response to our frustrations of trying to use legacy WAFs while enabling business initiatives like DevOps and cloud adoption. The Signal Sciences NGWAF works seamlessly across cloud/physical/containerized infrastructure, providing actionable security prioritization based on where your applications are targeted, and blocking attacks without breaking production traffic.