Tuesday, March 16, 2010

Inline vs. Out-of-Line WAF Deployments

Submitted by Ryan Barnett 03/16/2010

There was an article that just came out today entitled "Top considerations for selecting Web Application Firewall technology" that I had to comment on. First of all, the title is misleading as a more accurate title for this would have been "Proxy vs. Non-Proxy based WAF deployment models" as the article highlights why they think that a proxy-based WAF deployment is superior to non-proxy ones. Is this really the case? It depends. Each WAF deployment is different base on the use-case. Are you going to use it for virtual patching, http audit logging, tracking sensitive data, application DoS or App Defect identification? All of these scenarios are different and they don't always require an inline, proxy-base deployment model.

It is also important to note that there are hybrid deployment modes available for WAFs which include deploying sensors out-of-line to gather data and then communicating with agent applications installed on specific, individual web servers. The advantage of this approach is that for many large networks, they may only want to use an inline approach for some web applications without incurring the latency hit to other applications.

Keep in mind that this article was written by Evolution PR who represents WAF vendor Barracuda Networks - who does not offer an out-of-line/non-proxy based WAF solution. This makes it a bit more clear as to why they are trying to pitch proxy-based WAF as the only real solution. Breach Security's WebDefend appliance can be deployed in both out-of-line and inline modes so I am not promoting one over the other due to commercial interests. My aim here is to provide counterpoints to the data presented in this article. Let's look at the issues highlighted in more depth.

1. Cloaking

Hackers gather information in order to launch an attack on a Web server by trying to simulate error conditions on a Web site. Often, the resultant error messages expose information about the Web server, application server, or the database being used. This information is then used to launch a full-scale attack on the Web infrastructure.

A proxy-based WAF intercepts the response from the back-end server and forwards it to the client only if it is not an error. If the response is an error, the WAF can suppress the response containing debugging information and send out a custom response. The WAF also removes headers such as server banners, which can be used to identify servers.

The WASC Web Application Firewall Evaluation Criteria (WAFEC) document lists several alternative protection techniques that can be employed. In this section, the article is mainly talking about detailed error leakage prevention which isn't really what is considered web application cloaking. Cloaking involves attempting to obscure or remove tell-tale signs of the web application technology in use. These include encrypting or signing Cookies, URLs and parameter data to prevent tampering. While this is certainly a sexy concept it runs into issues in practice mainly due to the dynamic nature of today's web applications. Accurately parsing outbound response bodies in order to accurately identify/update/sign/encrypt all possible parameter data is not easy. You can thank AJAX, Flash, etc... for that. If is for this reason, that using behavioral profiling of inbound application usage is key.

2. Input validation

A WAF should secure applications where the incoming traffic may be encrypted or encoded using a non-standard character encoding.

A proxy based WAF decrypts and normalises data before running various types of checks, in order to ensure that no attacks are smuggled inside of encrypted or encoded packets. It also offers multiple ways of securing inputs - such as encrypting or digitally signing cookies to prevent against cookie tampering attacks. It can also recognise which fields are read-only or hidden and ensure that these fields are not altered. For other fields, it should offer a host of protection mechanisms such as checking for various attacks on the input fields and locking down those inputs based on data type, such as numeric or alpha numeric.

Non-proxy based WAFs do not provide effective input validation. Although some can encrypt and normalise data, because they are not proxy-based they are not able to enforce rules on individual form parameters passed to the application. They also cannot encrypt or digitally sign the application cookie; relying instead on signature matching for security.

Where to start with this section... First of all, the deployment model in use (inline vs. out-of-line) has absolutely nothing to do with the WAF's input validation capabilities. WAFs can do application profiling/learning and automatically create a positive security profile for URLs+Parameter payloads whether they are proxy-based or not. It is important to note, however, that there is a difference between detection and blocking. This section seems to indicate that non-proxy based WAFs can not detect these types of attacks and enforce input validation and this is not true. Once a violation of the learned profile occurs, however, if you want the WAF to block, then of course an inline WAF can block the request locally.

3. Data theft protection

Proxy based WAFs intercept outbound data, so they can be configured to ensure that sensitive data - like credit card numbers - are either masked or altogether blocked to protect data leakage.

This is only possible because the proxy-based WAF sits in line with the application server and secures data on both incoming and outgoing paths - so this is not offered by non-proxy based WAFs.

Proxy based WAFs do have one advantage when it comes to outbound data handling and that is if the user wants to actually change data on the fly to mask or delete sensitive data and still serve the response to the client. Again, while this sound like a great concept, there are issues in the real world. One specific issue which I have seen is when a WAF sanitized data doing outbound and this caused problems with processing of subsequent requests as this data was used within hidden fields. Remember my point from item #1 above in this regard as accurate parsing of outbound data is oftentimes difficult so properly sanitizing data is challenging as well.

4. Protect against application layer DOS attacks

There are many ways of launching an application layer denial of service attack. Web applications maintain state information - such as the number of items in a shopping cart - with the help of sessions, which require some memory resources on the Web servers. By forcing a Web server to create thousands of session leads, memory resources are locked up and this results in performance degradation and can lead to a server crash.

There are other ways these attacks can be done. The WAF should be able to control the rate at which requests reach the Web server, and track the rate of session creation. This is only possible with a system that proxies on behalf of the Web or application server.

Again - not true. Out-of-Line WAFs are also able to do rate-limiting and identify potential DoS scenarios. Breach Security's WebDefend appliance has Excessive Access Rate Detection capabilities which allow the user to set appropriate Anti-Automation rate-limiting thresholds to prevent brute force, scraping and DoS attacks. In an earlier blog post I also outlined how a WAF can Identify DoS Conditions through Performance Monitoring which helps to identify stealthy attacks that aim to open http connections and then sit idle and tie up processes. Under all of these circumstances, the issue is not about detection but how are you going to react when these attacks are identified. WAFs can choose to issue TCP resets based on increasing granularity: IP addresses, SessionIDs, or specific application usernames. If your site is under a heavy DDoS attack, it is usually appropriate to take evasion actions and actually push out the IP blocking to a network security device at the edge of your network.

5. Centralised security enforcement

The ability to enforce all security policies from a single control point allows for simplified operations and infrastructure. To ensure safer and more efficient security administration, it is advisable that controlling and enforcing attack prevention, privacy (SSL cryptography) and AAA (Authentication, Authorisation, Accounting) policy is done through a single control point.

Because a non-proxy WAF does not terminate TCP connections, it does not have the ability to request credentials from incoming users, issue cookies upon successful credential exchange, redirect sessions to particular destinations, or restrict particular users to particular resources. Proxy-based solutions, on the other hand, have the capability to be an AAA authority - or to fully integrate with existing AAA infrastructure.

Centralization of authentication/authorization mechanism is great from a management perspective but it isn't always appropriate from a WAF perspective. Most web applications handle user authentications themselves and are managed by different business units. Forget about WAFs for a minute - it is a larger undertaking to centralize web application account administration than to try and start this because you are going to implement a WAF. Where this makes sense is if/when you are create more of a portal environment and you want to then broker requests to different internal business units.

6. Control the response

Because of the wide range of security violations, it is important that the administrator is able to respond to threats differently. For example, in many cases it would be best to respond to a violation with a custom message or connection reset, while in others the administrator may want to follow up with the main action directly, with a longer block time.

Only proxy-based solutions are able to offer this sort of flexibility, as non-proxy based WAFs rely solely on sending TCP resets back to the attacker and temporary network ACLs as their protective mechanisms. Attacking packets will make it through to the server, and blocking actions are time-limited.

Don't forget about the hybrid deployment option I mentioned at the beginning which includes adding agents to specific web applications. This section does have a point, however, in that if you want to get more granular with handling custom error messages and redirecting the user under specific circumstances then having an inline WAF provides more options. As far as disruptive actions, out-of-line WAFs are not relegated to only using TCP resets. One interesting reactive action that Breach Security's WebDefend appliance has is called "Application Logout" in which the WAF initiates an http request to the application simulates the client actually logging out. This is similar in theory to doing TCP resets at lower OSI levels where you have to spoof the proper sequence numbers in order to terminate the connections. For the http layer, WebDefend will dynamically insert the proper application SessionID cookie value when submitting the app logout so it appears from the application's perspective that the logout was initiated by the user. Pretty slick. It is quite handy when used under certain policy violations such as suspected Session Hijacking events.

7. SSL architectural considerations

Application attacks use SSL cryptography and common encoding techniques to bypass traditional security measures, and hide their attacks. Proxy and non-proxy WAFs are quite different in the way they handle SSL cryptography and key management.

Non-proxy WAF vendors claim that they also have the technology to 'see' into an SSL encrypted packet as it passes by the non-proxy device. However, because decrypting and analysing the data takes time, by the time the non-proxy WAF is ready to make a decision, the attack will have already reached the back-end servers and completed the transaction.

Proxy based WAFs, on the other hand, are designed to serve as an SSL termination endpoint. Proxies tightly couple TCP, SSL and HTTP termination, giving them complete visibility into application content and allowing them to perform deep inspection on the entire session payload, including headers, URLs, parameters and form fields.

This section brings up and interesting trade-off that all WAF users must deal with - performance/latency of inspection vs. effective blocking. Out-of-line deployments are ideal for the former while inline deployments are the best for the latter. So, which items is more important to you? The second paragraph makes it seems as though out-of-line WAFs can't do the same SSL decryption/inspection and that is false as they can provide the same level of visibility. The issue is with that of latency and if, after inspection, disruptive actions are employed.

8. Accelerate and scale application delivery

It is important that a WAF product does not negatively affect end-user response time. Proxy based firewalls fully terminate the TCP, SSL and HTTP, reducing end user response time. They should be able to cache static content from the application, offloading servers and saving download time; pool TCP connections to the back-end servers and offload SSL processing, thereby reducing server load and end-user response time. Non-proxy based WAF products do not offer these features.

The first sentence is the key from a WAF perspective as all users want to add in the security inspection without negatively affecting end users. If you deploy an out-of-line WAF, then there will be no added performance or latency hit. If, on the other hand, you deploy an inline WAF then there is going to be a negative impact due to the SSL decryption, traffic inspection and probable SSL re-encryption on the back-end. It is for this reason that many inline WAFs have had to add on the application acceleration aspects to attempt to off-set this performance hit. So, you end up having a WAF vendor that is then trying to bolt on ADC types of functions and compete with other vendors who specialize in this space (such as an F5). On the flip side, you have ADC vendors (again like an F5) who specialize in application delivery who try and also bolt on add-on modules to provide web application firewall functionality. The main problem I see on both fronts is that they are going outside of their core competency. When deploying a WAF, it is best to do an architecture review to identify the ideal location for both inspection and blocking of traffic. This may include placing WAFs either before or after existing HTTP Load Balancers. There are benefits of both approaches. From a blocking perspective, an out-of-line WAF has a better chance of terminating a TCP connection if it is deployed directly in front of another layer 7 inspection device. On the performance front - if you can terminate SSL decryption on the load balancers, then placing the WAF behind them will make it more performant.

No comments: