iRules

The Potential Ramifications of Platform-Based Vulnerabilities on Cloud Computing

Lori MacVittie — Wed, 08 Feb 2012 13:26:00 GMT

#infosec #adcfw #cloud Alternate title: How to take out an entire PaaS cloud with one vulnerability

What do these two vulnerabilities have in common? Right, they’re platform-based vulnerabilities. Meaning they are vulnerabilities peculiar to the web or application server platform upon which applications are deployed. Mitigations for such vulnerabilities generally point to changes in configuration of the platform – limit post size, header value sizes, turn off some value in the associated configuration.

But they also have something else in common – risk. And not just risk in general, but risk to cloud providers whose primary value is in offering not just a virtual server but an entire, pre-integrated and pre-configured application deployment stack. Think LAMP, as an example, and providers like Microsoft (Azure) and VMware (CloudFoundry), more commonly adopting the moniker of PaaS. It’s an operational dream to have a virtual server pre-configured and ready to go with the exact application deployment stack needed and offers a great deal of value in terms of efficiency and overall operational investment, but it is – or should be – a security professional’s nightmare. It’s not unlike the recent recall of Chevy Volts – a defect in the platform needs to be mitigated. The only way to do it, for car owners, is to effectively shut down their ability to drive while a patch is applied. It’s disruptive, it’s expensive (you still have to get to work, after all), and it’s frustrating for the consumer. For the provider, it’s bad PR and negatively impacts the brand. Neither of which is appealing.

A vulnerability in the application stack, in the web or application server, can be operationally devastating to the provider – and potentially disruptive to the consumer whether the vulnerability is exploited or not.

STANDARDIZATION is a DOUBLE-EDGED SWORD

Assume a homogeneous cloud environment offering an application stack based on Microsoft ASP. Assume now an exploit, oh say like Post of Doom, is discovered whose primary mitigation lies in modifying the configuration of each and every instance. Virtualization of any kind provides a solution, of course, but introduces the possibility of disruption in the impact to consumer applications from the configuration change. A primary mitigation for the Post of Doom is to limit the size of data in a POST to under 8MB. Depending on the application, this has to potential to “break” application functionality, particularly those for which uploading big data is a focus. Images, video, documents, etc… These all may be impacted negatively, disrupting applications and angering consumers.

Patching, of course, is preferred, as it eliminates the underlying vulnerability without potentially breaking applications. But patching takes time – time to develop, time to test, time to deploy. The actual delivery of such patches in a PaaS environment is a delicate operation. You can’t just shut the whole cloud down and restart it after the patches are applied to the base images, can you? Do you wait, quiesce the vulnerable images and only force the patched ones when new instances are provisioned? A configuration-based mitigation, too, has these same issues. You can’t just shut down the whole cloud, apply the change, and reboot.

It’s a delicate balance of security versus availability that must struck for the provider, and certainly their position in such cases is one not to be envied. Damned if they do, damned if they don’t.

Then there is the risk of exploitation before any mitigation is applied. If I want to wreak havoc on a PaaS, I may be able to accomplish simply by finding one with the appropriate platform vulnerable to a given exploit, and attack. Cycling through applications deployed in that environment (easily identified at the network layer by the IP ranges assigned to the provider) should result in a wealth of chaos being wrought. The right vulnerability could take out a significant enough portion of the environment to garner attention from the outages caused.

Enterprise organizations that think they are immune from such issues should think again, as even a cloud provider is often not as standardized on a single application platform as an enterprise is, and it is that standardization that is at the root of the potential risk from platform-based vulnerabilities. Standardization, commoditization, these are good things in terms of many financial and operational benefits, but they can also cause operational risk to increase.

MITIGATE in the MIDDLE

There is a better solution, a better strategy, a better operational means of mitigating platform-based risks.

This is where the role of a flexible, broad-spectrum layer of security applies. One that enables security professionals to broadly apply security policies to quickly mitigate potentially disastrous vulnerabilities. Without disrupting a single running instance, an organization can deploy a mitigating solution that detects and prevents the effects of such vulnerabilities. Applying security policies that mitigate such vulnerabilities before they reach the platform is critical to preventing a disaster of epic (and newsworthy) proportions.

Whether stop gap or a permanent solution, by leveraging the application delivery tier of any data center – enterprise or cloud provider – such vulnerabilities can be addressed without imposing harsh penalties on applications and application owners, such as requiring complete shutdown and reboots.

Leveraging such a flexible data center tier insulates the platform from exploitation while insulating customers from the disruption required to mitigate immediately on the platform layer, allowing time to redress through patches or, at least, understand the potential implication to the application from the platform configuration changes required to mitigate the vulnerability.

In today’s data center, time is perhaps the biggest benefit afforded to IT by any solution, and yet the one least likely to be provided. A flexible application delivery tier capable of mitigating threats across the network and application stack without disruption is one of the few solutions available that offers the elusive and very valuable benefit of time. Providers and enterprises alike need to consider their current data center architecture and whether it supports the notion of such a dynamic tier. If not, it’s time to re-evaluate and determine whether a strategic change of direction is necessary to ensure the ability of operations and security teams to address operational risk as quickly and efficiently as possible.

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: F5,MacVittie,security,architecture,availability,cloud computing,virtualization,devops,threat mitigation,application delivery,blog

Mobile versus Mobile: An Identity Crisis

Lori MacVittie — Mon, 16 Jan 2012 13:00:00 GMT

#mobileThe expansive options consumers revel in creates an identity crisis for IT that is best resolved via context-aware mobile mediation.

Back in the days of the browser wars, when standards were still largely ignored and the battle for the desktop was highly competitive, developers had to make choices and compromises. They could either write extensive client-side scripts to detect the user’s browser and address the peculiarities of that environment or they could simply ignore them with a disclaimer that “this site (works best when viewed in | was written for) browser X.”

As time went by, developers were able to discontinue this annoying practice as browser features converged and a common, standardized platform emerged upon which all applications were able to be delivered to any popular browser without concern.

Then mobile phones appeared, and the user experience degraded again, this time driven by the relatively feeble processing power of the platform. Small screens aside, the memory and processing power available on a mobile phone was such that – when combined with a constrained networking environment – the delivery of increasingly chunky, graphic heavy, interactive applications to mobile phones was simply a bad idea for both organizations and its visitors.

Developers and web ops returned to the inspection of HTTP headers to determine whether or not a visitor was using a mobile platform, and began writing leaner, more compact interfaces specifically for those platforms.

Enter tablets. Neither desktop nor phone, these admittedly mobile platforms are compact but nearly as powerful as their overweight tethered cousins without sacrificing the mobility of their anorexic form-factor brethren.

COMMON GROUND: MOBILE OPERATING SYSTEM

Tablets are certainly able to take advantage of emerging web application standards such as HTML5 to deliver a full, rich interactive desktop-style experience on a mobile platform, but are rarely offered it. They are by default offered up a mobile experience because of a common element: the operating system. Even if the operating system was masked, still they’ll find out by checking a second, lesser known HTTP header: HTTP_X_WAP_PROFILE. And if not there, then potentially in HTTP_PROFILE.

Unfortunately, this is not necessarily the fault of the developer. There are no standards prescribing what should or should not identity a mobile device, and thus manufacturers are left to their own devices (pun only somewhat intended).

There is no reliable way, today, to identify a mobile device accurately. Developers are left writing complex scripts that strip apart user agents, profile strings and whatever other contextual data they can extract from HTTP headers to determine how best to serve up content.

That often means visitors are served content that is not wholly appropriate for their device. Even if they could, the market is so volatile at this point that it’s a sure bet that a new device will enter soon that requires modification to the application yet again. More code, more string manipulation, more latency in processing.

CONTEXT-AWARE MOBILE MEDIATION

It would be nice if developers could simply receive accurate inbound HTTP headers. Headers that clearly identify the device not only from an operating system perspective, but from a form-factor and network perspective in a standard way. But there are no standards, yet, and may never be. Thus a solution may be found in imposing standards upon inbound requests specifically for developers to better address the disparities in resolution, functionality, and performance between the various mobile device types.

This requires some amount of pre-planning. Design, if you will, or architecture up front. It requires that developers and devops sit down together and determine a standard means of communicating information between infrastructure and the applications it supports. Consider the possibility of two custom HTTP headers, one identifying the network type and one specifying form-factor and device:

HTTP_X_NETWORK = “WIFI | MOBILE | LAN”

HTTP_X_DEVICE = “TABLET | PHONE | DESKTOP”

If developers could rely on these two custom HTTP headers to exist for every inbound request, they could then develop applications based upon these characteristics that were more appropriate for the given device and network over which the device connected. Implementation requires only minimal inspection and insertion on a context-aware mediating device such as a network-side scripting capable application delivery controller.

Because the application delivery controller is topologically positioned in a strategic point of control, it has visibility into the network, client, and server-side environments. This gives it the ability to better interpret and execute policies that govern the delivery of applications to optimize performance and assure availability, but it also provides the ability to extract and share pertinent data with applications and other infrastructure. This data can be shared in a number of ways, including modification of the payload, of the headers, insertion of new headers, removal of old headers, etc…

The flexibility inherent in network-side scripting solutions, particularly those capable of side-band connectivity, allows devops and developers to design and develop a solution that works for them – in their environment.

The advantage to such a solution lies not only in more accurate, actionable data to share with applications, but in its ability to easily be modified without negatively impacting the application. A second advantage is the ability of developers to also take into consideration the network characteristics of the mobile device, data generally not available or, if available, generally inaccurate. A mobile device today may be accessing an application via WiFi or a mobile network, and that piece of information is quite pertinent as the performance and capabilities of each network are quite different and have a significant impact on the end-user experience from a delivery perspective.

Yet this data is not available by default to developers and it cannot reliably be inferred from device type. By leveraging a context-aware mediating solution, however, it becomes possible to share this data with developers such that they are able to take that information into consideration when putting together a response to a given request.

While not a panacea, such a solution certainly provides a more consistent and overall accurate environment in which to deliver applications to the increasingly broad and diverse spectrum of mobile devices.

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: F5,MacVittie,mobile,HTML5,tablet,context-aware,performance,architecture,network-side scripting,iRules,strategic point of control,blog

F5 Friday: Creating a DNS Blackhole. On Purpose

Lori MacVittie — Fri, 06 Jan 2012 12:32:00 GMT

#infosec #DNS #v11 DNS is like your mom, remember? Sometimes she knows better.

Generally speaking, blackhole routing is a problem, not a solution. A route to nowhere is not exactly a good thing, after all. But in some cases it’s an approved and even recommended solution, usually implemented as a means to filter out bad packets at the routing level that might be malformed or are otherwise dangerous to pass around inside the data center.

This technique is also used at the DNS layer as a means to prevent responding to queries with known infected or otherwise malicious sites. Generally speaking, DNS does nothing more than act like a phone book; you ask for an address, it gives it to you. That may have been acceptable through the last decade, but it is increasingly undesirable as it often unwittingly serves as part of the distribution network for malware and other malicious intent.

In networking, black holes refer to places in the network where incoming traffic is silently discarded (or "dropped"), without informing the source that the data did not reach its intended recipient.

When examining the topology of the network, the black holes themselves are invisible, and can only be detected by monitoring the lost traffic; hence the name.

(http://en.wikipedia.org/wiki/Black_hole_(networking))

What we’d like to do is prevent DNS servers from returning addresses for sites which we know – or are at least pretty darn sure – are infected. While we can’t provide such safeguards for everyone (unless you’re the authoritative server for such sites) we can at least better protect the corporate network and users from such sites by ensuring such queries are not answered with the infected addresses.

Such a solution requires the implementation of a DNS blackhole – a filtering of queries at the DNS level. This can be done using F5 iRules to inspect queries against a list of known bad sites and returning an internal address for those that match. What’s cool about using iRules to perform this function is the ability to leverage external lookups to perform the inspection. Sideband connections were introduced in BIG-IP v11 and these connections allow external, i.e. off device, lookups for solutions like this. Such a solution is similar to the way in which you’d want to look up the IP address and/or domain of the sender during an e-mail exchange, to validate the sender is not on the “bad spammer” lists maintained by a variety of organizations and offered as a service.

Jason Rahm recently detailed this solution as architected by Hugh O’Donnel, complete with iRules, in a DevCentral Tech Tip. You can find a more comprehensive description of the solution as well as the iRules to implement in the tech tip.

v11.1: DNS Blackhole with iRules

Happy (DNS) Routing!

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: F5,F5 Friday,MacVittie,DNS,routing,blackhole,network,iRules,architecture,security,context-aware,blog

F5 Friday: Load Balancing MySQL with F5 BIG-IP

Lori MacVittie — Fri, 09 Dec 2011 13:41:00 GMT

Scaling MySQL just got a whole lot easier

load balancing MySQL – any database, really – is not a trivial task. Generally speaking one does not simply round robin your way through a cluster of MySQL databases as a means to achieve scalability. It is databases, in fact, that have driven a wide variety of scalability patterns such as sharding and partitioning to achieve the ultimate goal of high-performance and scalability simultaneously.

Unfortunately, most folks don’t architect their applications with scalability in mind. A single database is all that’s necessary at first, and because of the way in which the application interacts with the database, it doesn’t make sense to code in support for multiple database instances, such as is often implemented with a MySQL master-slave cluster. That’s because the application has to actually open a connection to the database in question. If you’re only starting with one database, you really can’t code in a connection to a separate instance.

Eventually that application’s usage grows and the demands upon the database require a more scalable approach. Enter the MySQL master/slave relationship. A typical configuration is to maintain the master as the “write” database, i.e. all updates and/or inserts must use the master, while the slave instance is used as a “read only” instance.

Obviously this means the application code must be changed to support this kind of functional sharding. Unless you leverage network server virtualization from a load balancing service capable of acting as a full-proxy at layer 7 (application) like BIG-IP.

This solution leverages iRules to implement database load balancing. While this specific example is designed to perform the common functional sharding pattern of read-write separation for a master-slave MySQL cluster, the flexibility of iRules is such that other architectural solutions can easily be designed using the same basic functions. Location based sharding is another popular means of scaling databases, and using the GeoLocation capabilities of BIG-IP along with iRules to inspect and route database requests, it should be a fairly trivial architectural task to implement.

The ability to further extend sharding or other distribution methodologies for scaling databases without modifying the application itself is a huge bonus for both developers and operations. By decoupling the application from the database, it provides a more flexibility set of scalability domains in which technology targeted scalability strategies can be leveraged independent of the other layers. This is an important facet of agile infrastructure architecture and should not be underestimated as a benefit of network server virtualization.

MySQL Load Balancing Resources:

MySQL Proxy iRule
MySQL Proxy iApp (deployment package for BIG-IP v11)

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: F5,F5 Friday,MacVittie,MySQL,sharding,database,load balancing,application delivery tier,infrastructure 2.0,architecture,iApp,v11,iRules

F5 Friday: Platform versus Product

Lori MacVittie — Fri, 18 Nov 2011 12:16:28 GMT

There’s a significant difference between a platform and a product, especially when it comes to architecting a dynamic data center

In the course of nearly a thousand blogs it’s quite likely you’ve seen BIG-IP referenced as a platform, and almost never as a product. There’s a reason for that, and it’s one that is increasingly becoming important as organizations begin to look at some major transformations to their data center architecture.

It’s not that BIG-IP isn’t a product. Ultimately, of course, it is in the traditional sense of the word. But it’s also a platform, an infrastructure platform, designed specifically to allow the deployment of application delivery-related services in a modular fashion. In the most general way, modern browsers are products and platforms, as they provide an application framework through which additional plug-ins (modules) can be deployed. BIG-IP is similar to this model with the noted exception that its internal application framework is intended for use by F5 engineers to develop new and integrate existing functionality as “plug-ins” within the core architectural framework we call TMOS™.

There are myriad reasons why this distinction is important. Primarily among them is a unified internal architecture implies internal, high-speed interconnects that allow inbound and outbound data to be shared across modules (plug-ins) without incurring the overhead of network-layer communication. Many developers can explain the importance of zero-copy operations as it relates to performance. Those that can’t will still likely be able to describe the difference between pass by reference and pass by value which, in many respects, has similar performance implications as the former simply passes a pointer to a memory location and the latter makes a copy. It’s similar to the difference between collaborative editing in Google Docs and tracking revisions in Word via e-mail – the former acts on a single, shared copy while the latter passes around the entire document.

Obviously, working on the same document at the same time is more efficient and ultimately faster than the alternative of passing around a complete copy and waiting for it to return, marked up with changes.

FROM THEORY to PRACTICE

This theory translates well to the architectural principles behind TMOS and the BIG-IP platform: inbound and outbound data is shared across modules (plug-ins) in order to reduce the overhead associated with traditional network-based architectures that chain multiple products together. While the end-result may be similar, performance will certainly suffer and the loss of context incurred by architectural chaining may negatively impact the effectiveness (not to mention capabilities) of security-related functions.

The second piece of the platform puzzle are programmatic interfaces for external, i.e. third-party, development. This is the piece of the puzzle that makes a platform extensible. F5 TMOS provides for this with iRules, a programmatic scripting language that can be used to do, well, just about anything you want to do to inbound and outbound traffic. Whether it’s manipulating HTML, JSON, or HTTP headers or inspecting and modifying IP packets (disclaimer: we are not responsible for the anger of your security and/or network team for doing this without their involvement), iRules allows you to deploy unique functionality for just about any situation you can think of. Most often these capabilities are used to mitigate emergent threats – such as the THC SSL Renegotiation vulnerability – but they are also used to perform a variety of operational and application-specific tasks, such as redirection and holistic error-handling. And of course, who could forget my favorite, the random dice roll iRule. While certainly not of value to most organizations, such efforts can be good for learning. (That’s my story and I’m sticking to it.)

TMOS is a full proxy, and is unique in its ability to inspect and control entire application conversations. This enables F5 to offer an integrated, operationally consistent solution that can act based on the real time context of the user, network, and application across a variety of security, performance, and availability concerns.

That means access control and application security as well as load balancing and DNS services leverage the same operational model, the same types of policies, the same environment across all services regardless of location or form-factor. iRules can simultaneously interact with DNS and WAF policies, assuming both BIG-IP GTM and BIG-IP ASM are deployed on the same instance. The zero-copy nature of the high-speed bus that acts as the interconnect between the switching backplane and the individual modules insures the highest levels of performance without requiring a traversal of the network.

Because of the lack of topological control in cloud computing environments – public and private – the need for an application delivery platform is increasing. The volatility in IP topology is true for not only server and storage infrastructure, but increasingly for the network as well, making the architecture of a holistic application delivery network using individually chained components more and more difficult, if not impossible.

A platform with the ability to scale out and across both physical and virtual instances while simultaneously sharing configuration to ensure operational consistency is a key component to a successful, cloud-based initiative whether its private, public, or a combination of both. A platform provides the flexibility and extensibility required to meet head on the challenges of highly dynamic environments while ensuring the ability to enforce policies that directly address and mitigate operational risk (security, performance, availability).

A product, without the extensibility and programmatic nature of a platform, is unable to meet these same challenges. Context is lost in the traversal of the network and performance is always negatively impacted when multiple network-based connections must be made. A platform maintains context and performance while allowing the broadest measure of flexibility in deploying the right solutions at the right time.

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: F5,F5 Friday,MacVittie,architecture,infrastructure,application delivery,platform,operational risk,cloud computing,virtualization,devops,blog

F5 Friday: Mitigating the THC SSL DoS Threat

Lori MacVittie — Fri, 28 Oct 2011 12:33:00 GMT

The THC #SSL #DoS tool exploits the rapid resource consumption nature of the handshake required to establish a secure session using SSL.

A new attack tool was announced this week and continues to follow in the footsteps of resource exhaustion as a means to achieve a DoS against target sites.

Recent trends in attacks show an increasing interest in maximizing effect while minimizing effort. This means a move away from traditional denial of service attacks that focus on overwhelming sites with traffic and toward attacks that focus on rapidly consuming resources, instead. Both have the same ultimate goal: overwhelming infrastructure, whether server or router or <insert infrastructure component of choice>.

The latest SSL-based attack falls into the modern category of denial of service attacks in that it’s not an attempt to overwhelm with traffic, but rather to consume resources on servers such that capacity and the ability to respond to legitimate requests is eliminated.

The blog post announcing the exploit tools explains:

Establishing a secure SSL connection requires 15x more processing power on the server than on the client.

THC-SSL-DOS exploits this asymmetric property by overloading the server and knocking it off the Internet.

This problem affects all SSL implementations today. The vendors are aware of this problem since 2003 and the topic has been widely discussed.

This attack further exploits the SSL secure Renegotiation feature to trigger thousands of renegotiations via single TCP connection.

-- THC SSL DOS Tool Released

As the blog points out, there is no resolution to this exploit. Common mitigation techniques include the use of an SSL accelerator, i.e. a reverse-proxy capable device with specialized hardware designed to improve the processing capability of SSL and associated cryptographic functions. Modern application delivery controllers like BIG-IP include such hardware by default and make use of its performance and capacity-enhancing abilities to offset the operational costs of supporting SSL-secured communication.

BIG-IP MITIGATION

There are actually several ways in which BIG-IP can mitigate the potential impact of this kind of attack. First and foremost is simply its higher capacity for connections and processing of SSL / RSA operations. BIG-IP can manage myriad more connections – secure or not – than a typical web server and thus it may be, depending on the hardware platform on which BIG-IP is deployed, that the mitigation rests merely on having a BIG-IP in the path of the attack.

In the case that it is not, or if organizations desire a more proactive approach to mitigation, there are two additional options:

1. SSL renegotiation, which is in part the basis for the attack (it’s what allows a relatively few clients to force the server to consume more and more resources), can be disabled in BIG-IP v11 and v10.2.3. This may break some applications and/or clients so this option may want to be left as a “last resort” or the risks carefully weighed before deploying such a configuration.

2. An iRule that drops connections over which a client attempts to renegotiate more than five times in a given 60-second interval can be deployed. As noted by David Holmes and the iRule author, Jason Rahm, “By silently dropping the client connection, the iRule causes the attack tool to stall for long periods of time, fully negating the attack. There should be no false-positives dropped, either, as there are very few valid use cases for renegotiating more than once a minute.”

The full details and code for the iRule can be found in the DevCentral article “SSL Renegotiation DOS attack – an iRule Countermeasure”

UPDATE 11/1/2011: David Holmes has included an optimized version of the iRule in his latest blog, "The SSL Renegotation Attack is Back." His version uses the normal flow key (instead of a random key), adds a log message, and optimizes memory consumption.

Regardless of the mitigating technique used, BIG-IP can provide the operational security necessary to prevent such consumption-leeching attacks from negatively impacting applications by defeating the attack before it reaches application infrastructure.

Stay safe!

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: MacVittie,F5,F5 Friday,BIG-IP,iRules,SSL,DoS,THC,renegotiation,cryptography,security,mitigation

Infrastructure Architecture: Whitelisting with JSON and API Keys

Lori MacVittie — Wed, 12 Oct 2011 11:31:00 GMT

Application delivery infrastructure can be a valuable partner in architecting solutions ….

AJAX and JSON have changed the way in which we architect applications, especially with respect to their ascendancy to rule the realm of integration, i.e. the API. Policies are generally focused on the URI, which has effectively become the exposed interface to any given application function. It’s REST-ful, it’s service-oriented, and it works well.

Because we’ve taken to leveraging the URI as a basic building block, as the entry-point into an application, it affords the opportunity to optimize architectures and make more efficient the use of compute power available for processing. This is an increasingly important point, as capacity has become a focal point around which cost and efficiency is measured. By offloading functions to other systems when possible, we are able to increase the useful processing capacity of an given application instance and ensure a higher ratio of valuable processing to resources is achieved.

The ability of application delivery infrastructure to intercept, inspect, and manipulate the exchange of data between client and server should not be underestimated. A full-proxy based infrastructure component can provide valuable services to the application architect that can enhance the performance and reliability of applications while abstracting functionality in a way that alleviates the need to modify applications to support new initiatives.

AN EXAMPLE

Consider, for example, a business requirement specifying that only certain authorized partners (in the integration sense) are allowed to retrieve certain dynamic content via an exposed application API. There are myriad ways in which such a requirement could be implemented, including requiring authentication and subsequent tokens to authorize access – likely the most common means of providing such access management in conjunction with an API. Most of these options require several steps, however, and interaction directly with the application to examine credentials and determine authorization to requested resources. This consumes valuable compute that could otherwise be used to serve requests.

An alternative approach would be to provide authorized consumers with a more standards-based method of access that includes, in the request, the very means by which authorization can be determined. Taking a lesson from the credit card industry, for example, an algorithm can be used to determine the validity of a particular customer ID or authorization token. An API key, if you will, that is not stored in a database (and thus requires a lookup) but rather is algorithmic and therefore able to be verified as valid without needing a specific lookup at run-time. Assuming such a token or API key were embedded in the URI, the application delivery service can then extract the key, verify its authenticity using an algorithm, and subsequently allow or deny access based on the result.

This architecture is based on the premise that the application delivery service is capable of responding with the appropriate JSON in the event that the API key is determined to be invalid. Such a service must therefore be network-side scripting capable. Assuming such a platform exists, one can easily implement this architecture and enjoy the improved capacity and resulting performance boost from the offload of authorization and access management functions to the infrastructure.

1. A request is received by the application delivery service.

2. The application delivery service extracts the API key from the URI and determines validity.

3. If the API key is not legitimate, a JSON-encoded response is returned.

4. If the API key is valid, the request is passed on to the appropriate web/application server for processing.

Such an approach can also be used to enable or disable functionality within an application, including live-streams. Assume a site that serves up streaming content, but only to authorized (registered) users. When requests for that content arrive, the application delivery service can dynamically determine, using an embedded key or some portion of the URI, whether to serve up the content or not. If it deems the request invalid, it can return a JSON response that effectively “turns off” the streaming content, thereby eliminating the ability of non-registered (or non-paying) customers to access live content.

Such an approach could also be useful in the event of a service failure; if content is not available, the application delivery service can easily turn off and/or respond to the request, providing feedback to the user that is valuable in reducing their frustration with AJAX-enabled sites that too often simply “stop working” without any kind of feedback or message to the end user.

The application delivery service could, of course, perform other actions based on the in/validity of the request, such as directing the request be fulfilled by a service generating older or non-dynamic streaming content, using its ability to perform application level routing.

The possibilities are quite extensive and implementation depends entirely on goals and requirements to be met.

Such features become more appealing when they are, through their capabilities, able to intelligently make use of resources in various locations. Cloud-hosted services may be more or less desirable for use in an application, and thus leveraging application delivery services to either enable or reduce the traffic sent to such services may be financially and operationally beneficial.

ARCHITECTURE is KEY

The core principle to remember here is that ultimately infrastructure architecture plays (or can and should play) a vital role in designing and deploying applications today. With the increasing interest and use of cloud computing and APIs, it is rapidly becoming necessary to leverage resources and services external to the application as a means to rapidly deploy new functionality and support for new features. The abstraction offered by application delivery services provides an effective, cross-site and cross-application means of enabling what were once application-only services within the infrastructure. This abstraction and service-oriented approach reduces the burden on the application as well as its developers.

The application delivery service is almost always the first service in the oft-times lengthy chain of services required to respond to a client’s request. Leveraging its capabilities to inspect and manipulate as well as route and respond to those requests allows architects to formulate new strategies and ways to provide their own services, as well as leveraging existing and integrated resources for maximum efficiency, with minimal effort.

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: F5,MacVittie,JSON,HTTP,architecture,integration,application delivery,cloud computing,devops,network-side scripting,dynamic infrastructure

F5 Friday: Zero-Day Apache Exploit? Zero-Problem

Lori MacVittie — Fri, 26 Aug 2011 15:21:32 GMT

#infosec A recently discovered 0-day Apache exploit is no problem for BIG-IP. Here’s a couple of different options using F5 solutions to secure your site against it.

It’s called “Apache Killer” and it’s yet another example of exploiting not a vulnerability, but a protocol’s behavior.

UPDATE (8/26/2011) We're hearing that other Range-* HTTP headers are also vulnerable. Take care to secure against these potential attack vectors as well!

In this case, the target is Apache and the “vulnerability” is in the way multiple ranges are handled by the Apache HTTPD server. The RANGE HTTP header is used to request one or more sub-ranges of the response, instead of the entire response entity. Ranges are sometimes used by thin clients (an example given was an eReader) that are memory constrained and may want to display just portions of the web page. Generally speaking, multiple byte ranges are not used very often.

RFC 2616 Section 14.35.2 (Range retrieval request) explains:

HTTP retrieval requests using conditional or unconditional GET methods MAY request one or more sub-ranges of the entity, instead of the entire entity, using the Range request header, which applies to the entity returned as the result of the request:
      Range = "Range" ":" ranges-specifier
A server MAY ignore the Range header. However, HTTP/1.1 origin servers and intermediate caches ought to support byte ranges when possible, since Range supports efficient recovery from partially failed transfers, and supports efficient partial retrieval of large entities.

The attack is simple. It’s a simple HTTP request with lots – and lots – of ranges. While this example uses the HEAD method, it can also be used with a GET.

  HEAD / HTTP/1.1 Host:xxxx  Range:bytes=0-,5-1,5-2,5-3,…

According to researchers testing the vulnerability, a successful attack requires a “modest” number of requests.

BIG-IP SOLUTIONS

There are several options to prevent this attack using BIG-IP solutions.

HEADER SANITIZATION

First, you can modify the HTTP profile to simply remove the Range header. HTTP header removal – and replacement – is a common means of manipulating request and response headers as a means to “fix” broken applications, clients, or enable other functionality. This is a form of header sanitization, used typically to remove non-compliant header values that may or may not be malicious, but are undesirable. The Apache suggestion is to remove any Range header with 5 or more values.

Note that this could itself break clients whose functionality expects a specific data set as specified by the RANGE header. As it is a rarely used header it is unlikely to impact clients adversely, but caution is always advised. Collaborate with developers and understand the implications before arbitrarily removing HTTP headers that may be necessary to application functionality.

HEADER VALUE SCRUBBING

You can also use an iRule to scrub the headers. By inspecting and thus detecting large numbers of ranges in the RANGE header, you can subsequently handle the request based on your specific needs. Possible reactions include removal of the header, rejection of the request, redirection to a honey pot, or replacement of the header.

Sample iRule code (always test before deploying into production!)

when HTTP_REQUEST {

    # remove Range requests for CVE-2011-3192 if more than 5 ranges are requested

    if { [HTTP::header "Range"] matches_regex {bytes=(([0-9\- ])+,){5,}} } {

        HTTP::header remove Range

    }

}

Again, changing an HTTP header may have negative consequences on the functionality of the application and/or client, so tread carefully.

BIG-IP ASM ATTACK SIGNATURE

Another method of mitigation using BIG-IP solutions is to use a BIG-IP Application Security Manager (ASM) attack signature to detect and act upon an attack using this technique. The signature to add looks like:

pcre:"/Range:[\t ]*bytes=(([0-9\- ])+,){5,}/Hi";

It is important to be aware of this exploit and how it works, as it is likely that once it is widely mitigated, attacks will begin (if they already are not) to explore the ways in which this header can be exploited. There are multiple “range” style headers, any of which may be vulnerable to similar exploitation, so it may be time to consider your current security strategy and determine whether the field of potential exploitable headers is such that a more negative approach (default deny unless specifically allowed) may be required to secure against future DoS attacks targeting HTTP headers.

There are also alternative solutions available already, including this writeup from SpiderLabs with a link to an OWASP mod_security rule file for mitigations.

Stay safe out there!

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: F5,F5 Friday,MacVittie,Apache,exploit,Range,HTTP,security,web application firewall,BIG-IP ASM,iRules

Infrastructure Scalability Pattern: Sharding Streams

Lori MacVittie — Fri, 08 Jul 2011 12:44:25 GMT

JSON Activity Streams offers some interesting new scalability pattern possibilities via layer 7 (application) switching.

One of the most interesting aspects of deploying applications is figuring out how to scale them. There’s many options, from simple scale out and scale up to more advanced architectural designs that take advantage of external, application switching services.

The flexibility in the latter has become more obvious with the advent of not just cloud computing , but its underlying virtualized auto-scaling technologies. Combined with more targeted scalability strategies, infrastructure services provide a more operationally and financially efficient means of scaling applications on-demand with little to no human intervention.

Sharding streams is a variation on a more general theme of sharding at the application layer. It involves an architectural design based on the JSON Activity Streams 1.0 specification that separates and scales individually (hyper-local scaling) services based on a particular facet of the “activity”.

SHARDING ACTIVITY STREAMS

Let’s first take a look at an example of an Activity Stream. You’ll note there are quite a few descriptors – verb, target, object – any of which might be a good choice on which to base a scalability domain.

The key to determining how to shard a stream is recognizing what particular objects may be separated out and delivery platforms optimized specifically for that object type. This is usually very intimately related to content type: audio, video, image, etc… These types of categorization make it much easier to optimize a particular platform to process and deliver such types of content with less overhead as tweaking time outs, session configuration and connection parameters can be accomplished with no impact on other processing. Specialization is, oddly enough, a benefit of commoditization and virtualization.

One can imagine that specialized scalability domains based on object (content) type is but one architectural option. Leveraging verb-based specialization is also possible, and provides the means by which a read-write separation at the database layers can be used to improve scalability and performance. Alternatively, it may be possible to dissect a stream based on “id” – using specific tags to perform different types of processing and distribution or even as the means to trigger specific operational policies or processes.

What we want to do is leverage the context-aware capabilities of an application delivery controller (a really smart load balancer, if you will) to inspect requests and ensure that users are routed to the appropriate pool of application servers based on the processing profile required by the specific request, as determined by “verb” or “target” or “object”. This also requires that the application delivery controller be capable of full-proxy inspection and routing of every request. This can be accomplished using the inspection capabilities and a set of network-side scripts that intercept, inspect, and then conditionally route each request based on the determination of which pool of resources is designated as the processor for the type of stream.

There are myriad ways to slice and dice the data to design a highly-scalable and efficient architecture. The key is understanding the data and designing an architecture to take advantage of its natural separation of type or action as to maximize the utilization of available resources.

TRANSFORMATION of IT

The power of virtualization and cloud computing to “transform” the data center is not just organizational and cultural, but architectural.

We’re finally moving beyond the simplistic methods of scalability and availability and security into a new realm where intelligent, actionable data is leveraged to make smarter, more efficient decisions regarding processing. Not only does this improve performance, a key goal, but it increases efficiency and allows IT to do more with less. We can work smarter, not harder, to achieve our operational and business goals if we start leveraging the power of architectural solutions.

It’s no longer enough to simply scale an application, you have to scale an entire architecture. And to do that you need an architecture that incorporates a deep understanding not just of infrastructure but of the applications it delivers. Ops has to dig into the data applications exchange and recognize the unique processing demands required by that data and then architect solutions that exploit the ability to optimize platforms or processing or provisioning of resources based on function or type. Highly disruptive technologies like virtualization and cloud computing afford IT opportunities to change the precepts upon which the data center is designed. It allows IT to take stock of what it has been doing and make changes that enable a more seamless transition toward where it is going: a dynamic data center. Architecture is key and IT must be approached as a reusable and flexible architecture, not just a set of interconnected technologies.

Infrastructure scalability patterns are architectural artifacts, designed to bridge the gap between operations and applications and provide the means by which architecture can impact performance and scalability in a positive way, while remaining as efficient as possible. The divide between operations and applications must be addressed in order to realize the benefits of architectural solutions, and that requires a new breed of data center professionals: devops. Folks who have a firm grasp of both infrastructure and applications and understand how to marry the two in an architectural solution that is fast, secure and available. Folks who recognize that web applications today may be developed as a single entity, but have different processing needs not only based on the application’s characteristics, but the characteristics of the users and devices from which the applications are used.

Infrastructure components are no longer discrete entities with a single purpose, they are highly complex, flexible platforms upon which the infrastructure services necessary to enable a truly dynamic data center can be deployed. Architectures aren’t just for applications anymore, they’re for the data center. Successful IT organizations will recognize and exploit the capabilities of modern infrastructure components and services as part of a more flexible, intelligent delivery architecture.

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: F5,MacVittie,Scalability,architecture,infrastructure,sharding,JSON,enterprise,application delivery,virtualization

Gaming the System: The $23,698,655.93 per hour Cloud Computing Instance?

Lori MacVittie — Mon, 02 May 2011 15:12:01 GMT

An interesting look at how automation combined with cloud computing resource brokering could go very, very wrong

Automation is not a new concept. People – regular old people – have been using it for years for tasks that require specific timing or reaction to other actions, like bidding on eBay or other auction-focused sites.

The general concept is pretty simple as it’s just an event-driven system that automatically performs an action when the specified trigger occurs. Usually, at least when money is concerned, there’s an upper limit. The action can’t be completed if the resulting total would be above a specified maximum amount.

Sometimes, however, things go horribly wrong.

THE MOST EXPENSIVE BOOK IN HISTORY

I was out trolling Facebook and happened to see a link to an article claiming a book was actually listed on Amazon for – wait for it, wait for it - $23,698,655.93. Seriously, it was listed for that much for a short period of time.

There’s a lengthy explanation of why and it turns out that an automated “pricing war” of sorts was to blame. Two competing sellers tried to keep their prices within specific percentages – one slightly below 100% of the other while the other tried to stay slightly above 100% of the other. The mathematically astute can figure out what happens when the differences were not equal – specifically the seller keeping his price higher used a higher percentage off 100% than the seller trying to stay below the other guy. Stair-step increases over time ultimately resulted in the price hitting over $23 million dollars before someone noticed what was going on.

Needless to say neither seller found a buyer at that price, mores the pity for them.

THE POTENTIAL DANGER for CLOUD BROKERS

The concept of cloud brokers, services that provide competitive bidding and essentially auctioning of cloud resources, is one that plays well into the demesne of commoditized resource services.

Commoditization, after all, engenders an environment in which the consumer indicates the value and therefore price they will pay for a resource, and generally providers of that resource respond. Allowing consumers to “bid” on the resource allows the market to determine the value in a very agile manner. Seasonal or event-driven spikes in capacity needs, for example, could allow those resources that are most valuable in those moments to rise in price, while at other times it may drive the price downward. While making it difficult, perhaps, to budget properly across a financial reporting period, such volatility can be positive as it also indicates to the market the price consumers will bear in general.

But assume that, like the Amazon marketplace, two such brokers begin setting prices based on each other rather than through market participation. Two brokers that wish to remain competitive, each with different value propositions such that one sets its price slightly lower than other, automatically, while the other sets the pricing of instances slightly higher than the other, automatically.

Indeed, you could arrive at the nearly $24 million dollar per hour cloud computing instance. Or nearly $24 million dollar block storage, or gigabit per second of bandwidth or whatever resource the two brokers are offering.

THE POTENTITIAL DANGER for DATA CENTERS

Now certainly this is an extreme – and unlikely - scenario. But if we apply the same concept to a dynamic, integrated infrastructure tasked with delivering applications based on certain business and operational parameters, you might see that the same scenario could become reality with slightly different impacts to the data center and the business it serves.

While not directly related to pricing, it is other policies regarding the security, availability and performance of a applications that could be impacted and problems compounded if controls and limitations are not clearly set upon automated responses to conditions within the data center. Policies that govern network speeds and feeds, for example, could impose limitations on users or applications based on prioritization or capacity. Other policies regarding performance might react to the initiation of those policies in an attempt to counter a degradation of performance, which again triggers a tightening of network consumption, which again triggers… You get the picture. Circular references – whether in a book or cloud computing resource market or internal to the data center infrastructure – can cascade such that the inevitable result is a negative impact on availability and performance.

Limitations, thresholds, and clear controls are necessary in any automated system. In programming we use the term “terminal condition” to indicate at what point a piece of given code should terminate, or exit, a potentially infinite loop. Such terminal conditions must be present in data center automation as a means to combat a potentially infinite loop between two or more pieces of infrastructure that control the flow of application data. Collaboration, not just integration, is critical. While Infrastructure 2.0 enables the integration necessary to support a context-aware data center architecture capable of adapting on-demand to conditions as a means of ensuring availability, security and performance goals are met, that integration requires collaboration across people – across architects and devops and admins – who can recognize such potential infinite loops and address them by implementing the proper terminal conditions in those processes.

COLLABORATION without CONTROL is BAD, M’KAY?

Whether the implementation is focusing on automating a pricing process or the enablement of a security or performance policy in the data center, careful attention to controls is necessary to avoid an infinite regression of policies that counteract one another. Terminal conditions, limitations, thresholds. These are necessary implements to ensure that the efficiencies gained through automation do not negatively impact application delivery. The slow but steady increase of a book beyond a “normal” price should have been recognized as being out of bounds in context – the context of the market, of activity, of other similar book pricing. IN the data center, the same contextual-awareness is necessary to understand why more capacity may be needed or why performance may be degrading. Is it a multi-layer (modern) attack? Is it a legitimate flash crowd of traffic? These questions must be able to be answered in order to properly adjust policies and ensure the right folks are notified in the event that changes in the volume being handled by the data center may be detrimental to not only the security but the budget of the data center and applications it is delivering.

Collaboration and integration go hand in hand, as do automation and control.

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: MacVittie,F5,collaboration,integration,dynamic infrastructure,cloud computing,devops,automation