Can the inherent abstraction of virtualization succeed where SOA did not?

My first read through a post on the Cloud Front Office led me to scoff disdainfully at the re-emergence of a concept central to a successful SOA implementation: the service catalog. Oh, we called it "registry" and then "registry/repository (reg/rep)" and finally "governance" but the concept behind it was exactly the same. Take a gander at the description of a cloud service catalog apparently growing out of discussions that began at Structure 09:

Last week I attended Structure 09, one of the better cloud computing conferences. Everyone I could hear the echo of the need for a service catalog; often it was implicit.

For example, everyone spoke about the need for standardization and multi-tenancy, about the need to abstract computing resources from the application to enable moving workloads as needed.  Which in my mind requires you to a) know what the workload is, b) know the SLA of that workload, c) have well defined underpinning technical services that compose that application service, and therefore d) Have a SERVICE CATALOG to track all of that!

Now, interestingly enough SOA governance utilized standards-based metadata like WSDL and WSIL to describe services. These descriptions included the endpoint (the physical location) of the service as well as the operations available and how to invoke them. In a full SOA implementation, the registry was at the heart of the architecture; if you wanted to invoke a service you queried the registry (usually via UDDI), retrieved the description (WSDL, WSIL), and then gathered the information necessary to invoke the service. This abstraction allowed services, ostensibly, to move and change over time without, allegedly, negatively impacting the client (consumers).

So while I was ready to excoriate this idea at first, a quiet hour trying to get a restless toddler back to sleep in the dark o'night led me to reconsider what appears to have been a rather hasty judgment. Might not this notion of a service catalog be more apposite than I first believed?

Absolutely.

Furthermore, the inherent chaos that could be caused by virtual machine movement within an on-demand infrastructure could be soothed. When a virtual machine (service) was launched, the descriptor could be updated to include its new physical location. In fact, assuming such services were launched in multiple locations (dare I suggest even across multiple clouds, as might be the case with intercloud) the descriptor could be updated to include all locations, perhaps even prioritized.

One of the interesting things virtualization brings to the table that SOA did not is the ability to abstract management of services. While WSDL abstracted a service and its operations away from the underlying implementation, that underlying implementation was still very much relevant in that managing the service required an understanding of the platform (.NET, Java EE, etc...) on which the service was deployed. You could not use WSDL or WSIL, for example, to actually manage the service - that was the raison d'être for SOA management solutions.

But virtualization solutions - likely because of many cloud implementations heavy reliance - provide management endpoints (APIs) through which virtual machine instances can be managed. VMs can be started and stopped and monitored through these endpoints (APIs) and the APIs are almost universally web-based, meaning the endpoint is the equivalent of a URI. That fits nicely into a service catalog and could easily be automated by any number of existing solutions (BPM systems come to mind immediately) or incorporated into custom solutions with relative ease.

Perhaps this is an extension to DNS? Perhaps it's something completely different. While many of the core concepts of cloud computing and virtualization are not new at all and, in fact, appear to be heavily borrowed from programmatic models of the past (distributed systems, SOA, object-oriented programming) some of the solutions that will be created to help manage cloud and virtualization will be new, because we have a completely new set of problems and issues to address caused by the convergence of a variety of programmatic and architectural models.

The use of SOA governance solutions never truly took the world by storm, and that may be in part that the metadata it carried wasn't "meta" enough given the level of abstraction used by SOA. Virtualization and cloud computing take that abstraction far enough to be useful both in invocation and management. SOA, too, was hampered by the fact that automation of processes - while nice - was not a necessary piece of the value equation. For cloud computing (on-demand) automation is one of the key variables in the benefit equation, making abstraction of management a necessity.

The more I think about this concept, the more I am liking where it may be going. Service catalogs may, in fact, be one of the "movers" of cloud in the enterprise, as it would offer a firm foundation on which automation and management solutions could be more easily built and deployed.

Related blogs & articles:

• Intercloud: The Evolution of Global Application Delivery
• Emergent Consensus: Private Cloud Needs a Front Office Part 1
• Building an elastic environment requires elastic infrastructure
• Building a Cloudbursting Capable Infrastructure

The importance of stress-testing in production

Everyone is still a-twitter over the problems the web experienced last week right after the news of Michael Jackson’s death. There have been

numerous stories on the fact that the Internet nearly fell over itself and died under the strain of trying to support the rush of millions of users as they queried, clicked, watched video, read blogs and news reports on the subject.

The Internet itself, of course, was just fine. The infrastructure comprising our electronic highway was humming along, routing packets happily here and there. What fell down were web applications and servers, overwhelmed with more concurrent connections than they could possibly handle despite their (I assume) high-availability infrastructure.

Obviously this is a difficult scenario to test for – or at least it appears to be. No organization is really able to test its production environment against an onrush of such a magnitude. The cost in software or hardware alone just to generate that much traffic would break most budgets even in normal economic times.

But to avoid the embarrassment of being “unavailable” it’s a necessity. Organizations need to test against worst case scenarios, i.e. an onslaught of visitors of a magnitude most sites can only dream of seeing, to ensure not only that their web applications can handle the load but to work out any kinks that may be lurking in their infrastructure, waiting for a heavy load to appear.

Developers know that sometimes defects only show up under certain conditions. Sometimes it’s extended use (memory leaks) and sometimes it’s high capacity (out of memory, artificial limits on arrays, etc…). Those conditions are hard to simulate let alone find in a test environment because all too often organizations don’t have the means to adequately force those conditions. Time and traffic cost money, and any one who has worked inside an enterprise organization knows that QA and testing often have more limited budgets than most IT departments.

And sometimes, just sometimes, it’s not even the applications’ fault. Sometimes a problem will appear when an application is under heavy load in the infrastructure; a configuration choice or scripting error can appear suddenly at higher loads. It may have never have impacted the availability or performance of an application at “normal” load but brings it right to its knees at high load. For example, I was testing some application network infrastructure some years ago that exhibited strange behavior only when the device reached 48,000 concurrent users. I could reproduce the issue at will, and it led to the discovery that the device was not freeing up connections from its session state table. It was full and couldn’t handle any more. Once the engineers knew that they could fix it and it was able to handle hundreds of thousands of concurrent users. But it wasn’t a problem with the application, it was the infrastructure. Without the ability to stress that infrastructure beyond its typical usage we would have never found the problem and it would have been discovered, unhappily, by a real customer in a real situation.

I was chatting with an old colleague who is now with a startup, SOASTA, that offers cloud-based stress testing of web applications. They’ve seen the same types of problems in production environments. Spinning up a web application for a very busy seasonal rush an organization called on SOASTA to perform some stress testing. All was going well until the application hit heavy load – 80,0000 or more transactions a second – and then suddenly things went wonky (that’s official testing terminology there, really, trust me ;-)). The load balancer began sending all the requests to one pool, overwhelming it with connections and causing performance and availability to rapidly degrade. They quickly discovered the source of the problem (a conditional configuration error), addressed it and tried again the next night. Voila! The application and its supporting infrastructure performed as expected: supporting hundreds of thousands of users and transactions per second at a more than acceptable performance rate.

I can’t stress enough (sorry, pun intended) the importance today of stress-testing web applications in production. If your business whether because of revenue or presence requires high availability and well-performing applications then you can’t afford not to stress test your application before the inevitable occurs. It’s expensive to do it yourself, which is why so many web applications haven’t been tested to their full limits, but an increasing number of cloud-based services are making it not only affordable, but a no-brainer.

There are many other “EVENTS” which can occur that will drive usage of your web application to its limits. There are many other “EVENTS” that will drive your infrastructure past its limits, too, and those two limits may not be the same after all. It behooves any organization that depends on web applications to test both its application and infrastructure capacity before it buckles under the load of some “event” – the timing of which cannot be foreseen.

So get out there, take a deep breath, and boldly go where no production application has gone before: stressed to its limits purposefully just to see if – and when - it will break.

P.S. You get +8 geek points if you know the reference for EVENTS being capitalized.Tweet me if you think you know the answer.

Related blogs & articles:

• Jackson dies, almost takes Internet with him
• News sites swamped following Michael Jackson’s death
• Debate: Can the Internet handle big breaking news?
• Infrastructure Matters: Challenges of Cloud-based Testing
• Load Testing as a Service: A Look at Load Impact (beta)

The concept of an “intercloud” is floating around the tubes and starting to gather some attention. According to Greg Ness you can “Think of the intercloud as an elastic mesh of on demand processing power deployed across multiple data centers. The payoff is massive scale, efficiency and flexibility.”

Basically, the intercloud is the natural evolution of global application delivery. The intercloud is about delivering applications (services) from one of many locations based on a variety of parameters that will be, one assumes, user/organization defined. Some of those parameters could be traditional ones: application availability, performance, or user-location. Others could be more business-focused and based on such tangibles as cost of processing.

Greg, playing off Hoff, explains:

For example, services could be delivered from one location over another because of short term differentials in power and/or labor costs. It would also give enterprises more viable options for dealing with localized tax or regulatory changes.

The intercloud doesn’t yet exist, however---It has at least one missing piece: the automation of manual tasks at the core of the network. The intercloud requires automation of network services, the arcane collection of manual processes required today to keep networks and applications available.

Until there is network service automation, all intercloud bets are off.

What I find eminently exciting about the intercloud concept is that it requires a level of intelligence, of contextual awareness, that is the purview of application delivery. We’re calling them services again, like we did when SOA was all the rage, but in the end even a service can be considered an application – it’s a self-contained piece of code that executes a specific function for a specific business purpose. If it makes it  easier to grab onto, just call “application delivery” “service delivery” because there really isn’t too much of a difference there. But intercloud requires a bit more awareness than global application delivery; specifically it requires more business and data center specific awareness than we have available.

On the surface intercloud sounds a lot like what we do today in a globally load balanced environment: application services are delivered from the data center that makes the most sense based on variables (context) surrounding the request including the user, the state of the data center, the networks involved, and the applications themselves. Global application delivery decisions are often made based on availability or location, but when the global application delivery infrastructure is able to collaborate with the local application delivery infrastructure the decision making process is able to get a lot more granular. Application performance, network conditions, capacity – all can be considered as part of the decision regarding which data center should service any given request.

I rarely disagree with Greg and, on the surface at least, he is absolutely right in that we need to automate processes before the intercloud can come to fruition. But we are also missing one other piece: the variables that are peculiar to the business and data centers comprising the intercloud and the integration/automation that will allow global application delivery infrastructure to take advantage of those variables in an efficient manner. That data, likely, is assumed in the need to automate as without that data there’s not nearly enough to automate decisions across data centers in the way in which Greg and Hoff expect such systems to do.

Don’t get me wrong – an intelligent global application delivery system can be configured with such information today, but it’s a manual process and manual processes don’t scale well. This is why Greg insists (correctly) that automation is the key to the intercloud. Assuming that the cost of power, for example, changes throughout the day and, in fact, might be volatile in general means that manually reconfiguring the global application delivery system would be necessary. That simply wouldn’t be feasible. A system for providing that information – and any other information which would become the basis for request routing across distributed data centers – needs to be constructed and subsequently able to be integrated into the massive management system that will drive the intercloud.

It makes a certain amount of sense, if you think about it, that global application delivery would also need to evolve into something more; capable of context awareness at a higher point of view than local application delivery. Global application delivery will be the foundation for intercloud because it’s already performing the basic function – we just lack the variables and the automation necessary for global application delivery solutions to take the next step and become intercloud controllers.

But they will get there.

Related blogs & articles:

• The Infrastructure 2.0 Trifecta
• Infrastructure 2.0: As a matter of fact that isn't what it means
• The Context-Aware Cloud
• The Intercloud Makes Networks Sexy Again
• Hoff’s Metastructure and the Intercloud
• Cloudanatomy: Infrastructure, Metastructure & Infostructure

Somebody has to be first

Recently Microsoft came up with a solution, supported natively in IE8, to protect against clickjacking attempts. Apparently some folks have decided that because Microsoft has a history of implementing proprietary solutions that this one, too, must be proprietary. These same folks must also have very little understanding of today’s web application architectures, as they declared the solution pretty much useless based on some pretty poor assumptions regarding the implementation of said solution. 

As noted in the Register, “some critics have contended the protection [X-FRAME-OPTIONS custom HTTP header] will be ineffective because it will require millions of websites to update their pages with proprietary code.” 

Where to start, oh where to start…

Proprietary implies usable only by a certain subset of products: those endorsed by the creator of the “proprietary” solution. Proprietary implies peculiar to a specific technology. Proprietary implies closed. Proprietary means things like ActiveX, which only work properly on a Windows platform. Proprietary means keeping the implementation secret so no one else can implement it. Microsoft implemented a solution based on a custom HTTP header. A header, I might add, that anyone using any browser running on any platform is quite capable of seeing. A header that  can be easily handled by any other browser developer, if they so choose, to implement similar behavior. A header based on accepted standards.

A custom HTTP header can be interpreted by any browser if it so chooses as well as intermediaries capable of intelligently handling HTTP. The speedy support via NoScript for X-FRAME-OPTIONS proves the solution is hardly “proprietary” and not applicable to just Microsoft’s solution.

The term proprietary is often considered synonymous with Microsoft but in this case calling the solution proprietary is simply the result of a bad habit apparently difficult for some to break.

Unlike the use of other truly proprietary solutions in the browser – like XMLHTTPRequest object before it was widely adopted as a “standard” – custom HTTP headers are about as innocuous as you can get. Their existence neither hinders nor helps browsers that don’t support it and it does absolutely no harm to applications or the user-experience to carry it along even to clients that will see no benefit from it. And if the developers of those browsers/clients decide to support it, the existence of such headers will immediately provide a benefit.

“…it will require millions of websites to update their pages with proprietary code.” 

Excuse me while I decide whether to laugh or have a conniption fit at this one.

This assumption shows a decided lack of understanding of architecture. While it’s certainly true that one way to support the use of X-FRAME-OPTIONS is to update web applications (let us not fall into the trap of digressing into an argument over the use of the term “pages” and “code” to describe an application…) there are are other, less disruptive, solutions.

The organizations that are most likely to be the subject of a clickjacking attack are financial institutions, retailers, and perhaps social networking sites. It seems likely that these folks have built out an architecture capable of scaling and assuring availability of their sites. If you think not true, consider how often you’ve attempted to visit one of those sites and not been able to do so. Exactly. They’re smart people; they’ve built out an environment that almost certainly involves intermediaries (load balancers, application firewalls, application switches, application delivery controllers, proxies, etc…) that do other forms of application layer inspection and, perhaps, manipulation. They have other things to worry about, after all, like data leak prevention and content filtering and stopping SQLi and other nasty attacks from destroying their critical applications. So they’ve likely already got an intermediary in place that’s capable of easily adding a custom HTTP header on the way out the door.

As I previously noted in Clickjacking Protection Using X-FRAME-OPTIONS Available for Firefox folks can use network-side scripting solutions to effortlessly insert the custom header as necessary without modifying code. If an application is comprised of say, an average of ten pages (I’m low-balling the estimate, I know that, but you can do the math on your own applications yourself), then this solution is 10 times more efficient than modifying the associated code. And network-side scripting solutions are likely already in place at most organizations that are likely to be targeted by clickjacking attacks. So all they have to do is write a little script and voila! This “proprietary, ineffective” solution starts working. Then they encourage customers to upgrade to IE8 or install the proper version of NoScript and, well, the Internet is safe again (at least for about 2.71828182845904523536 or “e” seconds).

Just because an organization comes out with a solution first doesn’t automatically make it proprietary, unless they’ve already patented it and in that case, well it’s a whole different ball game. Microsoft saw a problem, decided on a solution, and implemented it.

What should be noted is that they did it in a way that made it possible for other browser vendors to take advantage of the same constructs to implement a similar solution. They used a custom HTTP header that’s portable and browser and operating system agnostic when they could have found any number of other, truly proprietary ways to solve this problem.

I’ve been down on Microsoft and its products in the past but in this case the software giant doesn’t deserve the negativity involved with calling this a proprietary solution. This is a far sight better than the framekiller options available today and it certainly doesn’t require the amount of work implied by these “critics” to leverage the solution.

Related blogs & articles:

• Clickjacking Protection Using X-FRAME-OPTIONS Available for Firefox
• Hypertext Transfer Protocol – HTTP/1.1 RFC 2616
• Picard and Dathon at El-Adrel
• Jedi Mind Tricks: HTTP Request Smuggling
• Why not network-side pre-fetching?
• How to brandalize your content with network-side scripting
• I am in your HTTP headers, attacking your application

This was recently. Like, this year. Like in the past couple of weeks.

So when folks start preaching about moving applications and, in some cases, the entire infrastructure to “the cloud”, I laugh. Ruefully. Cause the core of their critical systems run on the mainframe and are written in COBOL, and they’re not going anywhere.

As @jamesurquhart points out not-so-delicately via Twitter, forklift replacement approaches have never worked. Neither has “rip and replace” or any other approach that advocates “all or nothing” approaches to change in IT.

Rather new technology is often used for new applications where it makes sense, just as @monkchips explains was his initial intent. So yes, this Fortune 500 company is going to experiment with cloud bursting, and it has a wide variety of web services, leverages Java and COBOL at the same time, and implements some web applications using .NET. There’s something for everyone in their data center.

The reality is that, like the Fortune 500 my mother works for, organizations have a dizzying array of architectures and technologies mashed up together to support their business. COBOL applications, client-server, web applications, web services, and cloud are going to continue to work together in the same data center for the foreseeable. The prediction that no one will need to run their own data centers in the future due to cloud is simply a pipe-dream.

That means network and application network infrastructure has to support a wide variety of technology and architectures, and it must do it simultaneously. Even organizations who plan to move everything to a cloud environment – internal or external – aren’t going to do so “overnight.” There is no rip-and-replace for cloud, and even if there were it wouldn’t likely be a painless transition. So organizations are going to move slowly, as they are always wont to do, and need an infrastructure that is flexible enough to support whatever is thrown at it.

Unified application delivery and data services is one of the ways in which an organization can gain the agility necessary to move toward a hybrid – or pure if that’s the goal, though I doubt it – environment. The ability to integrate (collaborate) with applications and other systems in the overall IT ecosystem mean that a unified infrastructure solution can allow IT to benefit both traditional and emerging data center models at the same time.

As organizations try to move toward a more efficient, virtual infrastructure there needs to be a layer of abstraction in the infrastructure that insulates users and in some cases systems from the rapid rate of change that occurs as architecture is changed. Unified application delivery provides that abstraction by supporting both virtual application services at the same time as it supports traditional application services. indeed, unified application delivery can support an application comprised of both virtual and physical services at the same time in sort of an “internal cloudbursting” architecture.

Consider that you may have a dedicated set of physical resources for an application, but need to meet seasonal or event-based demand. A unified application delivery solution provides the “interface” to the customer/user for the application on the physical resources, but can also just as easily handle additionally provisioned virtual resources necessary to meet that demand.

That’s just one niche case where the traditional architecture mingles freely with emerging data center models and needs to be supported as a single, unified resource. Imagine this now as a step toward a fully on-demand architecture. Without interruption to clients/customers/partners you can easily replace the traditional resources with on-demand, without disruption of service. Folks familiar with SOA will recognize this concept, as will developers familiar with polymorphic methods: abstraction in the infrastructure works the same way and provides many of the same benefits.

It is not realistic to expect that existing data centers will ever fully move to an external cloud nor will they internally ever become fully “cloudized”. It’s just not feasible given the vast array of technology and architectures that have been put into place over the past fifty years. But certainly some aspects of the data center will move to new models, and the infrastructure must be able to support both at the same time. Even better if the infrastructure can abstract resources of any kind to provide a unified view of the applications which IT is tasked with delivering and securing.

Related blogs & articles:

• How Reality Blew the Cloud Away at Interop
• Structure 09: Facebook’s Jonathan Heiliger Talks Infrastructure and Usernames
• F5 and the Cloud
• Silos belong on farms, not in IT
• Your load balancer wants to take a level of fighter and wizard
• The Revolution Begins: Arming the Data Center
• The Revolution Continues: Let Them Eat Cloud
• Building a Cloudbursting Capable Infrastructure
• Please fasten your seatbelts, there’s turbulence in that there cloud

Whether you are aware of it or not, if you’re deploying applications in the cloud or building out your own “enterprise class” cloud, you’re going to be using load balancing. Horizontal scaling of applications is a fairly well understood process that involves (old skool) server virtualization of the network kind: making many servers (instances) look like one to the outside world. When you start adding instances to increase capacity for your application, load balancing necessarily gets involved as it’s the way in which horizontal scalability is implemented today.

The fact that you may have already deployed an application in the cloud and scaled it up without recognizing this basic fact may lead you to believe you don’t need to care about load balancing options. But nothing could be further from the truth. If you haven’t asked already, you should. But more than that you need to understand the importance of load balancing and its implications to the application. That’s even more true if you’re considering an enterprise cloud, because it will most assuredly be your problem in the long run. Do not be fooled; the options available for load balancing and assuring availability of your application in the cloud will affect your application – if not right now, then later.

So let’s start with the five most important things you need to ask about load balancing and cloud environments regardless of where they may reside.

---

#5 DIRECT SERVER RETURN

---

If you’re going to be serving up video or audio – real-time streaming media – you should definitely be interested in whether or not the load balancing solution is capable of supporting direct server return (DSR). While there are pros and cons to using DSR, for video and audio content it’s nearly an untouchable axiom of application delivery that you should enable this capability. DSR basically allows the server to return content directly to the client without being processed by any intermediary (other than routers/switches/etc… which of course need to process individual packets). In most load balancing situations the responses from the server are returned via the same path they took to get to the server, notably through the load balancer. DSR allows responses to return outside the path of the load balancer or, if still returning through it, to do so unmolested. In the latter scenario the load balancer basically acts as a simple packet forwarder and does no additional processing on the packets.

The advantage to DSR is that it removes any additional latency imposed by additional processing by intermediaries. Because real-time streaming media is very sensitive to the effects of latency (jitter), DSR is often suggested as a best practice when load balancing servers responsible for serving such content.

Question: Is it supported?

---

#4 HEALTH CHECKING

---

One of the ways in which load balancers/application delivery controllers make decisions regarding which server should handle which request is to understand the current status of the application. It’s part of being context-aware, and it provides information about the application that is invaluable not just to the load balancing decision but to the overall availability of the application. Health checking allows a load balancing solution to determine whether a server/instance is “available” based on a variety of factors. At the simplest level an ICMP ping can be used to determine whether the server is available. But that tells it nothing of the state of the application.

A three-way TCP handshake is the next “step” up the ladder, and this will tell the load balancing solution whether an application is capable of accepting connections, but still tells it nothing of the state of the application. A simple HTTP GET takes it one step further, but what’s really necessary is the ability of the load balancing solution to retrieve actual data and ensure it is valid in order to consider an application “available”.

As the availability of an application may be (and should be if it is not) one way to determine whether new instances are necessary or not, the ability to determine whether the actual application is available and responding appropriately are important in keeping costs down in a cloud environment lest instances be launched for no reason or – more dangerously – instances are not launched when necessary due to an outage or failure.

In an external cloud environment it is important to understand how the infrastructure determines when an application is “available” or “not”, based on such monitoring, as the subtle differences in what is actually being monitored/tested can impact application availability.

Question: What determines when an application (instance) is available and responding as expected?

---

#3 PERSISTENCE

---

Persistence is one of the most important facets of load balancing that every application developer, architect, and network professional needs to understand. Nearly every application today makes heavy use of application sessions to maintain state, but not every application utilizes a shared database model for its session management. If you’re using standard application or web server session features to manage state in your application, you will need to understand whether the load balancing solutions available supports persistence and how that persistence is implemented.

Persistence basically ensures that once a user has been “assigned” a server/instance that all subsequent requests go to that same server/instance in order to preserve access to the application session. Persistence can be based on just about anything depending on the load balancing solution available, but most commonly takes the form of either source ip address or cookie-based. In the case of the former there’s very little for you to do, though you should be somewhat concerned over the use of such a rudimentary method of enabling persistence as it is quite possible – probable, in fact – that many users will be sharing the same source IP address based on NAT and masquerading at the edge of corporate and shared networks.

If the persistence is cookie-based then you’ll need to understand whether you have the ability to determine what data is used to enable that persistence. For example, many applications used PHPSESSIONID or ASPSESSIONID as it is routine for those environments to ensure that these values are inserted into the HTTP header and are available for such use. But if you can’t configure the option yourself, you’ll need to understand what values are used for persistence and to ensure your application can support that value in order to match up users with their application state.

Question: How is persistence implemented?

---

#2 QUIESCING (BLEEDING) CONNECTIONS

---

Part of the allure of a cloud architecture is the ability to provision resources on-demand. With that comes the assumption that you can also de-provision resources when they are no longer needed. One would further hope this process is automated and based on a policy configurable by the user, but we are still in the early days of cloud so that may be just a goal at this point.

Load balancers and clustering solutions can usually be told to begin quiescing (bleeding off) connections. This means that they stop distributing requests to the specified servers/instances but allow existing users to continue using the application until they are finished. It basically takes a server/instance out of the “rotation” but keeps it online until all users have finished and the server/instance is no longer needed. At that point either through a manual or automated process the server/instance can be de-provisioned or taken offline. This is often used in traditional data centers to enable maintenance such as patching/upgrades to occur without interrupting application availability. By taking one server/instance at a time offline the other servers/instances remain in service, serving up requests. In an on-demand environment this is of course used to keep costs controlled by only keeping the instances necessary for current capacity online.

What you need to understand is whether that process is manual, i.e. you need to push a button to begin the process of bleeding off connections, or automated. If the latter, then you’ll need to ask about what variables you can use to create a policy to trigger the process. Variables might be number of total connections, requests, users, or bandwidth. It could also, if the load balancing solution is “smart enough” include application performance (response time) or even time of day variables.

Question: How do connections quiesce (bleed) off – manually or automatically based on thresholds?

---

#1 FAILOVER

---

We talk a lot about the cloud as a means to scale applications, but we don’t very often mention availability. Availability usually means there needs to be in place some sort of “failover” mechanism, in case an application or server fails. Applications crash, hardware fails, these things happen. What should not happen, however, is that the application becomes unavailable because of these types of inevitable problems. If one instance suddenly becomes unavailable, what happens?

That’s the question you need to ask. If there is more than one instance running at that time, then any load balancing solution worth its salt will direct subsequent requests to the remaining available instances. But if there are no other instances running, what happens? If the provisioning process is manual, you may need to push a button and wait for the new instance to come online. If the provisioning process is not manual, then you need to understand how long it will take for the automated system to bring a new instance online, and perhaps ask about the ability to serve up customized “apology” pages that reassure visitors that the site will return shortly.

Question: What kind of failover options are available (if any)?

Folks seem to talk and write as if cloud computing relieves IT staff (customers) of the need to understand the infrastructure and architecture of the environments in which applications will be deployed. Because there is an increasingly symbiotic relationship between applications and its infrastructure – both network and application network – this fallacy needs to be exposed for the falsehood it is. It is more important today, with cloud computing, than it ever has been for all of IT – application, network, and security – to understand the infrastructure and how it works together to deliver applications.

That means there are no stupid questions when it comes to cloud computing infrastructure. There are certainly other questions you can – and should – ask a potential provider or vendor in order to make the right decision about where to deploy your applications. Because when it comes down to it it’s your application and your customers, partners, and users are not going to be calling/e-mailing/tweeting the cloud provider; they’re going to be gunning for you if things don’t work as expected.

Getting the answers to these five questions will provide a better understanding of how your application will handle unexpected failures, allow you to plan appropriately for maintenance/upgrades/patches, and formulate the proper policies for dealing with the nuances of a load balanced application environment.

Don’t just ask about product/vendor and hope that will answer your questions. Sure, your cloud provider may be using F5 or another advanced application delivery platform, but that doesn’t mean that they’re utilizing the product in a way that would offer the features you need to ensure your application is always available. So dig deeper and ask questions. It’s your application, it’s your responsibility, no matter where it ends up running.

Related blogs & articles:

• And the Killer App for Private Cloud Computing Is…
• Not All Virtual Servers are Created Equal
• Infrastructure 2.0: The Feedback Loop Must Include Applications
• Cloud Computing: Is your cloud sticky? It should be
• The Disadvantages of DSR (Direct Server Return)
• Cloud Computing: Vertical Scalability is Still Your Problem
• Server Virtualization versus Server Virtualization

But browser support is only half the solution, don’t forget to implement the server-side, too.

Clickjacking, unlike more well-known (and understood) web application vulnerabilities, has been given scant amount of attention despite its risks and its usage. Earlier this year, for example, it was used as an attack on Twitter, but never really discussed as being a clickjacking attack. Maybe because aside from rewriting applications to prevent CSRF (adding nonces and validation of the same to every page) or adding framekillers there just haven’t been many other options to prevent the attack technique from being utilized against users. Too, it is one of the more convoluted attack methods out there so it would be silly to expect non-technical media to understand it let alone explain how it works to their readers.

There is, however, a solution on the horizon. IE8 has introduced an opt-in measure that allows developers – or whomever might be in charge of network-side scripting implementations – to prevent clickjacking on vulnerable pages using a custom HTTP header to prevent them from being “framed” inappropriately: X-FRAME-OPTIONS.

The behavior is described in the aforementioned article as:

If the X-FRAME-OPTIONS value contains the token DENY, IE8 will prevent the page from rendering if it will be contained within a frame. If the value contains the token SAMEORIGIN, IE will block rendering only if the origin of the top level-browsing-context is different than the origin of the content containing the X-FRAME-OPTIONS directive. For instance, if http://shop.example.com/confirm.asp contains a DENY directive, that page will not render in a subframe, no matter where the parent frame is located. In contrast, if the X-FRAME-OPTIONS directive contains the SAMEORIGIN token, the page may be framed by any page from the exact http://shop.example.com origin.

But that’s only IE8, right? Well, natively, yes. But a development version of NoScript has been released that supports the X-FRAME-OPTIONS header and will provide the same protections as are natively achieved in IE8.

The problem is that this is only half the equation: the X-FRAME-OPTIONS header needs to exist before the browser can act on it and the preventive measure for clickjacking completed. As noted in the Register, “some critics have contended the protection will be ineffective because it will require millions of websites to update their pages with proprietary code.”

That’s not entirely true as there is another option that will provide support for X-FRAME-OPTIONS without updating pages/applications/sites with proprietary code: network-side scripting. The “proprietary” nature of custom HTTP headers is also debatable, as support for Firefox was provided quickly via NoScript and if the technique is successful will likely be adopted by other browser creators.

HOW-TO ADD X-FRAME-OPTIONS TO YOUR APPLICATION – WITH or WITHOUT CODE CHANGES

Step 1: Add the custom HTTP header “X-FRAME-OPTIONS” with a value of “DENY” or “SAMEORIGIN” before returning a response to the client

Really, that’s it. The browser takes care of the rest for you. OWASP has a great article on how to implement a ClickjackFilter for JavaEE and there are sure to be many more blogs and articles popping up describing how one can implement such functionality in their language-of-choice. Even without such direct “how-to” articles and code samples, it is merely a matter of adding a new custom HTTP header – examples of which ought to be easy enough to find.

Similarly a solution can be implemented using network-side scripting that requires no modification to applications.

In fact, this can be accomplished via iRules in just one line of code:

when HTTP_RESPONSE { 
    HTTP::header insert "X-FRAME-OPTIONS" “(DENY || SAMEORIGIN)”
}

I believe the mod_rewrite network-side script would be as simple, but as I am not an expert in mod_rewrite I will hope someone who is will leave an appropriate example as a comment or write up a blog/article and leave a pointer to it.

A good reason to utilize the agility of network-side scripting solutions in this case is that it is not necessary to modify each application requiring protection, which takes time to implement, test, and deploy. An even better reason is that a single network-side script can protect all applications, regardless of language and deployment platform, without a lengthy development and deployment cycle.

Regardless of how you add the header, it would be a wise idea to add it as a standard part of your secure-code deployment requirements (you do have those, don’t you?) because it doesn’t hurt anything for the custom HTTP header to exist and visitors using X-FRAME-OPTIONS enabled browsers/solutions will be a lot safer than without it.

Related blogs & articles:

• Stop brute force listing of HTTP OPTIONS with network-side scripting
• Jedi Mind Tricks: HTTP Request Smuggling
• I am in your HTTP headers, attacking your application
• Understanding network-side scripting
• 9 ways to use network-side scripting to architect faster, scalable, more secure applications 

I am not a number, I am a free man! – "The Prisoner", sampled by Iron Maiden (edited because geeks are picky and well, they're right even though I always think of Maiden and Eddie first before getting to the actual origins)

We, meaning everyone who deals with technology for a living, know that the move to IPv6 is inevitable. We simply must migrate in order to maintain the scalability of the Internet and its infrastructure. Well, we could continue to use technologies like NAT and SNAT in order to conserve IPv4 addresses, but really that’s just not practical in the long run. Eventually even doing that we are likely to “run out” of IP addresses, so it’s about time we started doing something about it.

The migration has already begun, quietly, but as more consumer facing services begin to make the move expect to see more coverage and analysis of IPv6 in general, such as is the case with Comcast’s recent announcement it was beginning the transition.

Comcast is making a move to IPv6, making IPv6 transit services available to its wholesale customers.

This a significant development because in less than two years, the IPv4 address space will be exhausted, but it doesn't mean there won't be any more Internet addresses. IPv4 has a 32-bit address size, allowing for only 4.3 billion addresses. In contrast, IPv6 is a 128-bit address space allowing for a staggering 340,282,366,920,938,463,463,374,607,431,768,211,456 possible addresses.

Staggering, indeed. Remember not too recently we looked at statistics regarding the number of Internet users across the globe, with the estimates from Nielsen coming in around 1,596,270,108. That number, while large, seems incredibly insignificant next to the number of addresses possible with IPv6.

So what are we going to do with all those addresses?

The number of IPv6 addresses available makes it possible, and probable, that every device – mobile, fixed, dynamic – connected to the Internet could have its own IPv6 address. This would, of course, solve all sorts of problems with IPv4 and NAT (Network Address Translation) that has to worked around every day just so things on the Internet work as expected.

Along with this possibility comes another possibility; one that is both promising and petrifying at the same time: individually assigned IP addresses.

Think about it – if every person had an IP address assigned to them for their personal use, security would get a whole lot easier. Tracking down an attacker would be a simple case of looking up the IP address in a database. The Internet would, possibly, become a much nicer place, too, as the “anonymity” assumed by many would immediately disappear. If I know your IP address, I know who you are, Mr. Internet Tough Guy, and it would be simple to call you on your rude behavior. Assigning individual IP addresses could, ostensibly, solve some of the economic issues with publication and media. Advertising rates on the web are abysmal primarily because of the lack of qualification of visitors. If a site knows who you are and what you do for a living, it can actually charge a premium for advertising of products and solutions targeted toward specific audiences if it can prove a high enough percentage of its visitors are qualified and not just random teenagers out looking for elf porn.

The technical necessity of IPv6 could drastically change a lot of behavior and markets, whether individuals were assigned IPv6 addresses or not. Given the plethora of addresses available, ISPs can assign static addresses to each customer rather than leasing out IPv4 addresses. That IPv6 address being tied to your Internet activity is priceless in its value to marketing and advertisers.

But let’s look at the good side of IPv6 for a minute. If there are infinitely more IP addresses available, the costs of obtaining one or even a few of them should certainly decrease – dramatically. IP addresses will no longer be a “precious resource” for which customers must pay a high premium to obtain. Cloud providers, too, should be able to more freely hand out IP addresses without charging (as much) for their use and can actually manage them more effectively.

Remember that sharing of IP addresses could be detrimental to the availability of applications in the cloud. And even if that’s not the case, there’s still the little issue of IP address/domain propagation across the Internet. Getting an IP address associated with a host and domain is easy. Propagating that information such that customers and users can access the application takes time, and that time varies based on the configuration of a long chain of servers across the Internet over which you, as a cloud customer, have no control.

Being able to assign “fresh” IP addresses to cloud-based applications should solve this problem. If providers have a pool of IP addresses that have effectively “timed out” of cached DNS across the Internet, they can reassign them and get around all the little niggling problems that could crop up today.

As for Mr. & Mrs. Customer, hey, there’s good for you too. If an IP address was tied to an identity, wouldn’t it be possible to get rid of login and identity verification systems? Couldn’t we just leave a comment on a blog or a post in a forum and not have to remember which of our 30 passwords goes with the site (you do have more than one password on the Internet, right?).

None of the above is likely to happen (at least any time soon). The technical challenges inherent in having an IP address follow a user around the Internet are not insurmountable, but they would put unnecessary strain on the infrastructure. The security risks of making every device a consumer might own accessible via the Internet (cause that’s one of the potential results of everything IPv6) certainly outweighs the potential that we can more easily track attackers. Routing tables are big enough, and trying to dynamically modify them because a user moved from point A to point B is just not feasible, nor are the “benefits” tangible enough to entice someone to try. It’s an interesting exercise in exploring the possibilities that’s fun but not very helpful in terms of figuring out what a move to IPv6 means for organizations.

The reality is that even though ISPs and service providers may be moving to IPv6, inside the enterprise data center it is likely to remain IPv4 for quite some time. In some cases, that’s because of the cost of replacing legacy equipment not capable of supporting IPv6 – older routers, switches, servers, operating systems, mainframes, etc… – would be prohibitive, particularly today, and without much benefit internally, where there is no lack of IP addresses stemming from the use of IPv4.

Also prohibitive internally would be the need to rewrite hundreds or thousands of security and access policies based on IP addresses and networks and subnets. Testing such a massive change alone would require hundreds of man hours and, in the process, likely render the network and the applications it delivers unavailable.

What’s going to be necessary, at least in the short term, is a way to insulate internal architectures from external changes catalyzed by a migration to IPv6. An IPv6 <—> IPv4 gateway, for example, allows organizations to retain their internal IPv4 based architectures while supporting the external move to IPv6. Such technology will be necessary for some time while providers move to IPv6 and organizations can formulate a rational, sane strategy for moving its internal architectures to IPv6 without “rebooting” their data center.

Devices that typically reside at the “edge” of the network: routers, firewalls, application delivery, WAN optimization, need to support both IPv6 and IPv4. Initially they’ll need to present IPv6 externally and support IPv4 internally, but one day that, too, will move to full IPv6 support. When you’re out looking at solutions today, then, it’s important that IPv6 support end up higher on the list than perhaps it was yesterday.

Related blogs & articles:

• Six Surprising Facts About IPv6
• Comcast Begins Move to IPv6
• We’re sorry. The IPv4 address you are trying to reach has been disconnected.
• The Days of IP-based Management are Numbered
• What ever happened to IPv5?
• IPv5 – What was it?
• Internet Spring Cleaning
• Congratulations! You are a moron

The inclusion of a web server gives attackers clear line-of-sight to their targets

There’s been a few articles on Opera Unite that have called into question the security of the decision to include a web server with the browser. Most of those discussions have centered around the ability to muck with files not intended by the host to be shared, but given current infection techniques there’s a far greater danger to Opera: mass injection attacks.

As is often pointed out, current attack techniques are not necessarily targeting web sites per se, but are intended to infect the users of such websites. Attacks like NineBall, Gumblar, and Beladen infect web sites but only as a means to create a distribution network for its user-targeted malware.

Opera’s decision to include a web server removes the middleman, as it were, and gives miscreants the opportunity to go right to the source. A source that may very well be less protected than most web sites. After all, users rarely have the security infrastructure in place to detect let alone stop such attacks, and while turning on Window’s firewall may be helpful in stopping unsolicited traffic one cannot argue that purposefully running and advertising web services on your PC via Opera’s integrated web server is soliciting traffic. You want people to access your personal machine if you’re offering services on it, which means you’re opening yourself up to a variety of potential attacks.

Both W3CSchools and Haavard web analytics place Opera’s market share at about 2.2% of users. That number is somewhat meaningless without total Internet user statistics, which we’ll pull from Nielsen via InternetWorldStats. Current estimates put the total number of Internet users at 1,596,270,108. Assuming this is at accurate, that would mean Opera is currently in use by about 35,117,942. We’ll call it 35 million to make it easier. Not every Opera user will upgrade, so let’s say half of them will upgrade to Unite, about 17 million. Let’s further assume that not all of them will actually enable the services: figure that about half of those running Unite will actually do so, about 8 million.

That’s still a target rich environment. Imagine 8 million fairly unprotected users – miscreants intended targets – running services on their machines that are begging to be attacked. But don’t worry, unless someone is a “hacker” they won’t be able to get at anything.

a spokesperson from Opera told both ZDNet and CNET, when asked if the Unite platform would offer the ability for someone to access data on a host PC that the host didn’t intend to share, “Definitely not,” the spokesperson said, “unless they’re a hacker.”

Well, that answers that, doesn’t it? I’ll give the Opera spokesperson this: s/he’s honest about it, at least.

So we have 8 million target rich environments without any real security to prevent exploitation of vulnerabilities experts say is inevitable.

“Should vulnerabilities in Opera be discovered which permit code execution, an attacker would be able to turn on the file sharing capabilities of Opera Unite and share arbitrary content. Looking at the security track record of Opera, it's not a matter of if but when such a vulnerability will be discovered,” Sutton said.

I am not feeling good about this one at all.

Given the ease with which attackers have been able to infect websites with NineBall and Gumblar and a variety of other malware-focused hacks, it would not be difficult to imagine an infection which simply gathers information about the browser and sends that information off to a bot net for further exploitation. Such an infection would be infinitely more difficult to detect, as there would be no real evidence that the information was being gathered. Infections today are noticed because users are redirected or malware is introduced into their systems and it’s noticed by someone. Simply gathering the browser agent as a means to compile a list of targets is not necessarily going to include redirection or the download of anything. And once such a list is compiled the targets can be directly attacked.

Assuming a common vulnerability is discovered in Opera Unite, the attacks could potentially then turn 8 million (assuming our calculations and statistics are correct) unprotected users into a bot net capable of, well, just about anything.

Web site vulnerabilities are discovered almost by accident these days, with miscreants creating a generic attack and then blindly throwing it out against thousands of potentially vulnerable sites and hoping one of them will stick. That’s because the would-be attackers don’t have private access to the sites and applications they are attacking. But with Opera Unite, they can and will have private access to twiddle and muck and hack until they find what they need.

Web applications are traditionally deployed in an environment with additional security solutions in place to prevent attack and infection. While web applications acting as a “middleman” are generally better protected and therefore are almost an additional layer of security against client-infection, the infections out there today suggest it’s little better than nothing. But still better than nothing. The solutions available for a user to prevent such attacks and infections don’t even really exist, and even if they did the general user should not be expected to know how to configure something like mod_security or an IPS/IDS or a web application firewall. These users are completely at the mercy of Opera to ensure the safety of their environments against what certainly appears to be an imminent exploitation of the environment.

Given company responses to security concerns and Opera’s track record, thus far that’s not a comforting thought.

Unless you’re a hacker.

Related blogs & articles:

• The Secret of the Security Safety Dance
• The IT Security Flowchart
• Why you must dive Into the Breach
• ‘Gumblar’ attacks spreading quickly
• Beladen Loads Hacked Web Sites With Badness
• Nine Ball attack strikes 40,000 Web sites

 You can’t differentiate until you do something different

Gartner analyst and cloud pundit Lydia Leong reminds us that without differentiation, all clouds look pretty much the same. 

“These are traits that it doesn’t take a genius to think of. Most are known requirements established through a decade and a half of hosting industry experience. If you want to differentiate, you need to get beyond them.” [emphasis added]

She lists traits common to most cloud providers: premium equipment, VMWare-based, private VLANs, private connectivity, and co-located dedicated gear but doesn’t really get into what really is – or should be – the focus of cloud offerings: services. To be more specific, infrastructure services.

A cloud provider of course wants a solid, reliable infrastructure. That’s why they tend to use the same set of “premium” equipment. But as Lydia points out differentiation requires services above and beyond simple hosting of applications in somebody else’s backyard.

Cloud providers need to differentiate based on what kinds of infrastructure services they can provide including but certainly not limited to management. Going above and beyond the basics, however, requires investment and some roll-up-your-sleeves coding at this point because the management and process automation systems simply don’t exist. The means to build those systems do exist – that’s part of the allure and value of dynamic infrastructure, a.k.a. Infrastructure 2.0.

Using the tools available it is wholly possible to build out the “self-service” for infrastructure services that folks have been led to believe will one day, hopefully, maybe be part and parcel of a cloud computing offering. Building this out is not trivial, but it’s certainly worth it if you’re trying to (a) differentiate from your competitors and (b) find yet another revenue stream on which you can ride smoothly into the next round of technology advances.

Lydia mentions that most cloud providers are built atop premium equipment and cites Cisco networks and F5 application delivery infrastructure among others. There are more than enough “services” that can be abstracted via both companies’ APIs to keep cloud computing providers differentiating their hearts out. There’s presence and location, application acceleration, application security, commonly used web application functions like rewriting URIs and customized “apology” pages via network-side scripting just to name a few. There’s a veritable cornucopia of specialized functionality above and beyond simple network and application network functionality that simply isn’t being exploited to its full potential, both in terms of the technological capabilities and the ability to differentiate for a cloud computing provider.

There’s no reason why such differentiation of cloud providers cannot be achieved through infrastructure services and, in fact, its about the only way to differentiate aside from better management/control over design and maintenance of the deployment. The rest is simply price wars – competing based on cost per mega or kilo or peda or tera byte of something. The problem may very well be that we’re just too early in the game for such large-scale differentiation; that not enough customers are even sure what to do with the basics let alone advanced infrastructure service offerings. Problem is that many times customers won’t sign on to anything – a service or a product – without the promise that they can do more, either right now or in the future. Whether they want to – or will use those services – is irrelevant. The fact that they exist tells the customer that the provider is serious, dedicated, and ready to do business.

It’s kind of a Catch-22. Without the demand there’s no reason for a provider to invest in such an undertaking, but without investing in such an undertaking there may well continue to be a lack of demand. Remember that in many organizations data center infrastructure is the underlying foundation for their competitive advantage. If you can’t offer tit-for-tat you’re simply not a good replacement no matter how much “cheaper” you may be in the short term or long run.

Differentiation of cloud services will eventually come down to management and infrastructure services because there’s really no where else for it to go. What and how those services are packaged and offered to customers will go a long way toward cloud providers differentiating amongst their offerings.

Related blogs & articles:

• “Enterprise class” cloud
• The Infrastructure 2.0 Trifecta
• The Revolution Continues: Let Them Eat Cloud
• Cloud computing is not Burger King. You can’t have it your way. Yet.
• Control, choice, and cost: The Conflict in the Cloud
• Virtualization: How to Isolate Application Traffic

One of the tasks of an enterprise architect is to design a framework atop which developers can implement and deploy applications consistently and easily. The consistency is important for internal business continuity and reuse; common objects, operations, and processes can be reused across applications to make development and integration with other applications and systems easier. Architects also often decide where functionality resides and design the base application infrastructure framework. Application server, identity management, messaging, and integration are all often a part of such architecture designs.

Rarely does the architect concern him/herself with the network infrastructure, as that is the purview of “that group”; the “you know who I’m talking about” group. And for the most part there’s no need for architects to concern themselves with network-oriented architecture. Applications should not need to know on which VLAN they will be deployed or what their default gateway might be. But what architects might need to know – and probably should know – is whether the network infrastructure supports “server offload” of some application functions or not, and how that can benefit their enterprise architecture and the applications which will be deployed atop it. 

Server offload

You can think of server offload like a relay race. The application network infrastructure device runs the first leg and then hands off the baton (the request) to the server. When the server is finished, the application network infrastructure device gets to run another leg, and then the race is done as the response is sent back to the client.

There are basically two kinds of server offload functionality:

1. Protocol processing offload
   Protocol processing offload includes functions like SSL termination and TCP optimizations. Rather than enable SSL communication on the web/application server, it can be “offloaded” to an application network infrastructure device and shared across all applications requiring secured communications. Offloading SSL to an application network infrastructure device improves application performance because the device is generally optimized to handle the complex calculations involved in encryption and decryption of secured data and web/application servers are not.
   
   TCP optimization is a little different. We say TCP session management is “offloaded” to the server but that’s really not what happens as obviously TCP connections are still opened, closed, and managed on the server as well. Offloading TCP session management means that the application network infrastructure is managing the connections between itself and the server in such a way as to reduce the total number of connections needed without impacting the capacity of the application. This is more commonly referred to as TCP multiplexing and it “offloads” the overhead of TCP connection management from the web/application server to the application network infrastructure device by effectively giving up control over those connections. By allowing an application network infrastructure device to decide how many connections to maintain and which ones to use to communicate with the server, it can manage thousands of client-side connections using merely hundreds of server-side connections. Reducing the overhead associated with opening and closing TCP sockets on the web/application server improves application performance and actually increases the user capacity of servers. TCP offload is beneficial to all TCP-based applications, but is particularly beneficial for Web 2.0 applications making use of AJAX and other near real-time technologies that maintain one or more connections to the server for its functionality.
   
   Protocol processing offload does not require any modifications to the applications.
   
2. Application-oriented offload
   Application-oriented offload includes the ability to implement shared services on an application network infrastructure device. This is often accomplished via a network-side scripting capability, but some functionality has become so commonplace that it is now built into the core features available on application network infrastructure solutions.
   
   Application-oriented offload can include functions like cookie encryption/decryption, compression, caching, URI rewriting, HTTP redirection, DLP (Data Leak Prevention), selective data encryption, application security functionality, and data transformation. When network-side scripting is available, virtually any kind of pre or post-processing can be offloaded to the application network infrastructure and thereafter shared with all applications.
   
   Application-oriented offload works because the application network infrastructure solution is mediating between the client and the server and it has the ability to inspect and manipulate the application data.
   
   The benefits of application-oriented offload are that the services implemented can be shared across multiple applications and in many cases the functionality removes the need for the web/application server to handle a specific request. For example, HTTP redirection can be fully accomplished on the application network infrastructure device. HTTP redirection is often used as a means to handle application upgrades, commonly mistyped URIs, or as part of the application logic when certain conditions are met.
   
   Application security offload usually falls into this category because it is application – or at least application data – specific. Application security offload can include scanning URIs and data for malicious content, validating the existence of specific cookies/data required for the application, etc… This kind of offload improves server efficiency and performance but a bigger benefit is consistent, shared security across all applications for which the service is enabled.
   
   Some application-oriented offload can require modification to the application, so it is important to design such features into the application architecture before development and deployment. While it is certainly possible to add such functionality into the architecture after deployment, it is always easier to do so at the beginning.
   

Server offload is a way to increase the efficiency of servers and improve application performance and security. Server offload increases efficiency of servers by alleviating the need for the web/application server to consume resources performing tasks that can be performed more efficiently on an application network infrastructure solution. The two best examples of this are SSL encryption/decryption and compression. Both are CPU intense operations that can consume 20-40% of a web/application server’s resources. By offloading these functions to an application network infrastructure solution, servers “reclaim” those resources and can use them instead to execute application logic, serve more users, handle more requests, and do so faster.

Server offload improves application performance by allowing the web/application server to concentrate on what it is designed to do: serve applications and putting the onus for performing ancillary functions on a platform that is more optimized to handle those functions. Server offload provides these benefits whether you have a traditional client-server architecture or have moved (or are moving) toward a virtualized infrastructure. Applications deployed on virtual servers still use TCP connections and SSL and run applications and therefore will benefit the same as those deployed on traditional servers.

Related blogs & articles:

• I am wondering why not all websites enabling this great feature GZIP?
• 3 Really good reasons you should use TCP multiplexing
• SOA & Web 2.0: The Connection Management Challenge
• Understanding network-side scripting
• I am in your HTTP headers, attacking your application
• Infrastructure 2.0: As a matter of fact that isn't what it means

Two steps forward, three steps back

Every time there is a major shift in technology thought about architecture the question of how it will and should impact infrastructure arises. When SOA was the “next great thing” there was a spate of announcements regarding how infrastructure would not only support it but integrate into its ecosystem. This time it’s virtualization, and its impact on infrastructure both from a support standpoint and usage is getting a lot of mindshare. In a recent announcement around virtual network infrastructure Om Malik of GigaOm has some interesting commentary:

As we’ve outlined in the past, virtualization is going to force networking vendors to change. There were two scenarios that we outlined; one was that networking vendors could shift their business and sell to cloud operators — Cisco is doing that with its unified computing efforts.

The other option was for routers, load balancers and firewalls to run inside the “virtual appliances.” Some believe that the routing, blocking and distribution of traffic can be done well in a “cloud environment.”

It is the second option that keeps cropping up as more and more attention is brought to bear on the importance of infrastructure to emerging data center models. The problem is that it isn’t as easy as it first sounds to just pack up the software that powers hardware-based network infrastructure and replant it inside a virtual container.  There are both advantages and disadvantages to such an approach, and it’s important to examine how virtualization might affect the capabilities of infrastructure when moved to a virtualized environment before blindly accepting it as an option. 

Back in the old days of SSL now defunct companies like Rainbow and nCipher sold SSL acceleration cards. These cards were PCI-based and slipped into just about any server hardware you had. The trick was to get the software – primarily web and application servers – to be able to take advantage of the hardware accelerated functions for bulk encryption and SSL handshaking. This was accomplished via what was called “drivers” but in reality this was a misnomer. They weren’t really drivers as much as they were plug-ins that essentially replaced the libraries used by the web/application servers to provide the bulk encryption/decryption and SSL negotiation functions. These plug-ins were provided by the SSL acceleration vendor, not the software vendor, and thus supported a very limited set of web and application servers.

Over time the effort and cost of managing SSL acceleration cards in every server in a data center grew too unwieldy and was moved to an “off box” solution. Imagine trying to maintain a data center with even a hundred servers, each with an SSL accelerator card and each requiring upgrades, patches, and management and you’ll see why folks moved to off-box solutions. These “off box” solutions eventually merged with load balancers and SSL accelerated functions are now offered by every vendor offering a load balancer or application delivery controller solution.

So why couldn’t we just go back to the old PCI-based server solution? We could, if there was a way to “plug-in” that functionality to web and application servers – and every other piece of software out there that needed it. Now Microsoft’s CryptoAPI is often used by software for such tasks, replacing its core encryption and SSL functionality with one that took advantage of acceleration hardware – when it was present – would provide a nifty, neat and simple solution. But that’s only Microsoft-based servers; that leaves quite a number of solutions running on Linux and Unix out in the cold. And of course it returns us to the days of decentralized certificate management and all the additional burdens that a virtualized infrastructure brings to that mess including an increase in operational expenses associated with managing decentralized certificates.

Too, there is the issue that there’s never been a server solution for hardware accelerated compression. That’s not to say it can’t be done, but someone would need to provide that one if there was a call for it. Granted, SSL and bulk encryption/decryption is far more widely utilized and therefore likely the most important piece to provide first, but compression should certainly be considered as a second in line as the improvements in performance and resource savings from utilizing hardware assisted compression are noticeable enough to make a difference.

But until such solutions appear, moving network infrastructure to “virtual appliances” effectively wipes out the benefits of hardware-assisted acceleration of those functions, which means the equivalent solution in a virtual appliance is necessarily going to suck up more server resources which is going to translate into needing, well, more server resources. The average resource savings per server from offloading SSL to a hardware assisted device is 30%. By eliminating that savings you effectively need 30% more resources on a virtual network solution as compared to a network device. That’s likely going to result in an increase in the number of instances you need to deploy a similar network infrastructure in a fully virtualized architecture.

Now, luckily virtualization providers recognize the need for some solutions to bypass their network stack – for control, for performance, for whatever – and have the means by which third-party developers of software can route around the virtualization solution’s stack and use their own. In some cases the developers can go straight to the hardware in question without needing to implement what is essentially a “plug-in” replacement of the virtualization solution’s stack.

But this takes time, and it isn’t completely a panacea as there’s still the little issue of limited throughput (you’re reducing something originally meant to handle many ports to something with one or two or five) and the complications of trying to route through virtual switching technology. It can also seriously impact capacity as you’re now running in a virtual environment that’s likely more limited in terms of CPU and resources than what that infrastructure was used to having at its disposal in its native hardware environment.

So vendors either take the time to do it – assuming they can - or they don’t and in the process lose a lot of hard-won advances in technology regardless.

At the end of the day (as long as I’m used tired, worn-out phrases) there are definitely advantages to a virtual network infrastructure. Scalability becomes a matter of just firing up new instances, just like applications, though it is important to remember that scalability is not infinite. There are limitations based on your budget as well as physical limitations imposed by the size of the provider’s data center. Still, scalability is certainly perceived as being easier with a virtual network infrastructure as long as there is physical capacity, and it is unlikely a provider, at least, will run out of physical capacity. Automation of scalability of the infrastructure, too, would provide benefits not available with a physical deployment as the latter requires a physical device be racked and powered on while the former is more likely able to simply be “powered up” on an existing server. Control and management are infinitely easier from both the provider and the customer’s point of view with a virtual appliance model as it is completely left to the customer to configure and manage, and multi-tenant management in network infrastructure is still maturing.

What you’re most likely losing is reliability of the hardware as well as capacity and performance improvements from a flatter architecture and the inherent gains available from specialized hardware. Availability, too, may be impacted. As noted recently by analyst John Abbot of the 451 Group in his report, “Break Point – Virtualization and Availability” there is a danger in starting a virtualization project without an availability strategy because the likelihood of downtime actually increases with virtualization because more work relies on fewer (physical) servers.

The question of what is gained by a move to virtual network infrastructure is also rarely addressed. Network architects are certainly going to be loathe to deploy critical network infrastructure in a virtual machine that is shared by…anything else. A dedicated server would likely be required, which then begs the question why one would swap a dedicated piece of hardware for a dedicated piece of hardware. Sure, the possibilities of sharing resources sound good, but when we’re talking critical network infrastructure that’s something unlikely to happen unless it’s absolutely necessary.

So while core network infrastructure services certainly can be made available and deployed as a virtual infrastructure, the question is whether the benefits balance out the negatives or not. Comparisons of performance and capacity between hardware and software solutions already provide some baseline for understanding just what those negatives are and whether the costs for software to match hardware in a virtual infrastructure would end up justifying the investment in a hardware infrastructure or not. The equation is likely highly dependent upon whether you’re a cloud provider or an organization looking to implement an on-premise cloud architecture, as the former has little impetus to reduce the number of instances required for your environment to run in the cloud (more instances means more revenue, after all) or invest the time to leverage existing capabilities to provide better control for customers over the hardware-based infrastructure.

Related blogs & articles:

• The Infrastructure 2.0 Trifecta
• Hardware Acceleration Critical Component for Cost-Conscious Data Centers
• I am wondering why not all websites enabling this great feature GZIP?
• Optimize WAN and LAN Application Performance with TCP Express [white paper, PDF]
• Not All Virtual Servers are Created Equal
• Cloud outages don’t bother Stanley
• Do you control your application network stack? You should.

How to optimize compute resources in a heterogeneous environment using weight/ratio-based load balancing

Unless you’re starting from scratch your data center is full of physical servers of various and sundry sizes, colors, shapes, and compute resources. And even if you’re starting from scratch and you have beautiful racks of everything the same, it’s not likely to stay that way if for no other reason than, well, hardware moves on at an astonishing rate these days. So you’ve almost certainly got (or will have) a physically heterogeneous environment in terms of hardware compute resources.

When you’re scaling up servers – whether solely to assure availability or for capacity – you will end up with instances running on different servers. Or at least you’d better if availability is part of the equation. Now, in a traditional environment that would cause potential issues as one of the hallmarks of a highly available architecture is that if the primary server fails the secondary must be able to handle the load. All of it. In a virtualized environment that’s not necessarily the case as you may be able to simply bring up two or three instances with less capacity to meet demand if you have the physical resources available.

Here’s the catch: your infrastructure needs to understand the capacity of each server (physical or virtual) in order to maximize resources available. Specifically, the load balancing solution – whether a traditional “load balancer” or part of an application delivery controller – must be able to distribute requests based on what resources are available on any given instance. That means if an instance is running on a physical server with fewer total resources available than another instance, the instance with fewer resources should be used less frequently.

It is the fact that data centers are heterogeneous and comprised of myriad physical servers of varying capacity that makes it important for the folks architecting cloud environments to understand what’s going on “under the hood.” 

Not all servers are created equal, but if you’re consolidating and trying to eek out every last drop of CPU and RAM from your physical hardware to reduce capital expenditures you might need to get more creative with how you’re distributing your application load.

Using weighted or ratio-based load balancing in a heterogeneous environment offers the convenience and simplicity of traditional, simple load balancing algorithms with an eye toward balancing the differences inherent in physical hardware. Or, potentially in emerging data center models, the differences in virtual instances. Because the limitation on physical hardware necessitates limitations on virtual instances, particularly if there are ore than one instances of a virtual container running on the same hardware, it’s important for the solution that distributes request amongst application instances to understand that some have more headroom than others, as it were.

In load balancing there are a few “industry standard” old standby algorithms. These algorithms are often also implemented by application server clustering solutions, and should be fairly familiar to network and application folks alike: round robin, least connections, and fastest response time.

In most load balancers there is also the addition of a ratio-based algorithm in which “weights” are assigned to each member of a (pool|farm|cluster). Requests are distributed to each member based on that weight, which is really used more like a percentage ratio than anything else.

Using a ratio-based load balancing algorithms in a virtual or cloud environment in which the hardware and/or virtual containers may have different resource ceilings affords architects the ability to better distribute requests according to the capacity and health of each instance. Without taking into consideration these physical limitations it is easy to overwhelm one system while leaving another idle, which runs contrary to the concept behind cloud computing and on-demand data centers in which every ounce of compute resources is used in order to meet capacity.

Obviously there are other (and perhaps better) ways to make decisions on distributing requests when the environment comprises instances of varying capacities and resources. A ratio-based load balancing algorithm is one of the simplest and easiest to implement, but affords a great deal better use of resources than does an algorithm that does not take the physical constraints of disparate systems into consideration.

Related blogs & articles:

• Intro to Load Balancing for Developers – The Algorithms
• Dynamic Infrastructure: The Cloud within the Cloud
• Infrastructure 2.0: The Feedback Loop Must Include Applications
• Architects Need to Better Leverage Virtualization

I’m heading out today for a little time off and so you’ll have to make due the rest of the week without any (new) words of wisdom from me. I know, try to pull yourself together. You’ll live, really, and I’ll be back Monday with something interesting, promise.

While I’m out, you might consider checking out some of the blogs I follow myself on a regular basis. They’re always full of interesting tidbits and stories and wisdom on a variety of subjects, and if you don’t follow them yourself you might find something interesting in them.

So happy reading, and have a great weekend when you get to it!

The Virtual DC

Our very own Alan Murphy keeps you on your toes with everything virtualization. He’s also a bit of a security nut. Okay, he’s a lot a security nut, and if it’s security and virtualization, well…Alan is my go-to-guy when I have technical questions about the two worlds colliding.

Pete Silva

Our very own Pete Silva chimes in on security and compliance. Pete’s entertaining – he’s got a style all his own and a voice made for podcasting/audio. He’s the voice behind our audio whitepapers, if you were wondering.

Ken Salchow

Hey, he’s my boss, for crying out loud. Gots to have his blog in here. His perspective is unique having been an engineer once that’s now, as he would say, a pointy haired boss. He’s not, but he likes to use the excuse, methinks.

Don MacVittie

My other half. Don won’t be posting either (I’m taking him with me) but he has a great series on load balancing for developers and his opinions on storage and file virtualization are top notch.

Colin Walker

If you need to know anything – and I do mean anything – about iRules then Colin is your man. This guys speaks fluent iRule, and I think he’s using iRule commands to train his new puppy. Seriously.

BitCurrent

Some old (and new, too) friends over at BitCurrent keep you, well, current on application performance, cloud computing, and a variety of WebOps issues.

CodingHorror

What can one say but “Jeff Atwood”. Programming and coding with an attitude. Always informative, usually irreverent and entertaining.

DataCenter Knowledge

Rich Miller keeps me up to date on everything data center. And I mean everything. He just never stops with the info.

Infrastructure 2.0

Hey, if you’re interested in emerging data center models, cloud, dynamic infrastructure, and the like then you’ve got to keep this one in your reader. It’s not just one guy, it’s several from across the infrastructure industry including InfoBlox, Cisco, VMWare, and (shameless plug), F5.

Rational Survivability

Who can ignore Hoff? Even now that he works for the Empire (Cisco) he’s still an entertaining and informative read that will certainly keep you on your toes in matters related to cloud and security – and sometimes other random bits.

InfoSec Ramblings

Kevin Riggins keeps me up to date every week with his “Interesting Information Security Bits”. Succinct, digestible, and important happenings in the information security community.

Layer 8

I just realized I don’t know @shrdlu’s real name. Hunh. In any case, he writes some very interesting, thought provoking stuff. He’s also got an attitude about security, so that makes him an interesting read. And if you get a chance, he loves to be insulted on #rudethursday on Twitter. So go ahead, follow him, and tweak his nose tomorrow. Tell him @lmacvittie sent you.

I could keep going. Seriously. The list in my feed reader is really too long to enumerate here (I do read the whole Internet every day) but hopefully if you haven’t read any of these in the past you’ll find something interesting to keep you going while you pine away for me to write something new (you are pining away, right? RIGHT?)

Have a good one!

An interesting thing happened on the way to testing that application from the cloud. We broke the innertubes!

Pros and Cons of Application Testing in the Cloud

A firm wanted to test their application and need 100 browser instances. In the old days it would have required 100 machines -- that would be a massive undertaking. Even with hardware virtualization, you would need 5 to 10 machines, and there would be some complex configuration issues. However, by putting it all in the cloud, they were able to sync up 100 virtual instances of the browsers and take them down over a weekend at a dramatic cost reduction.

I won’t argue the point about needing one machine per browser instance for application testing (even though it’s not entirely accurate as plenty of tools support such requirements without requiring multiple machines – Ixia and Spirent to name just a couple of my favorites) because there’s so much more that needs to be said in terms of infrastructure and this kind of test environment.

The truth is that when you start spinning up multiple instances of any client for the purposes of testing an application you can inadvertently doom your test to fail due to underlying infrastructure attributes that aren’t always all that obvious to the uninitiated regardless of whether those clients are in the cloud or not.

Back in the days of AOL and Compuserve we had what the industry termed the “megaproxy”. The megaproxy caused all sorts of issues for web sites and web infrastructure because large blocks of visitors coming from one of the large providers all appeared to be accessing sites from the same IP address. The specific problem of load balancing, though largely a non-issue today, was solved primarily by moving from layer 4 source IP-based persistence to cookie-based layer 7 persistence.

It turns out that cookie-based persistence, however, is something often overlooked by neophyte load balancing administrators when scaling out an application for the first time. Combine this with a cloud-based client scenario, which essentially reintroduces the mega-proxy problem, and we’re right back in the early days of applications not working as expected due to configuration and infrastructure issues. Unless the cloud provider has an unlimited source of public IP addresses, they’re going to have to NAT (masquerade) all of the clients in the cloud as just a few IP addresses. That means the tested application – and the underlying infrastructure – is going to suddenly see a lot of requests coming from the same IP addresses.

This is where things get wonky.

There are a couple issues that can arise from this scenario, the most obvious being the problem with load balancing and persistence. If you’re only using Layer 4 persistence based on IP address, the load is going to be distributed very unevenly across the servers (virtual or physical) responsible for scaling your site. That means your testing is going to be useless for determining capacity and even average end-user performance, as more heavily loaded servers inherently degrade application performance.

A second, more well-hidden surprise for network administrators can arise if internal network infrastructure (that’s switches and routers, remember) is also doing any load balancing via port trunking. This is because the load balancing across a set of trunked ports is almost always based on a hash algorithm that relies on MAC or IP address, which in the aforementioned scenario will lead to traffic being routed over only a few – or one – port. If that port is unable to handle all the traffic it will quickly become oversubscribed and degrade performance due to the congestion, retransmissions, and potentially QoS policies that kick in.

There often exist security policies that dictate how many requests per second/minute/hour can be made by any single client in an attempt to thwart DoS (Denial of Service) attacks as well as to ensure the quality of the service for all users by not allowing a single client to “hog” the application. This obviously can have a very negative affect on the ability to stress/load test an application as the number of requests that can come from the client will be limited by said policies. Firewalls, too, may put artificial limits on connections and bandwidth based on any number of variables: time of day, location, ports, etc… and policies in its infrastructure may be triggered during a test.

Basically the most important thing to remember is that if you’re using a cloud-based testing service or a cloud-based testing environment that you’re probably testing a production environment and its underlying infrastructure.

Let me repeat that one because it’s the most important thing I’ve said in this post: you are not just testing the application, you are testing its infrastructure, too. And remember that when that traffic is generated externally that you’re also poking at the infrastructure of the Internet, and may well trigger policies “out there” that your ISP – or others – have set in place to protect your applications and network.

It’s therefore important to understand what policies may be triggered by sudden bursts of traffic as well as what security policies may be triggered by the constant hammering of a single client at an application. It’s also important to understand the flow of requests and responses through the infrastructure and recognize the various touch points along that path. Doing so may ease the troubleshooting process greatly.

Testing applications in a lab or even a QA environment has its own pitfalls as you are testing only the application and as we all know, no application is an island. But testing them in a production environment, while ideal for gathering a real view of how the application will perform when faced with users and customers, has its own challenges because you’re testing against a real, live environment with all the mechanisms in place to ensure quality, performance, and security of those applications.

Gather up the right folks before you start testing and understand the potential pitfalls and how to properly address them – or avoid them – before you spend money firing up hundreds of instances of clients in the cloud or you may find yourself wasting more money than you might have just testing them on your own.

Related articles & blogs:

• Tony Bourke: Mega Proxy Not So Mega, Akshually
• Cloud Computing: Is Your Cloud Sticky? It Should Be
• This network ain’t big enough for the both of us
• OVF: A Few Layers Short of a Full Stack
• Five Key Questions Developers Need to Ask Before Starting the Troubleshooting Process
• Load Testing as a Service: A Look at Load Impact (beta)