The W3C specification now offers the means by which cross-origin AJAX requests can be achieved. Leveraging network and application network services in conjunction with application-specific logic improves security of allowing cross-domain requests and has some hidden efficiency benefits, too.

The latest version of the W3C working draft on “Cross-Origin Resource Sharing” lays out the means by which a developer can use XMLHTTPRequest (in Firefox) or XDomainRequest (in IE8) to make cross-site requests. As is often the case, the solution is implemented by extending HTTP headers, which makes the specification completely backwards and cross-platform compatible even if the client-side implementation is not. While this sounds like a good thing, forcing changes to HTTP headers is often thought to require changes to the application. In many cases, that’s absolutely true. But there is another option: network-side scripting. There are several benefits to using network-side scripting to implement this capability, but more importantly the use of a mediating system (proxy) enables the ability to include more granular security than is currently offered by the Cross-Domain Request specification.

The basic premise of the W3C specification is that the server, i.e. the application, controls whether a remote location can send cross-site requests. Even though the request is actually made by the user (and we’ll get to how we might leverage that dynamically a bit later) it is the originating application – more specifically the domain - that must be authorized. In its simplest form, when a cross-origin request is received the application must respond with an Access-Control-Allow-Origin HTTP header containing a URI (or wildcard) that matches the value of the “Origin” HTTP Header. The browser then determines if the values of the two HTTP headers match and, if they do, the request is allowed to continue. If the value of the Access-Control-Allow-Origin does not exactly match the value of the Origin header – or does not contain a wildcard – the browser refuses to honor the response. The wildcard cannot be used in place of the host in a FQDN (Fully Qualified Domain Name), e.g. *.example.com, it can only be used to allow all domains to access the resource. While this is certainly the easiest way to enable cross-domain requests to be successful, it is not recommended because it is essentially the same as providing no security for the invocation of functionality from third-party domains.

Other variants of this scenario allow for the use of other HTTP methods (POST, DELETE, PUT) but require a pre-flight request to determine whether the method is allowed or not. The availability of these methods certainly forwards the REST API model as well as SOAP and makes it possible to develop web-based applications that can interact with multiple domains – think cross-cloud deployments – at the same time.

Of course the need to (a) determine access control based on the Origin header and (b) add the Access-Control-Allow-Origin header requires that the application must change its behavior, i.e. code to support cross-domain requests. But as is the case with many forms of authentication and web application security, this process can be relatively easily implemented in a network-side scripting capable load balancer or intermediary, such as a mod_headers/mod_rewrite enabled Apache-based proxy. Regardless of whether you use a proxy capable of inspecting and transforming requests and responses or a Load balancer similarly enabled the process should be the same.

The advantages of using an intermediary are that you don’t need to change existing applications to support this functionality, and a centralized proxy-based solution can provide the functionality for all applications at the same time. A second benefit of this architecture over tightly-coupling with the application is that the application doesn’t need to process requests that are not allowed. If the proxy-based solution determines the request is not legitimate or authorized, the server never sees the request. This means the application – and by extension the server – don’t waste resources processing requests that are unauthorized, which improves the capacity of the server/application to service legitimate users.

A final advantage of this solution is flexibility. While the specification calls for determining authority to access a resource based solely on the origin, this can easily be extended to include other factors if the intermediary platform is capable of doing so.

For example, if you’re using a network-side scripting capable application delivery controller that is able to leverage GeoLocation information, you can use that information to determine authorization as well as the origin. You can use cookies, other HTTP headers, network information, time of day, and of course any data that might be submitted with the request – just about any data from the network up to the application can be included in the determination of whether the request should be granted or not.

What the W3C provides is a framework for enabling the execution of cross-domain requests, but does not specifically detail how to determine whether any given request should be allowed or not. That is completely up to you. It is assumed by many that the only method of determining access rights is to keep a list of domains allowed to access resources via a cross-site request. This is simply not true. The use of additional HTTP headers as a means to allow or deny access makes the process dynamic and it is up to the developer to determine how access rights are derived. While the simplest case certainly uses nothing more than a list of domains, there are plenty of other ways in which access rights can be derived given an HTTP request. A context-aware application delivery controller used as the means by which such determinations are made can dramatically broaden the type of information upon which you can base that decision. You can extend the application’s reach, essentially, into the network. If you still want the application itself to authorize the request, you could still use network-side scripting to simply “screen” requests to determine whether they pass certain checks before being forwarded onto the server.

For example, the application delivery controller can inspect a request not only for the existence of the Origin header, but also the User-Agent. Based on the User-Agent the network-side script may be instructed to reject the request outright rather than allow it to be processed by the server. This may be desirable to prevent spiders and scripts – assuming they are capable of sending the Origin header - from accessing resources. It may be desirable to check for capabilities or language support first before forwarding on the request, to ensure the server-side application properly supports the request. This is also the best place to implement request throttling behavior, too, to ensure one client – or domain – does not consume resources at a rate that would degrade availability or performance for other clients – or domains.

This is “application delivery” as it applies to application architecture: the ability to leverage “network” and “application network” services to extend the reach and visibility of applications further to provide additional security and options that were heretofore unavailable due to inherent limitations in the application architecture. 

Related blogs & articles:

• Microsoft: AJAX – Introducing Cross-domain Request (XDR)
• Mozilla: HTTP access Control – MDC
• W3C Cross-Origin Resource Sharing
• Rockstarapps.com: Unleash the Browser with Cross-Domain Requests – eh not yet
• API Request Throttling: A Better Option
• Using Network-Side Scripting to Implement Mock API Endpoints
• Twitter's API limit: Static control in a dynamic world
• AJAX and Network-Side Scripting
• Impact of Load Balancing on SOAPy and RESTful Applications
• Understanding network-side scripting
 

Technorati Tags: MacVittie,F5,application security,security,web 2.0,network-side scripting,W3C,cross-domain request,HTTP,application delivery,context-aware,development,ajax,XMLHTTPRequest

Scaling applications that include AJAX and non-AJAX components may require more than just tuning your web server 

A common problem after deploying a Web 2.0 AJAX-based application shows itself through poor performance or lower capacity on the server, often both. Web serving tuning is almost always the first step in improving performance and capacity, but the inherently competing behavior of AJAX-requests and “normal” HTTP requests quickly becomes problematic as well. Tune for the AJAX requests and performance of regular old HTTP requests suffers. Tune for regular old HTTP requests, and performance of AJAX-requests suffer.

This is primarily because of the way in which the client-side application, the browser, interacts with the server. “Regular old HTTP requests” are typically those that GET a piece of content, static or dynamic, and that’s it. There may be many of these requests whenever a page (URI) is requested – all the images, client-side scripting files, style sheets, etc… – but they are not interactive. The browser requests them, receives them, and that’s it. AJAX-based requests, however, are inherently interactive. They are often automatically refreshed on an ongoing basis, on a prescheduled interval, or invoked by the user as they interact with the application. These requests are not “load and forget” like their traditional staticesque counterparts, but rather they are expected to be made often.

The overhead associated with opening and closing connections is well understood, and it is often the case that the web server configuration will  be adjusted to meet the more demanding nature of the AJAX-based requests in an application. This is often accomplished by ensuring the KeepAlive setting (in Apache) is “on” and that the KeepAliveTimeout (in Apache) is high enough that AJAX-based requests occur before the timeout closes the connection. This allows the continued reuse of an existing connection between the browser and the server and improves performance. But it also ties up resources on the server keeping that connection open, which reduces the overall capacity of the server in terms of its ability to serve users. Optimally a short KeepAliveTimeout, if any, is best for non-interactive requests and often disabling KeepAlive actually improves performance for non-interactive applications.

Obviously these two behaviors are completely at odds with one another.

There are a number of ways in which the competing needs and interests of the interactive (AJAX) and non-interactive portions of your web application can be addressed.

1. Configure two different servers: one to serve interactive content, i.e. AJAX-based requests, and one to serve non-interactive requests, i.e. everything else. This way, each server can be specifically tuned (and sized) according to the application behavior. This is beneficial for several reasons, including the ability to “scale out/up” only the interactive-serving functions when or if it becomes necessary. This can be achieved simply by using specific host names for specific requests. If you do not have a public IP address that can be assigned to each host, however, you’ll need a proxy, like a Load balancer, to sit in front of the servers and handle the direction of requests appropriately or you could use mod_rewrite to achieve a similar architecture. When a mediating solution like a load balancer is used to implement this solution, there are a several ways to achieve the behavior. One method is to rewrite requests directed at a specific URI, for example: http://www.example.com/ajax/request1.php would be redirected to the server designated as the “interactive” server while other requests would be forwarded to the non-interactive server. An application aware load balancer, i.e. application delivery controller, can examine the request itself and base the same decision on the actual data being exchanged. For example, many AJAX frameworks (XAJAX, SAJAX, Prototype, etc…) often use the HTTP POST method to send a request and use specific parameters such as “xjxfun” to indicate which function is being invoked on the server side. By examining the data being exchanged an application aware proxy (load balancer) can use that information to send the request to the appropriate server.

2. A second means of addressing the problem of resource depletion and performance with AJAX-based applications is to use a load balancing solution to mediate for the clients and employ the use of TCP multiplexing on the server-side to optimize resources. Because a load balancer is almost certainly capable of simultaneously handling a significantly higher volume of connections than a single web server, the competing behavior of interactive and non-interactive HTTP requests in a web application do not impede performance or impact its capacity. By allowing a load balancer to mediate for those requests, it can better manage the resources on the server and ensure that both capacity and performance are maintained. For every X client connections, the load balancer maintains only a fraction of X connections to the server and reuses them as the means to optimizing the server-side resources. This method is actually likely to increase overall capacity because it will reduce the number of connections required to be in use at any given time on the server(s) and eliminates the performance overhead associated with opening and closing TCP connections.

3. A third solution might be found in scaling up (beefier hardware) and leveraging virtualization. For web applications, specifically, it may be the case that virtualization of the application will actually improve performance. This is particularly true of I/O intensive web applications, but is also likely true of high-connection oriented applications as well. This is  because as a web server begins to reach its capacity in terms of connections it requires more processing to “find” a given connection. Nearly all TCP-based applications exhibit similar performance characteristics and, upon reaching a certain threshold of connections, performance degrades. By finding the “sweet spot” ,i.e. the highest number of connections that retains acceptable user response time, and deploying multiple instances of that application, each tuned for that upper bound, it may be possible to squeeze out better performance and higher capacity of your web applications. Multiple instances will require a proxy, i.e. load balancing, solution as well, but this would allow for a “scale up” solution that takes advantage of a single, beefy physical server that eliminates the IT management and maintenance overhead of additional hardware in the data center.

In all three cases the solution to the problem of competing resource utilization between interactive and non-interactive components of a web application involve architecture. Some might believe that simply moving the application to “the cloud” would address the problems and, in some ways, it will. Cloud computing environments can indeed be managed such that applications are automatically scaled out to maintain performance and increase capacity, but the interesting thing about that is the environments are essentially implementing a combination of the three solutions heretofore presented. The bad news is that such a solution does not optimize resource utilization, and thus the costs associated with a cloud computing solution to the problem may be surprising and even prohibitive depending on your IT budget. And the cloud computing solution, of course, is ultimately also about architecture, as it is the architecture that allows for automated scalability.

In most cases involving web applications the answer to scalability challenges is going to end up being architecture, and that architecture is increasingly requiring the use of application network components such as load balancers to implement. This is why it is often advised that applications are architected to take advantage of application networking components from the beginning, even if such solutions will not be necessary to address capacity and optimization on day one. By architecting a solution that includes application networking as part of its design and deployment model, there is far less disruption later when such a solution does become necessary.

Related blogs & articles:

• Improving PHP Performance on Apache
• XAJAX Perfect Choice to Build Scalable Web Applications for Cloud Computing
• How to Gracefully Degrade Web 2.0 Applications To Maintain Availability
• WILS: How can a load balancer keep a single server site available?
• Long Live(d) AJAX
• How To Use CoralCDN On-Demand to Keep Your Site Available. For Free.
• JSON versus XML: Your Choice Matters More Than You Think
• AJAX and Network-Side Scripting

We worry about VM sprawl but what about device sprawl? Management of a multitude of network-deployed solutions can be as operationally inefficient as managing hundreds of virtual machines, and far more detrimental to the health and performance of your applications. Turning them all into virtual network appliances that might need scaling themselves? That’s even badder.

But all you hardware fanbois best not smirk too much because the proliferation of hardware network devices is only slightly less badder than the potential problems arising from virtual network appliance sprawl.

All the same reasons cited by various pundits since the virtualization craze began regarding the difficulties associated with virtual machine sprawl can be applied to virtual network appliance sprawl. For the most part it applies to hardware network device sprawl, too, for that matter.

1. Cost of IPAM (IP Address Management)

This is probably even worse than is often described by Greg Ness when it’s applied to network solutions as compared to virtual machines simply because most network solutions have at least two IP addresses assigned to them – one for management and one to do its job – if not more. There are exceptions, of course, as some solutions are deployed inline and transparently, but there are other challenges associated with such configurations as they often require port mirroring which effectively ties the solution to a specific port on a specific switch. Obviously moving it or scaling it out horizontally as a virtual machine would prove problematic for these solutions. So let’s just ignore those for the purposes of this discussion, shall we? 

2. The impact on performance

Ignoring scalability – let’s assume a virtual network appliance is equal to the task for this post – the more points at which requests/traffic must stop and be processed the more latency is incurred. If you string together enough devices – regardless of the physical implementation – you are going to degrade performance. In some cases by a few milliseconds, in others perhaps by seconds. The amount of degradation relies heavily on the volume of requests, the type of processing being performed, and the capacity of each network device. Remember that the network is only as fast as its slowest hop, and that one poorly performing network device can destroy network and application performance.

3. Cost of management, power, and training

If you deploy five different network devices to address five different needs, you incur the cost of management, power, and training for each of them. This is true regardless of physical implementation as moving a solution from hardware to a virtual appliance doesn’t change the fact that it (1) needs to be managed, (2) has an interface/commands/quirks that need to be learned, and (3) consumes power.

4. Trouble with Troubleshooting (a.k.a. Lack of Visibility)

Even if every one of the X network solutions you have deployed individually has great visibility you’re still going to run into trouble troubleshooting. That’s because what one device may or may not do to a request/traffic isn’t easy to correlate by the time it’s passed through the fifth or sixth network device. It’s not as if all these devices add metadata that describes what they did to the traffic, they just do it and pass it along. The more devices, the more complicated this process becomes.

5. Special Issue with Virtual Network Appliances: Distributed Management

Remember how you didn’t want to shell out the extra cash for the vendor-specific distributed management solution? If you’re scaling out a network solution via multiple virtual network appliances you may want to reconsider that decision. Once you get past a couple of instances you’re going to need something to help you manage them and keep their configurations in synch or you’re asking for trouble. And don’t forget about the hypervisor management system, too. You’ll need that, I’m sure.

Sprawl of any kind incurs costs per node at a fairly consistent rate. Every instance – physical or virtual – adds to the combined total cost of ownership and investment in time. Every device through which traffic must flow also incurs a performance penalty, which to the business stakeholder is probably more dangerous than the hit on your budget.

Yeah, I know. Surprised me too when I found that out. There are actually solutions that aren’t focused on network or application networks. Whodda thunk it?

But for application delivery focused solutions – acceleration, optimization, caching, application security, load balancing – the solution to the problems of network device sprawl are unification onto a single, extensible (modular) platform. And while many network folks hear “modular” and think “chassis” (and that can be one approach) I’m talking about the core system itself. The solution, not the container.

By sharing a common core networking platform, a unified application delivery infrastructure mitigates the problems associated with extra hops/stops in the flow of requests/traffic by eliminating them. Requests that need to be passed through a web application firewall before being passed to a Load balancer do so, but because the common core networking platform is shared there’s no network or network stack overhead incurred by the passing of the data.

Network sprawl really is badder than VM sprawl because it not only increases the overall cost to deliver and secure applications but it can also negatively impact the performance and reliability of applications. A unified platform affords choice in the ability to add functionality as needed, to try out functionality to see if it’s worth it, and to scale out in a more efficient way on an as-needed (on-demand) basis.

One of the reasons virtualization is so appealing is it addresses nicely the “lots of little boxes” problem that causes management headaches throughout the data center. Consolidation through virtualization was the answer to that one, at least in terms of the sprawl associated with the physical devices. Unified infrastructure addresses the same “lots of little network boxes” problem that causes similar headaches on the network and application network side of the data center by consolidating many of the application delivery focused functions onto a single, shared and extensible application networking platform.

Related blogs & articles:

• The Application Delivery Deus Ex Machina
• The Question Shouldn’t Be Where are the Network Virtual Appliances but Where is the Architecture?
• What Are the Barriers to Entry and IT Diseconomies?
• Infrastructure 2.0: The Diseconomy of Scale Virus
• Disk May Be Cheap but Storage is Not
• Infrastructure 2.0 Is the Beginning of the Story, Not the End
• Virtual Network Infrastructure: Virtually Good Enough?
• Virtualization: Just how far are we willing to take it?
• Imagine...Manageability
• Two Different Sock(et)s
• The House that Load Balancing Built
• Building an elastic environment requires elastic infrastructure

We seem on the verge of repeating the mistakes associated with failed SOA implementations: ignoring the larger issue of architecture.

Everyone – from pundit to public – is asking the same question: “Where are the network virtual appliances?” But fewer people seem to be asking a question that needs to go hand-in-hand with that one: “Where are the architectural guidelines to support deployment of network virtual appliances?” SOA has been deemed by many to be a failure in part because it lacked true architectural guidance. Architects were simply unable – whether by lack of skills or training or lack of support from the rest of the organization – to design an architecture that took advantage of services and thus the result was often little more than “service sprawl.” Services did not scale well, they were not so easy to integrate, and no one really had a good handle on what services were available, and where.

Lack of an architectural strategy to accompany a network virtual appliance will likely lead to the same end: network sprawl and a lack of scalability or worse – scalability that’s costly in terms of expenses and resources. 

Rich Miller, who’ll be joining a panel of other industry notables at Cloud Connect to discuss Infrastructure 2.0 and what’s necessary to successfully move forward with these “new” infrastructures, may have inadvertently pointed out the lack of architectural guidance related to virtual network appliances when he said:

If a vendor is going to sell network virtual appliances, the nva's should be designed from the get-go to be scalable (both 'up' and 'out'), and designed with the notion that the 'appliance' is not just a physical appliance without the box. That is 'horseless carriage' product design, which casts new technologies in exactly the same roles as their precursors.

What Allan doesn't say is that this may require the wider deployment of network infrastructure designed specifically for virtualized appliances and converged IO. It's not just whitebox, commodity x86 hardware running general purpose virtual machine environments for server virtualization.

                                                                                 -- Rich Miller in “Where ARE the Network Virtual Appliances?”

Rich is focusing more on internal design in general, but any such “design” must also necessarily include how the VNA scales in the target environment. Scalability is at the heart of all definitions of cloud computing and without the ability to scale solutions – whether application, network, storage, or application network – any such implementation will almost certainly be deemed a failure.

Scaling up, i.e. vertical scalability, in a cloud computing or virtualized environment is in essence little more than “throwing more hardware” at the problem. Scaling “up” adds more compute resources, yes, but it is not “on-demand” today because it effectively requires re-provisioning of large chunks of resources. Cloud computing and virtualization in particular today are not capable of simply “adding on” more CPU or RAM to a virtual machine and even if it were there are hard, physical limitations imposed by the underlying hardware on the upper bounds of such a strategy.

Scaling “up” a virtual network appliance in practice is really no different than scaling up hardware. It leads to over-provisioning by necessity and in the event that capacity and physical constraints are reached, requires provisioning a new, higher capacity instance which while easier than upgrading hardware counterparts still requires much the same process in terms of deployment.

While I agree with Rich’s assessment that virtual network appliances should be designed to scale up as efficiently as possible, that doesn’t change the challenges associated with actually scaling up the solution in a dynamic environment or that it’s not all that much different than what we do today to try to future-proof the sizing of solutions.

Scaling out, i.e. horizontal scalability, is usually the more desirable choice in these discussions. This makes a great deal more sense even though scaling “out” is still essentially a “throw more hardware at the problem” solution, it’s a more temporary “toss” and is more flexible in terms of growing capacity on-demand. It’s certainly more efficient and agile to deploy another virtual network appliance than it is to acquire and deploy another physical network appliance.

The problem with this approach is not in the details. It’s in the broader architectural strategy applied to the process, which today is virtually non-existent. Scaling out is a proven method of addressing capacity constraints. We do it all the time with web and application servers, with firewalls, with XML gateways. load balancing as a method of implementing a horizontally scalable application and network infrastructure is nothing new and it is indeed efficient, scalable, and architecturally sound.

The issue is with how one scales out, and what. The call for “virtual network appliances” in general ignores the architectural implications in favor of some perception of increased flexibility and scalability. There are simply some functions within the data center that would not benefit from being “virtualized” and others that will not benefit without a strong set of architectural guidelines. Some functions should never be virtualized because such an architecture would not be feasible to implement and would do more harm than good to both network and application performance.

Let’s take core routing, for example. One of the reasons you’d want to “scale out” a core router is because it has hit an upper constraint on bandwidth. Perhaps it’s only capable of handling 10Gb of aggregate bandwidth entering the data center/cloud computing environment but you need to handle 20 or 30Gb of bandwidth. In a completely virtualized architecture you’d just scale “out” by adding another another virtual router, right? That will certainly increase aggregate bandwidth capacity, but fails to address a very important question: how is traffic directed to one instance or another? Do we have to scale the scalability? And if so, how does that work? Do the core routers deploy in an active-active configuration, both masquerading as the entry point into the data center? Sharing of “bogus” MAC addresses across active-active-n scaling architectures is the most common solution to this problem, but introduces others related to failover and network utilization. That latter piece is due to the natural behavior of switches and reliance on MAC address/port affinity; essentially this solution turns a switch into a giant hub, replicating data/traffic across all possible ports on which the “bogus” MAC address might be. As you scale out, more and more bandwidth will be consumed by this broadcasting behavior and can make troubleshooting more difficult, especially in environments where visibility is already limited such as cloud computing providers.

Is it the case that every virtual network appliance capable of being “scaled out” will essentially need to be capable of acting like a Load balancer? Because that’s how it looks from here. Horizontal scalability is based on the premise that something – some device, some solution – is load balancing requests/data/traffic across the multiple instances. Without the load balancing solution, such implementations are nearly impossible to achieve. So imagine the potential issue when the load balancer is virtualized, too. It, also, must scale “out” and thus must be “scaled” itself by … a load balancing solution. Such an implementation is certainly achievable, but also requires that the “primary” load balancing solution is scaled “up” in order to handle the aggregate request/data/traffic being directed at the infrastructure. Limitations on vertical scalability return us right back to a solution based on horizontal scalability, which puts us right back here where we are: how do we scale out the “more scalable” virtual network appliances that are so highly in demand? 

We haven’t even touched the large problem of sprawl in a virtual network infrastructure. Management systems aren’t quite ready for such an implementation, and ironically part of the reason cloud computing, virtualization, and infrastructure 2.0 are coming of age now is because we have issues with managing an increasing volume of servers, applications, devices, and IP addresses across the data center. Deploying an infrastructure comprised of virtual network appliances without a strong architectural strategy and a supporting management strategy is sheer folly, and puts us no better off than we are today.

We should be very careful to ask ourselves why we want a particular solution in a network virtual appliance and how it might impact the network and management of the network before we blithely toss it into our critical network and application network infrastructure. Architecture is inherently as important when designing any type of distributed system, and when moving from hardware to distributed software as a means to achieve scalability there needs to be more a lot more thought and strategy put into the process.

While there are certainly going to evolve architectures that take advantage of virtual network appliances, and traditional hardware appliances, and combinations thereof, we need to tread carefully forward and ensure that our driving desire for what appears to be flexibility doesn’t end up breaking the backbone of the data center: the network.

A well-thought planned architectural strategy for integrating virtual network appliances with traditional data center components will go a long way toward ensuring maximum flexibility without stretching the network so tightly that it breaks.

Related blogs & articles:

• You’re Asking the Wrong Question About Virtual Appliances
• Clouds Are Like Onions
• A Fluid Network is the Result of Collaboration Not Virtualization
• Infrastructure 2.0: Squishy Name for a Squishy Concept
• Pursuit of Intercloud is Practical not Premature
• WILS: Virtual Server versus Virtual IP Address
• Virtual Infrastructure in Cloud Computing Just Passes the Buck
• Scaling Security in the Cloud: Just Hit the Reset Button
• Vertical Scalability Cloud Computing Style

The difference between these two performance metrics is significant so be sure you know which one you’re measuring, and which one you wanted to be measuring. 

It may be the case that you’ve decided that SSL is, in fact, a good idea for securing data in transit. Excellent. Now you’re trying to figure out how to implement support and you’re testing solutions or perhaps trying to peruse reports someone else generated from testing. Excellent. I’m a huge testing fan and it really is one of the best ways to size a solution specifically for your environment.

Some of the terminology used to describe specific performance metrics in application delivery, however, can be misleading. The difference between SSL TPS (Transactions per second) and HTTP TPS over SSL, for example, are significant and therefore should not be used interchangeably when comparing performance and capacity of any solution – that goes for software, hardware, or some yet-to-be-defined combination thereof.

The reasons why interpreting claims of SSL TPS are so difficult is due to the ambiguity that comes from SSL itself. SSL “transactions” are, by general industry agreement (unenforceable, of course) a single transaction that is “wrapped” in an SSL session. Generally speaking one SSL transaction is considered:

1. Session establishment (authentication, key exchange)

2. Exchange of data over SSL, often a 1KB file over HTTP

3. Session closure

Seems logical, but technically speaking a single SSL transaction could be interpreted as any single transaction conducted over an SSL encrypted session because the very act of transmitting data over the SSL session necessarily requires SSL-related operations. SSL session establishment requires a handshake and an exchange of keys, and the transfer of data within such a session requires the invocation of encryption and decryption operations (often referred to as bulk encryption).

Therefore it is technically accurate for SSL capacity/performance metrics to use the term “SSL TPS” and be referring to two completely different things.

This means it is important that whomever is interested in such data must do a little research to determine exactly what is meant by SSL TPS when presented with such data. Based on the definition the actual results mean different things. When used to refer to HTTP TPS over SSL the constraint is actually on the bulk encryption rate (related more to response time, latency, and throughput measurements), while SSL TPS measures the number of SSL sessions that can be created per second and is more related to capacity than response time metrics. It can be difficult to determine which method was utilized, but if you see the term “SSL ID re-use” anywhere, you can be relatively certain the test results refer to HTTP TPS over SSL rather than SSL TPS. When SSL session IDs are reused, the handshaking and key exchange steps are skipped, which reduces the number of computationally expensive RSA operations that must be performed and artificially increases the results.

As always, if you aren’t sure what a performance metric really means, ask. If you don’t get a straight answer, ask again, or take advantage of all that great social networking you’re doing and find someone you trust to help you determine what was really tested. Basing architectural decisions on misleading or misunderstood data can cause grief and be expensive later when you have to purchase additional licenses or solutions to bring your capacity up to what was originally expected.

WILS: Write It Like Seth. Seth Godin always gets his point across with brevity and wit. WILS is an ATTEMPT TO BE concise about application delivery TOPICS AND just get straight to the point. NO DILLY DALLYING AROUND.

Related blogs & articles:

• The Anatomy of an SSL Handshake
• When Did Specialized Hardware Become a Dirty Word?
• WILS: Virtual Server versus Virtual IP Address
• Following Google’s Lead on Security? Don’t Forget to Encrypt Cookies
• WILS: What Does It Mean to Align IT with the Business
• WILS: Three Ways To Better Utilize Resources In Any Data Center
• WILS: Why Does Load Balancing Improve Application Performance?
• WILS: Application Acceleration versus Optimization
• All WILS Topics on DevCentral
• What is server offload and why do I need it?

Emerging architectures are conflating responsibilities up and down the application stack. Who is responsible for integration when services reside in the network?

While preparing for an upcoming panel I’m moderating at Cloud Connect (in the “New Infrastructure” track), the panelists and I had a great discussion on the topics we wanted to discuss in the session. During that discussion it became increasingly clear that an interesting phenomenon has been occurring: the conflation of network and application responsibilities in the traditional “stack.”

Much of this inversion is absolutely necessary for emerging models of networking and computing to be successful. Traditional methods of handling QoS (Quality of Service) and identity management, for example, are no longer adequate in the inherently volatile world of cloud computing and dynamic networks. Interestingly, the driver behind the inversion appears to be based largely on the ability of specific layers access to context, which is necessarily replacing IP addresses as a method of client – and server – identification.

Back in the day, QoS was a class of problem unto itself, with an entire market of products and solutions developed specifically to address the challenge of prioritizing traffic. Initially it was thought that the ToS (Terms of Service) bits in the IP header would suffice, but it quickly became obvious that this required every organization and provider to honor those bits as traffic flowed through and across the Internet.

Didn’t happen.

A market emerged that moved QoS “up the stack” to Layer 4 (transport protocol). A class of devices were deployed that employed either TCP rate shaping or packet queuing technologies to control the amount of bandwidth a given “application” could consume. It quickly became apparent that this method was not robust enough as more and more “applications” began to use the same protocol: TCP. The devices again moved “up the stack” to Layer 7 (application) and began to apply QoS policies based on actually identifying applications based on layer 7 protocols and data characteristics.

In recent years even this has become inadequate because these techniques were all focused on limiting, in some way, total bandwidth for an application. While these solutions were also able to, albeit rudimentarily, accomplish rate shaping on a per-user basis, they still focused on bandwidth as their metric of choice to control. Hence a single user could be limited to X Kbps for all HTTP traffic, and further limited to Y percent for application A and Z percent for application B, but bandwidth as a meter of usage for applications today is not an appropriate measurement.

Hence, QoS has again moved up the stack and is more granular than ever. Rather than worrying about bandwidth, which has grown increasingly cheap and available for both organizations and users, QoS now concerns itself with limiting requests on a per-user basis and, in some cases, a per-client-type basis. Consider Twitter’s rate limiting implementation for its API. This is a modern implementation of QoS that attempts to equalize access to its services for all users, effectively ensuring a consistent quality of service for everyone. Bandwidth is not a factor, because the amount of bandwidth consumed by any given client is highly variable and based on what data is being requested.

Similarly we often see requests for ways in which application usage can be limited based on application layer variables, with nary a mention of bandwidth. It’s always about users and usage patterns of a specific application.

What was once a “network” function, QoS, has moved “up the stack” and is now primarily the responsibility of the “application.”

It wouldn’t be an inversion of responsibility if traditionally “application” layer responsibilities weren’t being similarly pushed “down the stack.” A good example of how this is occurring today is in the area of “identity”, which traditionally includes authentication and authorization.

In the early days of web applications, identification was based on a user name and password (sometimes IP address, sometimes a combination thereof) and was expected to be handled by the application. After all, the application knew what users should be allowed and thus is was the demesne of the application to provide those mechanisms. The use of .htaccess files was widespread as a means to achieve this functionality.

But as technology began to merge the world of the web with the internal world of IT, it became increasingly common to leverage external applications as an identity store and the means by which users were authenticated and authorized to access applications. LDAP, Active Directory, RADIUS, DIAMETER. These protocols resided somewhere between the application layer and the transport layer and provide the data necessary for applications to make access decisions.

But again, this method has run into obstacles in adapting to volatile and large environments. Scalability and the need to execute complementary access policies the network layer in authentication and authorization decisions has continued to drive identity and authentication and authorization “down the stack” and into the “network”. In a highly scaled environment, for example, it is often preferable that an intermediary Load balancer authenticate users to an application because it is increasingly painful for developers to tightly integrate application access and security policies into the application. Traditional methods are brittle, static designs that are increasingly tossed out in favor of more policy-based access that resides somewhere “in the network” rather than tightly-coupled with the application.

What was once an “application” function has moved “down the stack” and is now increasingly the responsibility of the “network.”

The conflation of responsibilities up and down the “stack” point to either an increasingly flattened application architecture comprised of services; services that may reside in the  application layer or the network layer, but are leveraged by both in approximately the same way.

This is actually much of the brouhaha behind Infrastructure 2.0; behind the evolution of the network to become “smarter” and more “integrated” with the rest of the infrastructure. As the network takes on more and more responsibility from the applications, especially as is the case in an increasingly cloudy environment, the components in the network must be able to consume services provided by other components and collaborate as a means to ensure the fast and secure delivery of applications to their ultimate consumers.

One of the side-effects is that it will cause some amount of confusion in the organization, at “layer 9”, as it were, regarding what role is responsible for developing and ultimately deploying those policies. Will developers become more network-aware? Will administrators and operators begin to take on a more development-oriented role in order to integrate and orchestrate the data center using the collaborative capabilities of Infrastructure 2.0 services?

Maybe the answer to that depends on where you are, who you are, and whether you’ve drank from the bottle or not.

Related blogs & articles:

• Accelerating Your (Secure) Ride to the Cloud: Get in the Fast Lane
• WILS: Automation versus Orchestration
• The Cloud Metastructure Hubub
• Infrastructure Integration: Metadata versus API
• API Request Throttling: A Better Option
• Elastic Environment requires Elastic Infrastructure 
• Impact of Load Balancing on SOAPy and RESTful Applications
• Infrastructure 2.0: Squishy Name for a Squishy Concept
• A Fluid Network is the Result of Collaboration Not Virtualization
• Clouds Are Like Onions

Which of course are like Ogres. They’re big, chaotic, and have lots of layers of virtualization.

In discussions involving cloud it is often the case that someone will remind you that “virtualization” is not required to build a cloud. But that’s only partially true, as some layers of virtualization are, in fact, required to build out a cloud computing environment. It’s only “operating system” virtualization that is not required. Problem is unlike the term “cloud”, “virtualization” has come to be associated with a single, specific kind of virtualization; specifically, it’s almost exclusively used to refer to operating system virtualization, a la Microsoft, VMware, and Citrix. But many kinds of virtualization have existed for much longer than operating system virtualization, and many of them are used extensively in data centers both traditional and cloud-based. Like ogres, the chaotic nature of a dynamic data based on these types of virtualization can be difficult to manage.

Layer upon layer of virtualization within the data center, like the many layers of an onion, are enough to make you cry at the thought of how to control that volatility without sacrificing the flexibility and scalability introduced by the technologies. You can’t get rid of them, however, as some of these types of virtualization are absolutely necessary to the successful implementation of cloud computing. All of them complicate management and make more difficult the task of understanding how data gets from point A to point B within a cloud computing environment.

Yes, that’s right, eight kinds of virtualization exist though we tend to focus on just the one, operating system virtualization. Some may or may not be leveraged in a cloud computing environment, but at least four of them are almost always found in all data center environments.

1. Operating System Virtualization is what we tend to think of when we simply say “virtualization.” This is the virtualization of compute resources, the slicing and dicing of a single physical machine into multiple “virtual” machines typically used today to deploy several different applications (or clones of a single application) on the same physical hardware.
   
2. Network Virtualization is likely one kind of virtualization many don’t even consider virtualization, but it is and it’s even got standards that help ensure consistency across implementations. The VLAN (Virtual LAN) has existed since the early days of networking and is used in cloud computing environments to isolate customer data. VLANs essentially create a virtual network overlay atop an existing physical network, slicing and dicing the physical connections into multiple virtual (and hopefully smaller) networks that can be configured to provide security and network-layer functions like quality of service and rate shaping peculiar to the applications and users that are directed over the VLAN. VLAN tagging, used to identity traffic as “belonging” to a specific virtual network, is defined by IEEE 802.1q.
   
   Also a form of network virtualization is trunking or link aggregation as defined by IEEE 802.1ad. Trunking aggregates multiple physical ports on a switching device and makes them appear as one logical (virtual) link, providing additional bandwidth to high volume networks as well as load balancing traffic across the physical interconnects in order to maintain consistent network performance. Interestingly enough, VLANs are almost always used when trunking is used in a network.
   
   And of course there is NAT (Network Address Translation), which is also a form of network virtualization. Because of the dearth of IP addresses, most users internal to an organization are directed through a pool of one or more public IP addresses (routable, i.e. accessible by people across the Internet) to access resources external to the organization. The virtualization here again makes many IP addresses (internal, non-routable, private) appear to be one or a small number of IP addresses (public, routable, external). This process is also used on inbound connections, making one or a small number of external, public IP addresses appear to represent multiple, internal, private IP addresses.
   
3. Application Server Virtualization occurs when a Load balancer, application delivery controller, or other proxy-based application network device “virtualizes” one or more instances of an application. The process of virtualization an application server makes multiple servers appear to be one ginormous server to clients, and acts in a manner very similar to trunking in that this form of virtualization is about aggregation. When applied to application servers, this virtualization focuses on the aggregation of compute resources.
   
   This form of virtualization is almost always necessary in a data center, whether traditional or cloud-based. Application server virtualization is the foundation on which failover (reliability) and scalability are based, and one would be hard-pressed to find a modern data center in which this form of virtualization – whether provided by software or hardware – is not already implemented.
   
4. Storage Virtualization is another form of aggregation-based virtualization. Storage virtualization aggregates multiple sources of storage such as NAS (network attached storage) devices and NFS/CIFS shares hosted on various servers around the data center and “normalizes” them into a single, consistent interface such that users are isolated from the actual implementation and see only the “virtual” namespaces presented by the storage virtualization device.

There are four other “types” of virtualization, but it is these four that are primarily utilized today and with which most folks are already familiar – it just may be that they are using different terminology. Perhaps that’s because virtualization of the network and application server have existed for so long most people do not associate it with virtualization. All four of these kinds of virtualization end up forming layers of abstraction throughout the network, and like operating system virtualization introduce management and architectural challenges that are increasingly difficult to address as environments become more and more dynamic, a la a cloud computing environment.

It is these challenges that Infrastructure 2.0 is attempting to address. The increased strain on networks and infrastructure caused by virtualization of multiple types and the need to dynamically configure and manage all the various components that comprise a cloud computing or highly virtualized environment is enormous. The burden is often placed on the shoulders of operators and administrators who are tasked with keeping straight the myriad processes and tasks that must be complete ere a new resource is added to any one of the “virtual” pools of resources. Whether that’s storage, or application servers, or networks, or applications the challenges are similar in nature.

The ability of network, storage, and application network components to collaborate in a common, standards-based way will be imperative to the long-term success of virtualization and cloud computing. Infrastructure 2.0 enabled components already exist, true, but the means by which they are integrated into the broader data center ecosystem still vary from component to component. While the existence of these dynamic control planes makes it possible to reduce the strain associated with managing and running a dynamic data center, such variations also introduce difficulties and can lead to vendor lock-in. Addressing these concerns is paramount to ensuring the long-term viability of emerging data center models, and to making the introduction of virtualization into the data center a less painful process.

Clouds, like ogres and onions, have layers. Layer upon layer of abstraction through the use of virtualization to provide for scalability and security of the applications that are being delivered. Like onions, attempting to manage such a dynamic, virtual environment could easily make an operator cry. Infrastructure 2.0 is a necessary movement forward to address the challenges that will continue to plague data center architects and operators as they attempt to implement a dynamic data center that achieves IT agility to match the demand for business agility.

If you’re going to Cloud Connect, you may want to sign up for the “New Infrastructure” track and learn more about Infrastructure 2.0 and the challenges it is attempting to address.

Related blogs & articles:

• I Found the Missing Piece of the Virtualization Puzzle
• Server Virtualization versus Server Virtualization
• Virtual Machine Density as the New Measure of IT Efficiency
• Infrastructure 2.0: Squishy Name for a Squishy Concept
• A Fluid Network is the Result of Collaboration Not Virtualization
• WILS: Virtual Server versus Virtual IP Address
• Virtualization Defined-Eight Different Ways
• Reason #1 That You Need File Virtualization
• Reason #2 That You Need File Virtualization
• Reason #3 That You Need File Virtualization 
• Reason #4 That You Need File Virtualization
• Reason #5 That You Need File Virtualization

Using HTTP headers and default browser protocol handlers provides an opportunity to rediscover the usability and simplicity of the mailto protocol.

Over the last decade it's become unsafe to use the mailto protocol on a website due to e-mail harvesters and web scraping. No one wants to put their e-mail address out on teh Internets because two minutes after doing so you end up on a trillion SPAM lists and the next thing you know you're changing your e-mail address.

But people still wanted to share contact information, so it became common practice to spell out your e-mail address, such as l.macvittie AT F5 dot com. But e-mail harvesters quickly figured out how to circumvent that practice so people got even more inventive, describing how to type the @ sign instead. For example, you can send me an e-mail at l.macvittie SHIFT 2 f5.com. But that's inconvenient and isn't easily automated, and eventually the e-mail harvesters figure that one out, too.

You could use contact forms instead to hide the e-mail address, but that's not really sharing and it isn't convenient for the person trying to get a hold of you. Like many folks, if I have a need to contact you I’d like a record that I did so and contact forms rarely provide a copy of the message which makes managing communication more difficult. It also affords spammers an easily automated method of submitting spam. What you really want is to be able to share your e-mail address and avoid the automated e-mail harvesters. Some folks suggest using CSS tricks that manipulate selectors to hide the e-mail address, but the problem with this is that it (1) doesn’t automatically launch a mail client and (2) the e-mail address is still in the text of the page, it’s just located in a different place. Some techniques use pure CSS and pseudoclass selectors and others use CSS to expose the actually e-mail address that is “hidden” in one of the HREF attributes, often the title. But in both cases the address is still in the page – or in an external CSS file which bots might pull if they’re following all links - and a simple regular expression search will find it easily enough. 

The mailto link in the presentation page is changed to a standard HTTP link which, when clicked, executes logic that sends an HTTP redirect to a mailto location instead of a more standard HTTP location. The reason using this technique works is that the location to which the browser is being redirected is “hidden” in the HTTP headers, which bots and spots rarely interpret or expect to carry pertinent information and it is the browser that must interpret the location, which means any client-side supported protocol – like mailto – will cause the execution of the expected action. In this case it is launching the user’s e-mail client. This technique could, of course, be used to silently launch other client-side applications for which a protocol handler is defined as well.

A traditional HTTP redirect header to a web page would look like this:

   1: Location: http://www.w3.org/pub/WWW/People.html

And what we want is simply to make it look like this:

   1: Location: mailto:myemailaddress@example.com 

There are two easy ways to implement this solution: network-side and server-side scripting. 

First you need a URI which you can map to an e-mail address, e.g. /getmailto. The script needs to (1) look for that URI and (2) respond to the call to that URI with an HTTP redirect containing the appropriate e-mail address.

   1: when HTTP_REQUEST { 

   2:    set curr_uri [HTTP::uri]  

   3:    if {$curr_uri starts_with "/getmailto"} { 

   4:       HTTP::redirect "mailto: <insert e-mail address here>" 

   5:    }

   6: }

Now replace your mailto links with a link to the new URL. If your browser and mail client are configured properly, clicking on the link should bring up a new e-mail message with the e-mail address filled in. That supports usability needs (the e-mail address link should launch the user’s mail client) but it also keeps the address out of the page.

You'll probably want to further filter access to the URL by putting some iRule code in to detect bots and spiders and prevent them from exploring this one, but that's pretty easy, too. If you only have to replace one e-mail address, you could probably avoid rewriting the mailto links and simply use an iRule to transform the original mailto links to the new URL. And I'm sure someone out there will figure out how to change any mailto link to a new URL as well.

For example, if all e-mail addresses use the same formula, i.e. first initial, dot, lastname, you could construct a URL that sent the information as the URL, i.e. /lmacvittie. You can use a network-side script to then parse it into the right e-mail address and send the redirect back to the user. Using iRules you could also create a data group that maps URIs to e-mail addresses and do a quick lookup based on the URI to extract the appropriate e-mail address. As mentioned, you can do the redirect using mod_rewrite as well. I think iRules affords more flexibility in dealing with the actual data being manipulated (the e-mail address –> URI mappings), but you should be able to do it using other tools as well. The trick here is in putting the e-mail address in the HTTP header rather than in the body of the page where it is easily discovered by harvesting tools.

For example, in PHP, all you need to do is call the function header with the appropriate location:

   1: header('Location: mailto: myemailaddress@example.com');

Rather than add this code to every page with an e-mail address it might be advantageous to take a service-based approach and simulate network-side scripting capabilities by creating a single “page” for all mailto redirects and then implementing the lookups and return of the appropriate HTTP redirect in a centralized, more manageable service.

Note that while you could achieve the same effect using custom HTML pages with the appropriate META tag or a small piece of JavaScript, this will result in the e-mail address being in a static page that a bot or spider can find and parse. The best solution will use network or server side executed logic because such code is not generally retrieved and parsed by miscreants. This also allows the integration of lookups dynamically. For example, both server and network-side scripting solutions may integrate with systems such as LDAP or AD and could therefore create a request to lookup an e-mail address dynamically based on the HTTP request.

There are other solutions to prevent this type of web scraping behavior, and of course any solution combined with a good SPAM prevention solution will improve the quality of the e-mail received. SPAM may be a fact of life on the Internet, but anything we can do to preserve the user experience while cutting down on how much SPAM we receive has to be a good thing.

UPDATED NOTE: I just had a thought that because this essentially moves e-mail to a URI-based system, it should be possible to integrate techniques like a CAPTCHA to further secure access to e-mail addresses against bots, spiders, and scripts.

Related blogs & articles:

• Long Live(d) AJAX
• Advanced Web Security with BIG-IP v10.1 (including Web Scraping Detection and Prevention) [Slideshare Presentation]
• Extensive List of Methods to Hide E-mail Addresses from Harvesters
• W3C HTTP/1.1 Header Field Definitions
• When Is More Important Than Where in Web Application Security
• Using Network-Side Scripting to Implement Mock API Endpoints
• Understanding network-side scripting
• Excuse Me But Is That a Gazebo On Your Site?!
• I Can Has UR .htaccess File
• Clickjacking Protection Using X-FRAME-OPTIONS Available for Firefox
• Stop brute force listing of HTTP OPTIONS with network-side scripting
• All blogs related to “network-side scripting”

I haven’t heard the term “graceful degradation” in a long time, but as we continue to push the limits of data centers and our budgets to provide capacity it’s a concept we need to revisit.

You might have heard that Twitter was down (again) last week. What you might not have heard (or read) is some interesting crunchy bits about how Twitter attempts to maintain availability by degrading capabilities gracefully when services are over capacity.

“Twitter Down, Overwhelmed by Whales” from Data Center Knowledge offered up the juicy details:

The “whales” comment refers to the “Fail Whale” – the downtime mascot that appears whenever Twitter is unavailable. The appearance of the Fail Whale indicates a server error known as a 503, which then triggers a “Whale Watcher” script that prompts a review of the last 100,000 lines of server logs to sort out what has happened.

When at all possible, Twitter tries to adapt by slowing the site performance as an alternative to a 503. In some cases, this means disabling features like custom searches. In recent weeks Twitter.com users have periodically encountered messages that the service was over capacity, but the condition was usually temporary. At times of heavy load for more on how Twitter manages its capacity challenges, see Using Metrics to Vanquish the Fail Whale.

I found this interesting and refreshing at a time when the answer to capacity problems is to just “go cloud”, primarily because even if (and that’s a big if) “the cloud” was truly capable of “infinite scale” (it is not) it is almost certainly a fact that most organization’s budgets are not capable of “infinite payments” and cloud computing isn’t free.

It’s been many years, in fact, since the phrase “graceful degradation” has been uttered within my hearing, but that’s really what the article is describing and it’s something we don’t talk enough about. Perhaps that’s because it’s difficult to admit that there are limitations – whether technical or financial – on the ability to scale and meet demand. But there are, and if organizations are wise they’ll include in their application delivery strategy the means by which applications and services can “degrade gracefully.”

Twitter’s solution, the disabling of specific features, is a particularly easy way to implement such a strategy for Web 2.0 applications; at least it’s particularly easy if you have a network-side scripting capable solution mediating for the applications.

G

The reason it’s particularly easy to gracefully degrade Web 2.0 applications is that there is generally a 1:1 mapping between “functions” and “URIs.” This is often true for the web-facing interface, almost always true for RESTful APIs, and always true for SOAPy endpoints.

What you need to do is identify those “premium” URIs, i.e. those that can be disabled without negatively impacting core services, so that they can be “degraded” in the face of an overwhelming volume of requests.

You also need an intermediary. This can be a Load balancer, assuming it’s capable of providing the flexibility in configuration necessary to enable and disable service to specific URIs, i.e. it must be layer 7 aware. It has to be an intermediary through which all requests are routed because individual servers do not have the visibility required to be able to “see” the total requests and all responses. The fact that a server is throwing back 503 (Internal Error) errors indicates it doesn’t have the resources available to respond to a request, which means it won’t be able to respond to any requests, including those to disable services. Only an architecture that includes an intermediary of some kind (a reverse proxy) can achieve this solution.

The network-side script, which is deployed on the application delivery platform (load balancer), should implement logic that triggers degradation based on receiving 503 errors. It should probably not trigger on a single 503 or multiple 503s from the same application instance as such behavior could be indicative of a problem with that one instance as opposed to being produced due to a lack of capacity. That means the scripting solution needs to be able to take action based on a pattern of behavior coming from all application instances in conjunction with the total number of requests being received from users.

Yes, it has to be context-aware.

Once it’s determined that the errors are being generated due to a lack of capacity, the scripting solution needs to disable one or more of the specific URIs determined to be “premium” or ancillary. The intermediary can then respond to subsequent requests for the disabled URIs with custom content based on the expected response type. For example, if it’s an API call it might be appropriate to return a pre-formatted response in the appropriate data format indicating service is currently unavailable. Many network-side scripting solutions are capable of returning pre-formatted responses or they can be customized to provide more detail – it’s really up to the implementer to decide what information is included and how.

The premise is that as premium or ancillary services are degraded (disabled) that application instances will be able to focus on servicing core requests and return service to normal for those pieces of the application. When the volume of requests returns to within normal operating parameters for the capacity available, the intermediary can restore service to the previously degraded services.

S

From a technological point of view “infinite scale” is not possible. At some point the volume of requests will reach boundaries that simply cannot be overcome, be they limitations on the load balancer (there is a limit to how many servers can ultimately be load balanced, and bandwidth is not unlimited) or on the application infrastructure itself. After all, you can’t launch a new instance of an application if there are no physical resources left on which to launch it.

It is almost certainly the case, however, that before reaching the technical limits of an “infinitely scalable” environment that you will hit a financial limitation. Or it may be the case that you haven’t jumped on the “cloud” bandwagon and what you see is what you get: a limited number of physical resources running a finite number of application instances, and that’s it. In either case, there are limitations on capacity and at some point you may reach them. How you respond to those limitations is an organizational decision, but graceful degradation in a controlled manner is probably more desirable than random, uncontrolled service outages.

Graceful degradation is an acceptable strategy for responding to availability issues and is especially easy to implement for a Web 2.0 application or API. It’s certainly more appealing than the alternative, which leaves every user essentially playing a game of Russian Roulette with availability of your web application.

Related blogs & articles:

• Twitter Down, Overwhelmed by Whales
• HTTP Session Limit iRule – Limit access based on pre-determined total connection limits
• The Infrastructure 2.0 Trifecta
• WILS: How can a load balancer keep a single server site available?
• How To Use CoralCDN On-Demand to Keep Your Site Available. For Free.
• It’s 2am: Do You Know What Algorithm Your Load Balancer is Using?
• The Application Delivery Deus Ex Machina
• To Take Advantage of Cloud Computing You Must Unlearn, Luke
• Vertical Scalability Cloud Computing Style
• Putting a Price on Uptime
• Cloud Computing: Vertical Scalability is Still Your Problem

Nope. Wasn’t under the couch. In fact it turns out it wasn’t even missing, it’s just been overlooked and might already be in your data center.

As more organizations continue to make virtualization a core part of their overall application deployment strategy they are finding challenges associated with managing and, apparently, optimizing their newly created heterogeneous infrastructure. Kevin Fogarty, in “10 Virtualization Vendors to Watch in 2010”, writes of some of the challenges with virtualization to come in the next year. One of those challenges is, apparently, optimization of resources across physical and virtual assets, at least according to Mark Bowker of the Enterprise Strategy Group.

"Anybody who can fill the gaps the big guys don't in helping virtualization admins provision and control their infrastructure is worth a look," adds Mark Bowker, virtualization specialist at Enterprise Strategy Group. "The real missing piece, though, is the ability to optimize performance across both physical and virtual assets."

Later, Kevin addresses the challenges associated with capacity planning in a virtualized environment.

In the virtual world, however, capacity management is something of a black art -- not because few people have thought of it, but because few have built tools to look at both the physical and virtual servers and see how many of one will overwhelm the other. VKernel's product works on both VMware and Microsoft's Hyper-V. Without detailed capacity planning based on real data -- not imagination -- large-scale virtualization of production systems is not practical, according to Chris Wolf, analyst at The Burton Group.

One of the easiest and most effective ways to address the challenge of optimizing performance of heterogeneous environments comprising both physical and virtual assets is to optimize the applications being delivered by those assets. Optimizing the application means making more efficient the way the application and its application stack, i.e. the web and/or application server, makes use of compute resources. Doing so means a single virtual machine increases its capacity while lowering its resource requirements, which can  translate into a higher VM density.

What seems to be ignored – either purposefully or accidentally – is that applications and their infrastructure stacks are the same whether deployed on a virtual machine or a physical server. For the most part the application doesn’t change, just the number of layers between it and the server. What’s necessary to optimize both resources, then, is to attack performance issues that are common to both models, i.e. those related to TCP, HTTP, and application-specific protocols. If you can optimize connections and application behavior through better resource management at the protocol level, leveraging caching when available, and compressing only when it will be beneficial you can significantly improve the efficiency of the platform – virtual or physical – such that capacity is increased. And because you’re optimizing components common to both deployment models, it doesn’t matter whether those resources are “virtual” or “physical.”

It’s also true that it is possible to instruct application delivery solutions to leverage physical and virtual assets based on their unique properties and capabilities. For example, if a virtual resource has a capacity that is half that of a physical resource, the application delivery solution can certainly be configured in a way to take that into consideration when determining what resource is best suited at the time the request is received to respond. Optimizing the distribution of requests across physical and virtual resources should not be overlooked especially if the environment is heterogeneous both in type of resource (physical or virtual) and capacity (maximum resources available). A poorly chosen load balancing algorithm as a means to distribute requests, for example, can make even more inefficient a heterogeneous environment. But a well-chosen algorithm can significantly increase performance and overall capacity and make more efficient the entire application infrastructure, whether virtual, physical, or both.

Part of the problem with capacity planning for applications is, certainly, ignoring the overhead associated with virtualization. The hypervisor is, after all, still an application and requires resources. It is the “abstraction layer” between the virtual machines and the underlying operating system and hardware and all the “virtual” activity must be channeled through it, which is going to consume resources that will not be available to applications. In other words, a virtualized server will never provide 100% resource capacity for applications, and this needs to be considered when performing capacity planning. Microsoft, for example, notes in a TechEd Europe presentation, “Microsoft Exchange Server Virtualisation: Does It Make Sense? [UNC03-IS]”, that the “hypervisor adds processor overhead” citing the hypervisor consumes an approximate ~12% of processor resources in its own Exchange 2010 testing. Benchmarking tests conducted by VMWare on web servers, too, indicate a varying amount of hypervisor overhead (~16% at its highest) depending on configuration and resources available [Consolidating Web Applications Using VMware Infrastructure, PDF, VMWare].

There’s nothing anyone can really do about hypervisor overhead right now. Virtualization vendors are working with hardware vendors and “virtualization aware” solutions will at some point address much of this overhead. But still, some overhead has to exist when deploying hypervisor-based virtualization solutions because, well, it’s part of the solution. That means when you’re looking at capacity you must take into consideration that overhead and subtract it from the available resources you can provision to your applications. That shouldn’t be a problem as capacity planners have long held that you never plan for 100% capacity anyway. But it still must be accounted for and addressed. One of the ways to offset the loss of resource capacity associated with a hypervisor is to make the applications that will be run atop the hypervisor are as optimized as possible. If you can gain more through optimization than you lose in associated overhead, you win.

Cloud computing and content delivery networks (CDN) are both good ways to assist in improving capacity in the face of sudden, high demand for specific content but require preparation and incur operational and often capital expenditures. How about an option that’s free, instead?

While it’s certainly in the best interests of every organization to have a well-thought out application delivery strategy for addressing the various events that can result in downtime for web applications it may be that once in a while a simple, tactical solution will suffice. Even if you’re load balancing already (and you are, of course, aren’t you?) and employing optimization techniques like TCP multiplexing you may find that there are sudden spikes in traffic or maintenance windows during which you simply can’t keep your site available without making a capital investment in more hardware.

Yes, you could certainly use cloud computing to solve the problem, but though it may not be a capital investment it’s still an operational expenditure and thus it incurs costs. Those costs are not only incurred in the event that you need it, but in the time and effort required to prepare and deploy the application(s) in question for that environment.

Consider that you generally serve a fairly consistent patronage, such as would be the case for a local media outlet. No doubt you’ve got the infrastructure in place to handle the thousands of local visitors you receive on a daily basis, but what happens if a blog or editorial or news story is posted that catches someone’s eye? Often it’s relayed to Slashdot, or Digg, or Fark. And if it garners interest there, well, you may in real trouble and have a difficult time maintaining availability. You need a solution that can reliably handle just such a situation, but you can’t predict when that situation may arise. After all, “odd” or breaking news doesn’t often happen with any amount of notice. The budget to build out a larger infrastructure to handle a “could happen, might happen, can’t guarantee will happen” scenario is impossible to justify. 

What you need is a down and dirty, inexpensive (as in free) solution as an “insurance” plan against losing availability of your site. If that’s the case, perhaps what you need is to leverage the Coral Content Distribution Network.

I could describe it myself, but really the description offered up by the best source (the creators) says it far better than I could:

CoralCDN is a decentralized, self-organizing, peer-to-peer web-content distribution network. CoralCDN leverages the aggregate bandwidth of volunteers running the software to absorb and dissipate most of the traffic for web sites using the system. In so doing, CoralCDN replicates content in proportion to the content's popularity, regardless of the publisher's resources---in effect democratizing content publication. 
                                                                                                                                                     -- Coral Content Distribution Network | Overview

According to its Wikipedia entry, it is simplicity itself to take advantage of Coral Cache:

A website can be accessed through the Coral Cache by adding .nyud.net to the hostname in the site's URL, resulting in what is known as a 'coralized link'. So, for example, http://example.com becomes http://example.com.nyud.net. For websites that use a non-standard port for example, http://example.com:8080 becomes http://example.com.8080.nyud.net.

Basically you can leverage Coral to mirror a given host such that your site remains available in the face of an onslaught of traffic, and it’s free. What is not explained is how to get users to access your site via Coral Cache in an on-demand way, such as when a sudden spike in traffic would otherwise make your site inaccessible. Think of Coral as an on-demand, instantly provisioned content distribution network that will mirror your site and keep it available. All you need to do is take advantage of it.

Certainly if you know ahead of time you can create a link as described above and use it instead of your normal link, but it’s not always evident ahead of time that you’ll need the extra bandwidth/capacity and it would be nice if you could leverage such a solution on-demand. So what would be nice is a way to invoke these external services on-demand, in a way that’s not unlike the way in which caching solutions alter URLs, i.e. rewrite them, to take advantage of commercial content delivery networks (CDN).

There are quite a few ways to leverage such a service on-demand, but all require that you have some amount of visibility into the current operational state of your site and infrastructure. You can’t execute the logic necessary to take advantage of Coral if you don’t know you need it, after all. I’ll offer up three different ways in which you could integrate Coral into your availability strategy; there are many more, I’m sure. The methods included here require that you have a network-side scripting enabled solution at your disposal. If you’ve already got a load balancing solution, check with the vendor; it’s possible that you have the capability. If you don’t, you may want to consider using something like mod_rewrite that gives you similar capabilities, though you’d need to deploy the rules created on every server if you do that unless you create a proxy for your web servers and implement the rules there. That’s one of the advantages of a Load balancer/application delivery controller: it by nature virtualizes multiple servers and acts as a proxy for them, providing a single, centralized location in which to implement these kinds of solutions.

1. Maintenance Window Redirect
   Use case: During specific times of the week/day you’d like the ability to “take down” your servers for maintenance and you’d like to take them all down at the same time to reduce the time required to update/patch them all. In this case you’ll want to codify the times during which your servers will be unavailable and create a redirect (HTTP 302) to the Coral Distribution network as specified above, e.g. www.example.com.nyud.net
   
2. Referrer Based Redirect
   Use case: Generally speaking the chances of quickly being overwhelmed by traffic are directly related to where the requests are coming from, i.e. Slashdot, Fark, Digg. Thus to handle this scenario you’ll want to create a network-side scripting rule that examines the HTTP_REFERRER header and, if it matches one of the “oh-lord-we’re-about-to-get-hammered” sites, redirect to the Coral Distribution network.
   
3. Connection/Request Limit Redirect
   Use case: If you have a good idea what the total capacity of your servers is (and you do, because you’ve tested it under load, right?) then you can monitor current load on the load balancer/application delivery controller and upon nearing* those limits begin to redirect subsequent users to Coral. This solution requires a bit more intelligence and flexibility in the network-side scripting capabilities as you’ll need to track statistics, execute redirects based on variables, and end the redirection as requests slow down/decrease.

*The way that Coral works requires that it be able to access your site at least once to mirror it. Thus you cannot simply begin redirecting all requests to Coral without first allowing it to mirror the site by processing a request. This limitation necessarily requires that the network-side scripting solution you employ to implement such a solution be capable of allowing you to codify some amount of logic to allow this process to happen.

Okay, I lied – I’ll offer up a fourth option that requires no scripting and can be utilized without a load balancer:

4. Publish Coralized URI
Use case: If you’re publishing social media quick links on a story/blog/site, use the Coral-enabled URL instead of the origin content as the “link” to share. This won’t stop people from cutting and pasting from the address bar in their browser, but it will make sure that any “sharing” of the content immediately leverages the CoralCDN to distribute.

The reason I was leery of offering up the fourth option is because you lose visibility into statistics when users are directly sent to the CoralCDN. The other three options will be “counted” in logs and in statistics because they first connect to your site (the load balancer/application delivery controller) and then connect to the CoralCDN. Because the load balancer/application delivery controller is almost guaranteed to be able to handle more traffic than your servers, it can easily respond to requests with a redirect. But because it is responding it is counting the connections – and has all the relevant information about the client you might be aggregating - and therefore you don’t lose visibility.

If visibility isn’t an issue, then encouraging users to access the content directly via CoralCDN will certainly be one way to achieve the goal of keeping your content available.

There it is then; a free content distribution network that can be leveraged on-demand. Using CoralCDN is not a panacea and has limitations, of course, in that it’s not as flexible as cloud computing; it essentially mirrors your site, it doesn’t distribute it. But if it’s specific content that’s experiencing high demand and it’s not a normal occurrence, then a limited, tactical solution like CoralCDN may be just what you need to keep your site available and enjoy your 15 megabytes of fame.

Related blogs & articles:

• WILS: How can a load balancer keep a single server site available?
• How to use CoralCDN to help reduce bandwidth usage / server resources [mod_rewrite / mod_headers solution ]
• 3 Really good reasons you should use TCP multiplexing
• It’s 2am: Do You Know What Algorithm Your Load Balancer is Using?
• The Application Delivery Deus Ex Machina
• To Take Advantage of Cloud Computing You Must Unlearn, Luke
• Vertical Scalability Cloud Computing Style
• Putting a Price on Uptime
• Cloud Computing: Vertical Scalability is Still Your Problem 

One of the concerns with cloud bursting specifically for the use of addressing seasonal scaling needs is that cloud computing environments are not necessarily PCI-friendly. But there may be a solution that allows the application to maintain its PCI-compliance and still make use of cloud computing environments for seasonal scaling efficiency.

Cloud bursting, a.k.a. overdraft protection, is a great concept but in some situations, such as those involving PCI-compliance, it can be difficult if not impossible to actually implement. The financial advantages to cloud bursting for organizations requiring additional capacity on only a seasonal basis are well understood, but the regulatory issues that surround such implementations hinder adoption of this method to address cost-effective capacity increases when necessarily only for short periods of time.

But what if we architected a solution based on cloud bursting that offers the same type of advantages without compromising compliance with regulations and guidelines like PCI-DSS?

The ability to implement such an architecture would require that the PCI-compliant portions of a web application are separated (somehow, perhaps as SOA services or independently accessible RESTful services) from the rest of the application.

The non-PCI related portions of the application are cloned and deployed in a cloud environment. The PCI-related portions stay right where they are. As the PCI related portions are likely less heavily stressed even by seasonal spikes in demand, it is assumed that the available corporate compute resources will suffice to maintain availability during a spike, mainly because the PCI compliant resources have at their disposal all local resources. It is also possible –and likely – that the PCI-related portions of the application will not consume all available corporate compute resources, which means there is some capacity available to essentially reverse cloud burst into the corporate resources if necessary.

In a very simple scenario, the global server load balancer basically “reverses” the priority of data centers when answering queries during the time period in which you expect to see spikes. So all application requests are directed to the cloud computing provider’s instance first except for queries that require the PCI-compliant portion, which are always directed to the corporate (cloud computing perhaps) instance. This is basically a “cloud balancing” scenario: distributing application requests intelligently between two cloud computing environments.

The variations on this theme can become more complex and factor in many more variables. For example, you could set a threshold of capacity on the corporate data center instance that allows enough corporate compute resources available to handle the highest expected transaction rate and only burst into the cloud if the corporate capacity reaches that level. That’s traditional “cloud bursting.” You could also reverse the burst by dipping into corporate compute resources based on thresholds designated at the cloud computing provider’s instance to minimize the financial impact of utilizing a cloud computing provider as the primary delivery mechanism for the application. That would be “reverse cloud bursting.” The key is to ensure that no matter where the compute resources are coming from for the primary application components it does not negatively impact the availability and performance of the PCI-compliant processes executing in the corporate cloud environment.

Without the flexibility to deploy individual components of an application (a.k.a. services) into different environments these scenarios simply don’t work. Applications developed based on tightly-coupled frameworks and principles will never truly be capable of taking advantage of cloud balancing, bursting, or any architecture that relies upon specific components residing in a specific location because of regulatory issues or other concerns.

This is one of the core principles of SOA – separation of not only interface from implementation, but location-agnosticism. There are many ways to achieve this kind of location-agnosticism including on-demand generation of WSDL for client consumption that specifies end-point location based on the context of the initial request and the use of global server load balancing combined with context-aware application delivery. What’s vitally important, though, is the flexibility of the underlying application architecture and the ability to separate components in a way that makes it possible to distribute across multiple locations in the first place.

If that means SOA is the answer, then SOA is the answer. If that means a well-designed set of RESTful components, so be it. Whatever is going to fit into your organizational development and architectural practices is the right answer, as long as the answer includes “location agnosticism” and loosely-coupled applications. Once you’ve got that down the possibilities for how to leverage external and internal cloud computing environments is limited only by your imagination and, as always, your budget.

Related blogs & articles:

• SOA Announces Comeback Tour
• Use The Source, Luke!
• Cloud is Not a Big Switch
• Cloud Computing Makes Servers Obsolete
• Governance: Service Catalogs and the Cloud
• Have a can of Duh! It’s on me
• Forklifts, Rip and Replace, and Other IT Fairy Tales
• Intercloud: The Evolution of Global Application Delivery
• Cloud Balancing, Cloud Bursting, and Intercloud
• The Infrastructure 2.0 Trifecta
• The Context-Aware Cloud

Most people don’t start thinking they need a “load balancer” until they need a second server. But even if you’ve only got one server a “load balancer” can help with availability, with performance, and make the transition later on to a multiple server site a whole lot easier.

Before we reveal the secret sauce, let me first say that if you have only one server and the application crashes or the network stack flakes out, you’re out of luck. There are a lot of things load balancers/application delivery controllers can do with only one server, but automagically fixing application crashes or network connectivity issues ain’t in the list. If these are concerns, then you really do need a second server.

But if you’re just worried about standing up to the load then a Load balancer for even a single server can definitely give you a boost.

1. A modern load balancer, a.k.a. application delivery controller, can optimize TCP connections via TCP multiplexing. This will improve resource (RAM, CPU) utilization and increase the total number of concurrent users you can serve on a single server. In the face of a request onslaught, this one feature may be the difference between users  seeing “Connection Timed Out” and your content.
   
2. Offloading CPU intense operations like compression and SSL operations also improves capacity by letting your application spend time on application logic rather than ancillary encryption functions. Depending on the size and type of content and length of keys, this can net you a nice boost in not only capacity but also performance.
   
3. Applying security at the edge of the network before it gets to the server can alleviate a lot of painful processing that essentially results in nothing more than a rejection (or worse, a compromised site). Protocol layer security detects and mitigates DoS attacks, manipulation of protocols as an attempted exploit of the network, and other protocol related attacks. Rather than wasting server resources on these useless packets, a load balancer/application delivery controller can do it at the point of entry, thus improving the capacity of the server to handle legitimate requests.
   

Now it is absolutely true that what these techniques offer is a way to increase capacity, which may in most cases keep a site available. But there are always situations in which the load is just too much for a single server and in that case, you’re going to have to bite the bullet and either build out a cloud bursting architecture, invest in more servers, or move to a cloud environment. The good news is that these techniques work just as well for two or three or four (hundred) servers as it does for one.

There’s also the added benefit that if you do need to scale out and add a second (or third) server in the future that it can be done in a non-disruptive manner if you already have a load balancing/application delivery solution in place. Just add the server to the load balancer and voila! Scalability. It’s really that easy. If you don’t start with one you may have some network and/or server reconfiguration that needs to be accomplished and that can often result in the dreaded “D” word: downtime.

So if you were thinking that you didn’t need a load balancing solution because you only had one server, or that there’s really not much that can be done to improve the capacity of a single server and keep a site available, think again. There just might be a solution after all.

WILS: Write It Like Seth. Seth Godin always gets his point across with brevity and wit. WILS is an ATTEMPT TO BE concise about application delivery TOPICS AND just get straight to the point. NO DILLY DALLYING AROUND.

Related blogs & articles:

• Long Live(d) AJAX
• When Is More Important Than Where in Web Application Security
• I am wondering why not all websites enabling this great feature GZIP?
• It’s 2am: Do You Know What Algorithm Your Load Balancer is Using?
• WILS: Three Ways To Better Utilize Resources In Any Data Center
• WILS: Why Does Load Balancing Improve Application Performance?
• WILS: Application Acceleration versus Optimization
• All WILS Topics on DevCentral
• What is server offload and why do I need it?

The benefits of automation and orchestration do not come solely from virtualization.

Virtualization has benefits, there is no arguing that. But let’s not get carried away and attribute all the benefits associated with cloud computing and automation to one member of the “game changing” team: virtualization. I recently read one of the all-too-common end-of-year prediction blogs on virtualization and 2010 that managed to say with what I think was a straight face that virtualization of the network is what makes it “fluid”.

From: 2010 Virtualization Predictions - The Year the Network Becomes Fluid and Virtual

Virtualizing the network provides the similar benefits as server virtualization through abstraction and automation … The bottom line: In 2010, the network is going to become as fluid and dynamic as the data center is today.

The first problem with these statements is the separation of the network from the data center. The last time I checked the network was the core foundation upon which data centers are built, making them not only a part of the data center but an integral part of it. The second is implying that the automation from which a fluid network is derived is somehow achieved through virtualization. No. No, it isn’t. Both virtual and physical infrastructure require more than just being shoved into a virtual machine or automatically provisioned to enable the kind of benefits expected.

Infrastructure 2.0 is the enabling technology behind a fluid network,  no matter what the physical / virtual status of the infrastructure may be. Without the ability to integrate and then automate and include in data center orchestration network components it really doesn’t matter whether they’re virtual or not. Virtualization in and of itself provides little more  than abstraction and the ability to easily replicate (clone) itself. Even with the anticipated availability of VMware’s Redwood project, it is only the automation of the virtualization layer that appears to be addressed:

Presumably, VMware's Redwood would help partners like Savvis avoid these sorts of development efforts, especially as VMware evolves its platform over time. "There's an argument about how good and robust that [third-party] code is," the source said, "and how well it will work with future versions of vSphere right out of the box."

In particular, Redwood uses Lab Manager's network fencing technology to quarantine virtual environments "so there's no bleed-through," the source said, and VMware Orchestrator for automating the configuring and provisioning of a VMware cloud workload.

There’s little discussion of the inclusion of infrastructure – network and application network – components in the automation and orchestration, which makes sense given that there are today too many differences in API and methods of automating these components to presume VMware – or anyone else – will soon offer up a comprehensive solution.

The claim appears to be that virtualization magically enables networking components with the ability to configure themselves appropriately. At least that’s the way I read it, because in order for it to “reduce operational costs and increase security, manageability and scalability by abstracting and automating network services” in a volatile environment – such as public or private cloud computing – it would be necessary for the network components to adjust, add, and otherwise modify configuration in a way that is reflective of the underlying volatility inherent in the moving around of applications provisioned and released via virtualization. Too, the actual provisioning and subsequent configuration of infrastructure components will be highly specific to each cloud computing environment, and based on the unique processes required by each provider or organization. That’s not something that can come out of a box, off the shelf – unless organizations and providers are willing to have their processes prescribed by external vendors.

Yes, virtual infrastructure makes it easy to deploy a networking component anywhere you need it at the moment you need it, but it doesn’t do a thing to make sure it’s actually configured to do what it’s supposed to do when it launches. The only way that happens is through automation, which is not peculiar to virtualization. That’s something that’s enabled through APIs and integration, both of the virtualization technology and the infrastructure.

In the case of network infrastructure, components already enabled with a dynamic control plane can be scripted and integrated into automation and orchestration solutions to achieve this; virtualization of that same infrastructure adds no additional capabilities in this area at all. The layer of virtualization (abstraction) actually adds a layer of opaqueness into the architecture; a layer through which components must still be managed and integrated into the orchestration that will drive cloud and dynamic data centers. In fact, if you take a network component that is not enabled with a dynamic control plane via an API then it is no more able to address this challenge than its hardware counterpart. It is the existence of the API and its utilization that makes a network “fluid”, not its deployment form factor. It is the intelligence, the leveraging of the dynamic control plane, the collaboration that makes a network – hardware or virtual based – fluid.

Yes, virtual machines and hypervisor technologies also have an API through which … the virtual container can be manipulated. It can be stopped, started, and managed but that’s the container, not what application or infrastructure might be running inside that container. The automation enabled by virtualization is strictly management of what is the equivalent of the data center operating environment fabric. The benefits that come from automation of the actual infrastructure components, the orchestration of operational processes that eliminates redundancies and the need for manual execution, comes from collaboration of infrastructure through integration based on standards-based control planes.

The network will become fluid – I absolutely agree – but that metamorphosis will not solely because of virtualization.

Related blogs & articles:

• Putting the Cloud Before the Horse
• Cloud, Standards, and Pants
• Infrastructure 2.0: Squishy Name for a Squishy Concept
• Pursuit of Intercloud is Practical not Premature
• Virtualization Changes Application Deployment But Not Development
• Infrastructure 2.0 Isn’t Just For Cloud Computing
• Infrastructure 2.0 Is the Beginning of the Story, Not the End
• The Cloud Metastructure Hubub
• WILS: Automation versus Orchestration
• If You Focus on Products You’ll Miss the Cloud
• How do you get the benefits of shared resources in a private cloud?
• Virtual Network Infrastructure: Virtually Good Enough?

There’s been increasing interest in Infrastructure 2.0 of late that’s encouraging to those of us who’ve been, well, pushing it uphill against the focus on cloud computing and virtualization for quite some time now. What’s been the most frustrating about bringing this concept to awareness has been that cloud computing is one of the most tangible examples of both what infrastructure 2.0 is and what it can do and virtualization is certainly one of the larger technological drivers of infrastructure 2.0 capable solutions today. So despite the frustration associated with cloud computing and virtualization stealing the stage, as it were, the spotlight is certainly helping to bring the issues which Infrastructure 2.0 is attempting to address into the fore. As it gains traction, one of the first challenges that must be addressed is to define what it is we mean when we say “Infrastructure 2.0.”

Like Web 2.0 – go ahead and try to define it simply – Infrastructure 2.0 remains, as James Urquhart put it recently, a “squishy term.”

James Urquhart in “Understanding Infrastructure 2.0”:

Right now, Infrastructure 2.0 is one of those "squishy" terms that can potentially incorporate a lot of different network automation characteristics. As is hinted at in the introduction to Ness' interview, there is a working group of network luminaries trying to sort out the details and propose an architectural framework, but we are still very early in the game. [link to referenced interview added]

What complicates Infrastructure 2.0 is that not only is the term “squishy” but so is the very concept. After all, Infrastructure 2.0 is mostly about collaboration, about integration, about intelligence. These are not off the shelf “solutions” but rather enabling technologies that are designed to drive the flexibility and agility of enterprise networks forward in a such as way as to alleviate the pain points associated with the brittle, fragile network architectures of the past.

Greg Ness summed it the concept, at least, very well more than a year ago in “The beginning of the end of static infrastructure” when he said, “The issue comes down to static infrastructure incapable of keeping up with all of the new IP addresses and devices and initiatives and movement/change already taking place in large enterprises” and then noted that “the notion of application, endpoint and network intelligence thus far has been hamstrung by the lack of dynamic connectivity, or connectivity intelligence.”

What Greg noticed is missing is context, and perhaps even more importantly the ability to share that context across the entire infrastructure.  I could, and have, gone on and on and on about this subject so for now I’ll just stop and offer up a few links to some of the insightful posts that shed more light on Infrastructure 2.0 – its drivers, its requirements, its breadth of applicability, and its goals - to date:

• Greg Ness’ Virtualization, Clouds, and Meta Orchestration
  Greg walks though where we are, how we got here ,and what we need for the future.
  
• Christofer Hoff’s “Cloudanatomy” and subsequent Cloud Taxonomy
  Hoff’s ontology and taxonomy clearly shows just how large the problem space for Infrastructure 2.0 really is.
  
• James Urquhart’s Why virtualization is shaking up IT data centers
  James offers a great analogy that illustrates well exactly why it is that virtualization is one of the primary drives of the need for Infrastructure 2.0.
  
• And a few of my own offerings to the cause:
  
• Infrastructure 2.0 Is the Beginning of the Story, Not the End
  Understanding the means (Infrastructure 2.0) to the end (a dynamic infrastructure)
  
• Pursuit of Intercloud is Practical not Premature
  Fred Cummins of HP believes Intercloud is premature, but standards take time and much of the technology that will be applied using those standards already exists.
  
• Cloud, Standards, and Pants
  Why the creation of standards for Intercloud and Infrastructure 2.0 are so difficult
  
• Cloud Balancing, Cloud Bursting, and Intercloud
  How Intercloud and Infrastructure will move hybrid architectures that leverage cloud computing to achieve a variety of business goals
  
• Intercloud: The Evolution of Global Application Delivery
  The goal of Intercloud is essentially a dynamic infrastructure that comprises local and cloud-based services. It is an evolutionary process that relies heavily on global application delivery.

James believes "Infrastructure 2.0" will “evolve into a body of standards that will have the same impact as BGP or DNS” and I share that belief. The trick is going to be in developing standards that allow for the “squishiness” that is required to remain flexible and adaptable across myriad architectures and environments while being able to standardize how that happens.

Related blogs & articles:

• All Infrastructure 2.0 Topics on DevCentral
• Infrastructure 2.0 Blog
• Cloud Computing Requires Infrastructure 2.0 -- Seeking Alpha
• Infrastructure 2.0 (An oldie but a goodie)
• Server Den Asks Infoblox: What's Infrastructure 2.0?
• Infrastructure 2.0: The Modernization of the Datacenter – Doug Gourlay of Cisco
• Integration and Infrastructure 2.0