DevCentral Weblogs

Extend Cross-Domain Request Security using Access-Control-Allow-Origin with Network-Side Scripting

Lori MacVittie — Tue, 09 Feb 2010 12:18:36 GMT

The W3C specification now offers the means by which cross-origin AJAX requests can be achieved. Leveraging network and application network services in conjunction with application-specific logic improves security of allowing cross-domain requests and has some hidden efficiency benefits, too.

The latest version of the W3C working draft on “Cross-Origin Resource Sharing” lays out the means by which a developer can use XMLHTTPRequest (in Firefox) or XDomainRequest (in IE8) to make cross-site requests. As is often the case, the solution is implemented by extending HTTP headers, which makes the specification completely backwards and cross-platform compatible even if the client-side implementation is not. While this sounds like a good thing, forcing changes to HTTP headers is often thought to require changes to the application. In many cases, that’s absolutely true. But there is another option: network-side scripting. There are several benefits to using network-side scripting to implement this capability, but more importantly the use of a mediating system (proxy) enables the ability to include more granular security than is currently offered by the Cross-Domain Request specification.

HOW CROSS-ORIGIN ACCESS CONTROL WORKS

The basic premise of the W3C specification is that the server, i.e. the application, controls whether a remote location can send cross-site requests. Even though the request is actually made by the user (and we’ll get to how we might leverage that dynamically a bit later) it is the originating application – more specifically the domain - that must be authorized. In its simplest form, when a cross-origin request is received the application must respond with an Access-Control-Allow-Origin HTTP header containing a URI (or wildcard) that matches the value of the “Origin” HTTP Header. The browser then determines if the values of the two HTTP headers match and, if they do, the request is allowed to continue. If the value of the Access-Control-Allow-Origin does not exactly match the value of the Origin header – or does not contain a wildcard – the browser refuses to honor the response. The wildcard cannot be used in place of the host in a FQDN (Fully Qualified Domain Name), e.g. *.example.com, it can only be used to allow all domains to access the resource. While this is certainly the easiest way to enable cross-domain requests to be successful, it is not recommended because it is essentially the same as providing no security for the invocation of functionality from third-party domains.

Other variants of this scenario allow for the use of other HTTP methods (POST, DELETE, PUT) but require a pre-flight request to determine whether the method is allowed or not. The availability of these methods certainly forwards the REST API model as well as SOAP and makes it possible to develop web-based applications that can interact with multiple domains – think cross-cloud deployments – at the same time.

USING an INTERMEDIARY

Of course the need to (a) determine access control based on the Origin header and (b) add the Access-Control-Allow-Origin header requires that the application must change its behavior, i.e. code to support cross-domain requests. But as is the case with many forms of authentication and web application security, this process can be relatively easily implemented in a network-side scripting capable load balancer or intermediary, such as a mod_headers/mod_rewrite enabled Apache-based proxy. Regardless of whether you use a proxy capable of inspecting and transforming requests and responses or a Load balancer similarly enabled the process should be the same.

The advantages of using an intermediary are that you don’t need to change existing applications to support this functionality, and a centralized proxy-based solution can provide the functionality for all applications at the same time. A second benefit of this architecture over tightly-coupling with the application is that the application doesn’t need to process requests that are not allowed. If the proxy-based solution determines the request is not legitimate or authorized, the server never sees the request. This means the application – and by extension the server – don’t waste resources processing requests that are unauthorized, which improves the capacity of the server/application to service legitimate users.

A final advantage of this solution is flexibility. While the specification calls for determining authority to access a resource based solely on the origin, this can easily be extended to include other factors if the intermediary platform is capable of doing so.

For example, if you’re using a network-side scripting capable application delivery controller that is able to leverage GeoLocation information, you can use that information to determine authorization as well as the origin. You can use cookies, other HTTP headers, network information, time of day, and of course any data that might be submitted with the request – just about any data from the network up to the application can be included in the determination of whether the request should be granted or not.

APPLICATION DELIVERY EXTENDS VISILBIITY and REACH of APPLICATIONS

What the W3C provides is a framework for enabling the execution of cross-domain requests, but does not specifically detail how to determine whether any given request should be allowed or not. That is completely up to you. It is assumed by many that the only method of determining access rights is to keep a list of domains allowed to access resources via a cross-site request. This is simply not true. The use of additional HTTP headers as a means to allow or deny access makes the process dynamic and it is up to the developer to determine how access rights are derived. While the simplest case certainly uses nothing more than a list of domains, there are plenty of other ways in which access rights can be derived given an HTTP request. A context-aware application delivery controller used as the means by which such determinations are made can dramatically broaden the type of information upon which you can base that decision. You can extend the application’s reach, essentially, into the network. If you still want the application itself to authorize the request, you could still use network-side scripting to simply “screen” requests to determine whether they pass certain checks before being forwarded onto the server.

For example, the application delivery controller can inspect a request not only for the existence of the Origin header, but also the User-Agent. Based on the User-Agent the network-side script may be instructed to reject the request outright rather than allow it to be processed by the server. This may be desirable to prevent spiders and scripts – assuming they are capable of sending the Origin header - from accessing resources. It may be desirable to check for capabilities or language support first before forwarding on the request, to ensure the server-side application properly supports the request. This is also the best place to implement request throttling behavior, too, to ensure one client – or domain – does not consume resources at a rate that would degrade availability or performance for other clients – or domains.

This is “application delivery” as it applies to application architecture: the ability to leverage “network” and “application network” services to extend the reach and visibility of applications further to provide additional security and options that were heretofore unavailable due to inherent limitations in the application architecture.

Related blogs & articles:

Technorati Tags: MacVittie,F5,application security,security,web 2.0,network-side scripting,W3C,cross-domain request,HTTP,application delivery,context-aware,development,ajax,XMLHTTPRequest

Audio White Paper - A Green Architectural Strategy That Puts IT in the Black

DevCentral Interview — Tue, 09 Feb 2010 00:01:34 GMT

Green computing tends to focus on individual servers and devices, but even greater benefits can be realized with an architectural strategy that leverages green computing practices across the data center.

Get the Flash Player to see this player.

Technorati Tags: F5,DevCentral,BIG-IP,Application Delivery Controllers,Pete Silva

Scaling AJAX Applications is More About Architecture than Apache

Lori MacVittie — Mon, 08 Feb 2010 12:35:52 GMT

Scaling applications that include AJAX and non-AJAX components may require more than just tuning your web server

A common problem after deploying a Web 2.0 AJAX-based application shows itself through poor performance or lower capacity on the server, often both. Web serving tuning is almost always the first step in improving performance and capacity, but the inherently competing behavior of AJAX-requests and “normal” HTTP requests quickly becomes problematic as well. Tune for the AJAX requests and performance of regular old HTTP requests suffers. Tune for regular old HTTP requests, and performance of AJAX-requests suffer.

This is primarily because of the way in which the client-side application, the browser, interacts with the server. “Regular old HTTP requests” are typically those that GET a piece of content, static or dynamic, and that’s it. There may be many of these requests whenever a page (URI) is requested – all the images, client-side scripting files, style sheets, etc… – but they are not interactive. The browser requests them, receives them, and that’s it. AJAX-based requests, however, are inherently interactive. They are often automatically refreshed on an ongoing basis, on a prescheduled interval, or invoked by the user as they interact with the application. These requests are not “load and forget” like their traditional staticesque counterparts, but rather they are expected to be made often.

The overhead associated with opening and closing connections is well understood, and it is often the case that the web server configuration will be adjusted to meet the more demanding nature of the AJAX-based requests in an application. This is often accomplished by ensuring the KeepAlive setting (in Apache) is “on” and that the KeepAliveTimeout (in Apache) is high enough that AJAX-based requests occur before the timeout closes the connection. This allows the continued reuse of an existing connection between the browser and the server and improves performance. But it also ties up resources on the server keeping that connection open, which reduces the overall capacity of the server in terms of its ability to serve users. Optimally a short KeepAliveTimeout, if any, is best for non-interactive requests and often disabling KeepAlive actually improves performance for non-interactive applications.

Obviously these two behaviors are completely at odds with one another.

SOLUTIONS

There are a number of ways in which the competing needs and interests of the interactive (AJAX) and non-interactive portions of your web application can be addressed.

1. Configure two different servers: one to serve interactive content, i.e. AJAX-based requests, and one to serve non-interactive requests, i.e. everything else. This way, each server can be specifically tuned (and sized) according to the application behavior. This is beneficial for several reasons, including the ability to “scale out/up” only the interactive-serving functions when or if it becomes necessary. This can be achieved simply by using specific host names for specific requests. If you do not have a public IP address that can be assigned to each host, however, you’ll need a proxy, like a Load balancer, to sit in front of the servers and handle the direction of requests appropriately or you could use mod_rewrite to achieve a similar architecture. When a mediating solution like a load balancer is used to implement this solution, there are a several ways to achieve the behavior. One method is to rewrite requests directed at a specific URI, for example: http://www.example.com/ajax/request1.php would be redirected to the server designated as the “interactive” server while other requests would be forwarded to the non-interactive server. An application aware load balancer, i.e. application delivery controller, can examine the request itself and base the same decision on the actual data being exchanged. For example, many AJAX frameworks (XAJAX, SAJAX, Prototype, etc…) often use the HTTP POST method to send a request and use specific parameters such as “xjxfun” to indicate which function is being invoked on the server side. By examining the data being exchanged an application aware proxy (load balancer) can use that information to send the request to the appropriate server.

2. A second means of addressing the problem of resource depletion and performance with AJAX-based applications is to use a load balancing solution to mediate for the clients and employ the use of TCP multiplexing on the server-side to optimize resources. Because a load balancer is almost certainly capable of simultaneously handling a significantly higher volume of connections than a single web server, the competing behavior of interactive and non-interactive HTTP requests in a web application do not impede performance or impact its capacity. By allowing a load balancer to mediate for those requests, it can better manage the resources on the server and ensure that both capacity and performance are maintained. For every X client connections, the load balancer maintains only a fraction of X connections to the server and reuses them as the means to optimizing the server-side resources. This method is actually likely to increase overall capacity because it will reduce the number of connections required to be in use at any given time on the server(s) and eliminates the performance overhead associated with opening and closing TCP connections.

3. A third solution might be found in scaling up (beefier hardware) and leveraging virtualization. For web applications, specifically, it may be the case that virtualization of the application will actually improve performance. This is particularly true of I/O intensive web applications, but is also likely true of high-connection oriented applications as well. This is because as a web server begins to reach its capacity in terms of connections it requires more processing to “find” a given connection. Nearly all TCP-based applications exhibit similar performance characteristics and, upon reaching a certain threshold of connections, performance degrades. By finding the “sweet spot” ,i.e. the highest number of connections that retains acceptable user response time, and deploying multiple instances of that application, each tuned for that upper bound, it may be possible to squeeze out better performance and higher capacity of your web applications. Multiple instances will require a proxy, i.e. load balancing, solution as well, but this would allow for a “scale up” solution that takes advantage of a single, beefy physical server that eliminates the IT management and maintenance overhead of additional hardware in the data center.

IT’S THE ARCHITECTURE

In all three cases the solution to the problem of competing resource utilization between interactive and non-interactive components of a web application involve architecture. Some might believe that simply moving the application to “the cloud” would address the problems and, in some ways, it will. Cloud computing environments can indeed be managed such that applications are automatically scaled out to maintain performance and increase capacity, but the interesting thing about that is the environments are essentially implementing a combination of the three solutions heretofore presented. The bad news is that such a solution does not optimize resource utilization, and thus the costs associated with a cloud computing solution to the problem may be surprising and even prohibitive depending on your IT budget. And the cloud computing solution, of course, is ultimately also about architecture, as it is the architecture that allows for automated scalability.

In most cases involving web applications the answer to scalability challenges is going to end up being architecture, and that architecture is increasingly requiring the use of application network components such as load balancers to implement. This is why it is often advised that applications are architected to take advantage of application networking components from the beginning, even if such solutions will not be necessary to address capacity and optimization on day one. By architecting a solution that includes application networking as part of its design and deployment model, there is far less disruption later when such a solution does become necessary.

Related blogs & articles:

Technorati Tags: MacVittie,F5,architecture,web 2.0,virtualization,load balancer,AJAX,apache,mod_rewrite,cloud computing,application delivery,performance,scalability,optimization

ARX Config – Something NASty and An End to the Stumbling

Don MacVittie — Mon, 08 Feb 2010 10:57:40 GMT

Well, over the weekend one of our NAS boxes – the NetGear – started throwing SMART errors. Yeah, it was telling me that more and more blocks are going bad and we need to do something about it.

After due consideration (more below) Lori and I decided to replace it with a lower-end enterprise-class NAS.

Now this may sound like odd timing to you, but there’s something I haven’t told you. The NetGear was our tier two because it has a bad channel. It’s been running one disk shy for quite a while, and the problem is with the controller, not the disk – we tried replacing the disk right-off, only to discover that pre-NetGear versions of this box and issues with the first channel on the card. Lose another disk and POOF! No more tier two.

So we’re going to make the Seagate BlackArmor our tier two and place a shiny new Dell PowerVault NX3000 into our network. We picked the lowest end model they had that included CIFs, NFS, and ADS support in one box. Funny thing, neither Lori nor I has touched a PowerVault since we had a prototype in the NWC lab back when they were just starting the line up. Should be fun.

This is a “for us” thing, F5 isn’t subsidizing it in any way, and really shouldn’t be. Our NAS devices hold our stuff – our written works, pictures, PDFs we’ve purchased, even rips of our CD collection. This box is pretty, and we’re stoked, but with this box there is both good news and bad news…

You see, the Dell is a Dell, and it’s an enterprise product, so they don’t have one laying around that they can just ship to us, they have to put the disks in, test, etc. So it’s going to put this series off by another week. The good news is that once the box is here, I can sidetrack writing about configuring it and moving our network around, then we’ll be all set to actually talk about the cool things we hope to achieve with the ARX.

Until then though, I won’t be saying much. Let’s face it, I could play with the ARX for the week and tell you about all the switches I toggled, but you’re not going to use the box to play with, you’re going to put it in and tell it to manage your storage. So until I have the environment set such that I can do the same, it makes no sense to write about stuff that is fluff. In short, I’m not going to blog about stuff that doesn’t matter to you just so I can say I’m blogging.

So I’ll focus on other topics this week, and then you’ll get a flurry of updates when the new device arrives. The only thing I plan to do between now and then is rip down the ADS server (as in shut it off again), and make sure our Seagate plays nice via NFS, so all is set for this box to take the lead. Oh yeah, and back up both NAS boxes, so I can move the Seagate stuff onto the PowerVault, and the Netgear stuff onto the Seagate. So I guess I’ll be doing routine admin stuff, but nothing worthy of a blog unless something goes wrong and I think I can make you smile by blogging about it.

Until then, don’t get NAS-ty, be patient, we’ll be back.

Don.

DevCentral Live Tour – It’s a Wrap!

Jason Rahm — Fri, 05 Feb 2010 19:54:23 GMT

After a week of presentations throughout the Middle East and Europe by Joe & Jeff, I took my turn on the tour, beginning with a couple days in Johannesburg, South Africa, and finishing up the week with a few stops in Europe as well. Today’s session in Antwerp, Belgium, also featured the iRules Contest grand prize winner in the partner division, Sake Blok, with a fine presentation on writing clean iRules and a walk through of his winning iRule. Oh, and he delivered his presentation from his brand new 17” MacBook Pro—won in the content—just to rub in the fact that I do in fact not have one. Just kidding, Sake. It’s a really nice toy, by the way.

Anyway, I believe the presentations were well received (If I’m wrong about that, don’t tell Jeff!) The agenda was fairly broad spread, covering in part:

iRules & iControl basics
Advanced iRules tips & tricks
Case studies on iRules from some of this year’s iRules Contest winners
Case study on a similarly functional iControl script written both in Powershell & Python for comparison
New v10.1 features, including geolocation, tmsh scripting, the table command, etc.

Among Joe, Jeff, myself, and the hundreds of partners and end-users we met with these past two weeks, we have great feedback on product specific things as well as some constructive commentary on how DevCentral can be improved. To that end, we’re working feverishly in the shadows on deliver some improvements and new functionality to the DevCentral community. Stay tuned…

Tech Demo - LDAPS Configuration on BIG-IP Edge Gateway

DevCentral TV — Fri, 05 Feb 2010 17:44:00 GMT

Peter Silva and Joel Dujsik show how to configure LDAPS Authentication on the BIG-IP Edge Gateway with BIG-IP LTM functionality.

Technorati Tags: f5,f5networks,big-ip,edge,edge gateway,secure,secure access,consolidate,ssl vpn,remote,LAN,wireless,policy,control,LDAP,LDAPS,LTM,authentication,management

VM Sprawl is Bad but Network Sprawl is Badder

Lori MacVittie — Fri, 05 Feb 2010 12:02:29 GMT

We worry about VM sprawl but what about device sprawl? Management of a multitude of network-deployed solutions can be as operationally inefficient as managing hundreds of virtual machines, and far more detrimental to the health and performance of your applications. Turning them all into virtual network appliances that might need scaling themselves? That’s even badder.

But all you hardware fanbois best not smirk too much because the proliferation of hardware network devices is only slightly less badder than the potential problems arising from virtual network appliance sprawl.

WAIT, WHY IS DEVICE SPRAWL BAD AGAIN?

All the same reasons cited by various pundits since the virtualization craze began regarding the difficulties associated with virtual machine sprawl can be applied to virtual network appliance sprawl. For the most part it applies to hardware network device sprawl, too, for that matter.

1. Cost of IPAM (IP Address Management)

This is probably even worse than is often described by Greg Ness when it’s applied to network solutions as compared to virtual machines simply because most network solutions have at least two IP addresses assigned to them – one for management and one to do its job – if not more. There are exceptions, of course, as some solutions are deployed inline and transparently, but there are other challenges associated with such configurations as they often require port mirroring which effectively ties the solution to a specific port on a specific switch. Obviously moving it or scaling it out horizontally as a virtual machine would prove problematic for these solutions. So let’s just ignore those for the purposes of this discussion, shall we?

2. The impact on performance

Ignoring scalability – let’s assume a virtual network appliance is equal to the task for this post – the more points at which requests/traffic must stop and be processed the more latency is incurred. If you string together enough devices – regardless of the physical implementation – you are going to degrade performance. In some cases by a few milliseconds, in others perhaps by seconds. The amount of degradation relies heavily on the volume of requests, the type of processing being performed, and the capacity of each network device. Remember that the network is only as fast as its slowest hop, and that one poorly performing network device can destroy network and application performance.

3. Cost of management, power, and training

If you deploy five different network devices to address five different needs, you incur the cost of management, power, and training for each of them. This is true regardless of physical implementation as moving a solution from hardware to a virtual appliance doesn’t change the fact that it (1) needs to be managed, (2) has an interface/commands/quirks that need to be learned, and (3) consumes power.

4. Trouble with Troubleshooting (a.k.a. Lack of Visibility)

Even if every one of the X network solutions you have deployed individually has great visibility you’re still going to run into trouble troubleshooting. That’s because what one device may or may not do to a request/traffic isn’t easy to correlate by the time it’s passed through the fifth or sixth network device. It’s not as if all these devices add metadata that describes what they did to the traffic, they just do it and pass it along. The more devices, the more complicated this process becomes.

5. Special Issue with Virtual Network Appliances: Distributed Management

Remember how you didn’t want to shell out the extra cash for the vendor-specific distributed management solution? If you’re scaling out a network solution via multiple virtual network appliances you may want to reconsider that decision. Once you get past a couple of instances you’re going to need something to help you manage them and keep their configurations in synch or you’re asking for trouble. And don’t forget about the hypervisor management system, too. You’ll need that, I’m sure.

Sprawl of any kind incurs costs per node at a fairly consistent rate. Every instance – physical or virtual – adds to the combined total cost of ownership and investment in time. Every device through which traffic must flow also incurs a performance penalty, which to the business stakeholder is probably more dangerous than the hit on your budget.

UNIFIED APPLICATION DELIVERY INFRASTRUCTURE

Unified application delivery infrastructure can’t completely eliminate every other network device because generally speaking some network devices aren’t focused on application delivery but are instead wholly focused on network security or compliance or business functions that really have very little to do with managing networks or delivering applications.

Yeah, I know. Surprised me too when I found that out. There are actually solutions that aren’t focused on network or application networks. Whodda thunk it?

But for application delivery focused solutions – acceleration, optimization, caching, application security, load balancing – the solution to the problems of network device sprawl are unification onto a single, extensible (modular) platform. And while many network folks hear “modular” and think “chassis” (and that can be one approach) I’m talking about the core system itself. The solution, not the container.

By sharing a common core networking platform, a unified application delivery infrastructure mitigates the problems associated with extra hops/stops in the flow of requests/traffic by eliminating them. Requests that need to be passed through a web application firewall before being passed to a Load balancer do so, but because the common core networking platform is shared there’s no network or network stack overhead incurred by the passing of the data.

Network sprawl really is badder than VM sprawl because it not only increases the overall cost to deliver and secure applications but it can also negatively impact the performance and reliability of applications. A unified platform affords choice in the ability to add functionality as needed, to try out functionality to see if it’s worth it, and to scale out in a more efficient way on an as-needed (on-demand) basis.

One of the reasons virtualization is so appealing is it addresses nicely the “lots of little boxes” problem that causes management headaches throughout the data center. Consolidation through virtualization was the answer to that one, at least in terms of the sprawl associated with the physical devices. Unified infrastructure addresses the same “lots of little network boxes” problem that causes similar headaches on the network and application network side of the data center by consolidating many of the application delivery focused functions onto a single, shared and extensible application networking platform.

Related blogs & articles:

Technorati Tags: MacVittie,F5,application delivery,unified application delivery and data services,infrastructure,unfiied infrastructure,application security,caching,acceleration,optimization,load balancing,web,internet,blog,virtualization

ARX Config – Week Three

Don MacVittie — Fri, 05 Feb 2010 05:47:27 GMT

Well, I’ll bet you’re wondering how it’s going?

First, the reasons for my silence that you haven’t heard. Last Thursday my wonderful Dell Latitude D820 died. I loved this machine, thought so much of it that last time I updated my home machine I got a D830. But sadly, it was over three years old, and I spend 8+ hours a day abusing it, so no surprise.

The warranty ran out in December, so that left me (IT actually) no option but to replace it. The real reason to include this is to point out to you that F5 IT rocks, and many IT departments could learn from them. I’m a remote worker, I was limping by working on my home machine which had most of what I needed, but some key software like MS Project wasn’t installed, and webmail is… painful in any situation.

I told IT the machine was definitely dead late on Tuesday, on Wednesday I had a new machine. With my login info and the licensed corporate software installed. You don’t do any better than that.

So they sent me a Latitude E6400, and honestly, I’m pleased as can be. The only little problem I’ve had with it was (so far) not work related. I listen to DVD lectures from The Teaching Company in the evenings while working or painting or writing for non-work, and for some reason my newest set of DVDs plays fine on the machine but doesn’t have sound. Local WMV files play and have sound, the DVDs work on my home machine… So I don’t exactly know what’s going on there, but everything else works perfectly, so I’m happy. I’ll figure out what oddity makes them work on other Dell Machines and not on this one.

And I was complaining that I was out of space for VMs… No more! Much larger hard disk.

Anyway, you can imagine that getting the machine, pulling the hard disk from my old one (don’t tell IT, I’m not certain they’d approve), hooking up the disk via USB and dumping all the important stuff, reconfiguring just about everything – from bookmarks to networking settings – to work well in my environment sucked up just a bit of my time.

On the bright side, the days that I had no machine (it was nearly a week because we’d hoped we could fix the old one, but alas, Dell said “motherboard, fixing is a bad choice”) gave me a chance to get my storage house in order.

What did I do? Well I wiped the box running ADS and started over. It had ADS and DNS installed from who-knows-how-long-ago, but it was shut down… So I tried with the installed copies, but wasn’t real confident and it wasn’t working the best.

So I wiped the server and reinstalled, set ADS up again, joined my home laptop to the ADS domain, then worked at getting the storage into the domain. One required using the WINS name instead of the domain name to get it to work, the other required that I add it by had to ADS and DNS, and THEN tell the storage to join the domain. And as usually happens in that case, all went well.

Finally a chance to join the ARX to the domain. This is something I had not attempted up to that point because I wanted to have the things an ARX requires – storage and users – in ADS so that once it was joined I could get rolling. So I went to join the ARX into the domain… And realized I did not have the faintest idea how to do so.

AD Forest/domain list.

RTFM time, so I went and looked. The help on the system I have is very nice, and coworkers tell me that the help on DMOS 5.X is indeed very nice overall. That helped me get rolling, as did the logs, which are very verbose and I cannot recommend enough. In fact, all the oddities I’ve encountered to date – failure to access disks for metadata, failure to connect to shares, failure to negotiate NFS versions… All were ultimately the fault of my storage, and all we ultimately made clear to me via the ARX logs.

Lots of logs – and this does not count the automatically generated reports for lots of common activities.

The Managed Volume created under ADS.

One really odd thing I ran into that I am working around by ignoring it – because I can – is that I have a Namespace whose drive mountings failed – a leftover from the work I was doing in NFS and CIFS without Active Directory. It is stuck in the “starting” state, and I can’t get it out. Since the ARX won’t let me delete it, I’m ignoring it for now, and need to look up how to point out to the ARX that it will never finish starting since it has no volumes allocated to it. I’m pretty certain that this is a user error, so don’t judge the ARX poorly, even if it is an ARX error, you can ignore a single Namespace easily enough. Or better, don’t use SMB class storage so you’re not jerking the poor ARX around for three weeks ;-).

A Virtual defined on the ADS domain Internal.

Everything before that last picture that I’ve talked about has been the backend. Now that all the backend pieces were working together, it was time for me to set up the user-facing bit… The Virtual Service. This is the presentation “volume” – where the device advertises the Virtual Directory Tree to the network. It went easy enough on creation and making CIFS exports, and it’s up and running now.

The problem I’m stopped at now is another RTFM – I need to join the Virtual to the domain, but haven’t read how – it told me that I needed to and how to do so when I created the exports on the Virtual Service, but it was 3am and I thought “I’ll figure that out later…” And indeed I will, for this blog is long enough, and that’s where I’ll pick up the next installment.

Until then, enjoying my new laptop and seeing this all working together.

Oh yeah, and I have to make my regular everyday user not be SuperADSMan. I toggled him up to Enterprise ne’er-do-well while testing, and don’t want to forget to make him normal, and create the storage background users I need. More on that next time, when I’m sure what storage background users I want/need.

Don.

DevCentral Weekly Roundup Episode 121 - Mr Miyagi Says No IE6

DevCentral Weekly Podcast — Thu, 04 Feb 2010 23:26:45 GMT

Welcome to the one hundred and twenty first edition of the DevCentral Weekly Roundtable Podcast! A weekly recap of the interesting things that have been going on within the DevCentral community.

Hosts: Colin Walker, Don Mac Vittie, Jason Rahm, Jeff Browning, Joe Pruitt, Scott Koon.

In this weeks podcast we did a trip report on Jeff and Joe's trip through Europe last week talking to DevCentral users and partners and were lucky enough to get Jason on the line from Antwerp Belgium to give us a overview the first few days of his trip. We finished things up with some DevCentral content of the week including a video with iRule Contest finalist Henrick Gyllkrans, two posts on 64 bit numbers in perl, iRule-No-Snat, Geolocation iRules, Returning 1x1 gif's from iRules, and Scott's favorite iRule of all time: Get Out Of IE6.

All of the links from this recording can be found with the dcpodcast121 tag at Delicious.com. You can also watch the video recording from our UStream.TV channel: DevCentral Podcast 121.

Get the Flash Player to see this player.

Technorati Tags: F5,DevCentral,iControl,iRules,Geolocation,Henrick Gyllkrans,Perl,IE6,EMEA,Colin Walker,Don Mac Vittie,Jason Rahm,Jeff Browning,Joe Pruitt,Scott Koon

Interview with iRule Contest Finalist Henrik Gyllkrans

DevCentral TV — Thu, 04 Feb 2010 19:00:40 GMT

Jeff and Joe take some time after our presentation in Stockholm to talk with iRule Contest finalist Henrik Gyllkrans about writing fast iRules, his company Advanced IP, and his winning cookie tampering iRule.

Technorati Tags: F5,DevCentral,iRules,Stockholm,Henrik Gyllkrans,Joe Pruitt

The Question Shouldn’t Be Where are the Network Virtual Appliances but Where is the Architecture?

Lori MacVittie — Thu, 04 Feb 2010 12:43:15 GMT

We seem on the verge of repeating the mistakes associated with failed SOA implementations: ignoring the larger issue of architecture.

Everyone – from pundit to public – is asking the same question: “Where are the network virtual appliances?” But fewer people seem to be asking a question that needs to go hand-in-hand with that one: “Where are the architectural guidelines to support deployment of network virtual appliances?” SOA has been deemed by many to be a failure in part because it lacked true architectural guidance. Architects were simply unable – whether by lack of skills or training or lack of support from the rest of the organization – to design an architecture that took advantage of services and thus the result was often little more than “service sprawl.” Services did not scale well, they were not so easy to integrate, and no one really had a good handle on what services were available, and where.

Lack of an architectural strategy to accompany a network virtual appliance will likely lead to the same end: network sprawl and a lack of scalability or worse – scalability that’s costly in terms of expenses and resources.

Rich Miller, who’ll be joining a panel of other industry notables at Cloud Connect to discuss Infrastructure 2.0 and what’s necessary to successfully move forward with these “new” infrastructures, may have inadvertently pointed out the lack of architectural guidance related to virtual network appliances when he said:

If a vendor is going to sell network virtual appliances, the nva's should be designed from the get-go to be scalable (both 'up' and 'out'), and designed with the notion that the 'appliance' is not just a physical appliance without the box. That is 'horseless carriage' product design, which casts new technologies in exactly the same roles as their precursors.

What Allan doesn't say is that this may require the wider deployment of network infrastructure designed specifically for virtualized appliances and converged IO. It's not just whitebox, commodity x86 hardware running general purpose virtual machine environments for server virtualization.

-- Rich Miller in “Where ARE the Network Virtual Appliances?”

Rich is focusing more on internal design in general, but any such “design” must also necessarily include how the VNA scales in the target environment. Scalability is at the heart of all definitions of cloud computing and without the ability to scale solutions – whether application, network, storage, or application network – any such implementation will almost certainly be deemed a failure.

SCALING UP

Scaling up, i.e. vertical scalability, in a cloud computing or virtualized environment is in essence little more than “throwing more hardware” at the problem. Scaling “up” adds more compute resources, yes, but it is not “on-demand” today because it effectively requires re-provisioning of large chunks of resources. Cloud computing and virtualization in particular today are not capable of simply “adding on” more CPU or RAM to a virtual machine and even if it were there are hard, physical limitations imposed by the underlying hardware on the upper bounds of such a strategy.

Scaling “up” a virtual network appliance in practice is really no different than scaling up hardware. It leads to over-provisioning by necessity and in the event that capacity and physical constraints are reached, requires provisioning a new, higher capacity instance which while easier than upgrading hardware counterparts still requires much the same process in terms of deployment.

While I agree with Rich’s assessment that virtual network appliances should be designed to scale up as efficiently as possible, that doesn’t change the challenges associated with actually scaling up the solution in a dynamic environment or that it’s not all that much different than what we do today to try to future-proof the sizing of solutions.

SCALING OUT

Scaling out, i.e. horizontal scalability, is usually the more desirable choice in these discussions. This makes a great deal more sense even though scaling “out” is still essentially a “throw more hardware at the problem” solution, it’s a more temporary “toss” and is more flexible in terms of growing capacity on-demand. It’s certainly more efficient and agile to deploy another virtual network appliance than it is to acquire and deploy another physical network appliance.

The problem with this approach is not in the details. It’s in the broader architectural strategy applied to the process, which today is virtually non-existent. Scaling out is a proven method of addressing capacity constraints. We do it all the time with web and application servers, with firewalls, with XML gateways. load balancing as a method of implementing a horizontally scalable application and network infrastructure is nothing new and it is indeed efficient, scalable, and architecturally sound.

The issue is with how one scales out, and what. The call for “virtual network appliances” in general ignores the architectural implications in favor of some perception of increased flexibility and scalability. There are simply some functions within the data center that would not benefit from being “virtualized” and others that will not benefit without a strong set of architectural guidelines. Some functions should never be virtualized because such an architecture would not be feasible to implement and would do more harm than good to both network and application performance.

Let’s take core routing, for example. One of the reasons you’d want to “scale out” a core router is because it has hit an upper constraint on bandwidth. Perhaps it’s only capable of handling 10Gb of aggregate bandwidth entering the data center/cloud computing environment but you need to handle 20 or 30Gb of bandwidth. In a completely virtualized architecture you’d just scale “out” by adding another another virtual router, right? That will certainly increase aggregate bandwidth capacity, but fails to address a very important question: how is traffic directed to one instance or another? Do we have to scale the scalability? And if so, how does that work? Do the core routers deploy in an active-active configuration, both masquerading as the entry point into the data center? Sharing of “bogus” MAC addresses across active-active-n scaling architectures is the most common solution to this problem, but introduces others related to failover and network utilization. That latter piece is due to the natural behavior of switches and reliance on MAC address/port affinity; essentially this solution turns a switch into a giant hub, replicating data/traffic across all possible ports on which the “bogus” MAC address might be. As you scale out, more and more bandwidth will be consumed by this broadcasting behavior and can make troubleshooting more difficult, especially in environments where visibility is already limited such as cloud computing providers.

RIGHT BACK WHERE WE STARTED

Is it the case that every virtual network appliance capable of being “scaled out” will essentially need to be capable of acting like a Load balancer? Because that’s how it looks from here. Horizontal scalability is based on the premise that something – some device, some solution – is load balancing requests/data/traffic across the multiple instances. Without the load balancing solution, such implementations are nearly impossible to achieve. So imagine the potential issue when the load balancer is virtualized, too. It, also, must scale “out” and thus must be “scaled” itself by … a load balancing solution. Such an implementation is certainly achievable, but also requires that the “primary” load balancing solution is scaled “up” in order to handle the aggregate request/data/traffic being directed at the infrastructure. Limitations on vertical scalability return us right back to a solution based on horizontal scalability, which puts us right back here where we are: how do we scale out the “more scalable” virtual network appliances that are so highly in demand?

We haven’t even touched the large problem of sprawl in a virtual network infrastructure. Management systems aren’t quite ready for such an implementation, and ironically part of the reason cloud computing, virtualization, and infrastructure 2.0 are coming of age now is because we have issues with managing an increasing volume of servers, applications, devices, and IP addresses across the data center. Deploying an infrastructure comprised of virtual network appliances without a strong architectural strategy and a supporting management strategy is sheer folly, and puts us no better off than we are today.

We should be very careful to ask ourselves why we want a particular solution in a network virtual appliance and how it might impact the network and management of the network before we blithely toss it into our critical network and application network infrastructure. Architecture is inherently as important when designing any type of distributed system, and when moving from hardware to distributed software as a means to achieve scalability there needs to be more a lot more thought and strategy put into the process.

While there are certainly going to evolve architectures that take advantage of virtual network appliances, and traditional hardware appliances, and combinations thereof, we need to tread carefully forward and ensure that our driving desire for what appears to be flexibility doesn’t end up breaking the backbone of the data center: the network.

A well-thought planned architectural strategy for integrating virtual network appliances with traditional data center components will go a long way toward ensuring maximum flexibility without stretching the network so tightly that it breaks.

Related blogs & articles:

Technorati Tags: MacVittie,F5,virtualization,architecture,cloud computing,infrastructure 2.0,infrastructure,load balancing,routing,scalability

Consolidate and Dedicate to Eradicate

Pete Silva — Wed, 03 Feb 2010 21:30:16 GMT

Whether it be due to cloud computing, last year’s economic mess, or just the general cyclical nature of the Tech Industry, Consolidation has been a huge focus of IT departments of late. Data Center consolidation, hardware consolidation, staff consolidation and tech sector consolidation to name a few. I remember the days of single purpose boxes that did one thing well. In fact, a decade ago at Exodus, that was one of my positioning points for BIG-IP over such LB units as Alteon, ArrowPoint and LocalDirector since they were switched/hardware-based appliances. I’d say something like, ‘It’s a Floor Wax and a Dessert Topping while the BIG-IP is software based, focused only on Load Balancing.’ Boy, times have changed.

Single purpose appliances, while still big business for their particular specialty, are becoming fewer and fewer – just look at the handheld your using. The printer was one of the first to go that route becoming printer/copier/fax/scanner in an effort to make them more useful and appealing to the customer. Ads tout, ‘No more bulky equipment to buy – it’s all here in this great new thing that you must have!! All for the incredibly low price of…..’ IDS graduated to IPS and now we have IDPS units and UTM (Unified Threat Management) systems or the Next-Gen Firewalls. They have firewall, anti-virus, spam controls, web filter, IDS and more. We are in a multi-task society and expect our devices to behave the same. For a while, adding more and more functionality to a piece of IT equipment would either slow it to a crawl or make it very difficult to troubleshoot. The processing power available today allows multi-function appliances to dedicate resources to ensure all the functions run smoothly.

Having multiple point solutions, interfaces and GUIs also makes it difficult to manage the various entities, especially if it’s a security device. Managing multiple points of entry and enforcing a consistent security policy across the board can be challenging. You got users connecting and requesting application access via VPN, some over the air on Wireless and others hooked right to the LAN. They also are probably using various types of computing devices; from IT issued laptops, to home/personal machines to mobile devices. You might have a specific policy for each type of access method/device or you enforce the same security, no matter what the connection. Why wouldn’t you do a host check on LAN users similar to the scrutiny your remote users must pass? In many cases, that might involve a NAC type controller and I thought we were trying to reduce the number of power suckers in the data center. Today, IT needs a single management interface and policy enforcement point that’s easy to navigate and quick to deploy. During a crisis, like a potential intrusion or breach, you can waste precious time trying to get to all the different appliances to assess the situation.

As consolidation continues, and more functionality is added to these multi-dedicated appliances, management of such an infrastructure especially if it’s part of a cloud, will continue to be an important driver for IT. So, as you consolidate and are able to dedicate, that will enable you to eradicate costs, multiple management interfaces, multiple point products and with the right device, eradicate many of the threats that appear every day, the CDE way!

Related resources:

In 5 Minutes or Less Video: Consolidate Access with BIG-IP Edge Gateway
BIG-IP Version 10.1: An Integrated Application Delivery Architecture [Whitepaper, PDF]
Unified Access and Optimization [Whitepaper, PDF]
F5 Delivers Next-Generation Application Delivery Services Giving Enterprises More Control with Context-Aware Networking [Press release]
BIG-IP v10.1 Now Available

External articles:

Technorati Tags: F5,BIG-IP,v10.1,Edge Gateway,WOM,application delivery,Pete Silva,F5,security,application security,network security

WILS: SSL TPS versus HTTP TPS over SSL

Lori MacVittie — Wed, 03 Feb 2010 12:10:34 GMT

The difference between these two performance metrics is significant so be sure you know which one you’re measuring, and which one you wanted to be measuring.

It may be the case that you’ve decided that SSL is, in fact, a good idea for securing data in transit. Excellent. Now you’re trying to figure out how to implement support and you’re testing solutions or perhaps trying to peruse reports someone else generated from testing. Excellent. I’m a huge testing fan and it really is one of the best ways to size a solution specifically for your environment.

Some of the terminology used to describe specific performance metrics in application delivery, however, can be misleading. The difference between SSL TPS (Transactions per second) and HTTP TPS over SSL, for example, are significant and therefore should not be used interchangeably when comparing performance and capacity of any solution – that goes for software, hardware, or some yet-to-be-defined combination thereof.

The reasons why interpreting claims of SSL TPS are so difficult is due to the ambiguity that comes from SSL itself. SSL “transactions” are, by general industry agreement (unenforceable, of course) a single transaction that is “wrapped” in an SSL session. Generally speaking one SSL transaction is considered:

1. Session establishment (authentication, key exchange)

2. Exchange of data over SSL, often a 1KB file over HTTP

3. Session closure

Seems logical, but technically speaking a single SSL transaction could be interpreted as any single transaction conducted over an SSL encrypted session because the very act of transmitting data over the SSL session necessarily requires SSL-related operations. SSL session establishment requires a handshake and an exchange of keys, and the transfer of data within such a session requires the invocation of encryption and decryption operations (often referred to as bulk encryption).

Therefore it is technically accurate for SSL capacity/performance metrics to use the term “SSL TPS” and be referring to two completely different things.

This means it is important that whomever is interested in such data must do a little research to determine exactly what is meant by SSL TPS when presented with such data. Based on the definition the actual results mean different things. When used to refer to HTTP TPS over SSL the constraint is actually on the bulk encryption rate (related more to response time, latency, and throughput measurements), while SSL TPS measures the number of SSL sessions that can be created per second and is more related to capacity than response time metrics. It can be difficult to determine which method was utilized, but if you see the term “SSL ID re-use” anywhere, you can be relatively certain the test results refer to HTTP TPS over SSL rather than SSL TPS. When SSL session IDs are reused, the handshaking and key exchange steps are skipped, which reduces the number of computationally expensive RSA operations that must be performed and artificially increases the results.

As always, if you aren’t sure what a performance metric really means, ask. If you don’t get a straight answer, ask again, or take advantage of all that great social networking you’re doing and find someone you trust to help you determine what was really tested. Basing architectural decisions on misleading or misunderstood data can cause grief and be expensive later when you have to purchase additional licenses or solutions to bring your capacity up to what was originally expected.

WILS: Write It Like Seth. Seth Godin always gets his point across with brevity and wit. WILS is an ATTEMPT TO BE concise about application delivery TOPICS AND just get straight to the point. NO DILLY DALLYING AROUND.

Related blogs & articles:

Technorati Tags: MacVittie,F5,SSL,HTTP,performance,metrics,TPS,testing,WILS

Introducing: Long Distance VMotion with VMWare

Nojan Moshiri — Tue, 02 Feb 2010 22:28:35 GMT

It seems like I blinked and 2009 went by, but in that time I've been working on so many interesting projects at F5, I have a backlog of information to share with the community. The first post this year is about the long distance VMotion with VMWare's ESX system. This is a solution that enables the movement of live running virtual machine hosts from one data center to another.

The main problems in routing VMotion between data centers are latency, bandwidth, client traffic and security. In BIG-IP 10.1 we have a solution that compresses, encrypts and shields the ESX servers from prevailing WAN conditions, to enable long distance motion of running hosts. Take a look at the following screencast to see how this works:

In the chart below are some of the typical improvement times we see with long distance VMotion with BIG-IP. When latency goes up, VMotion is often not possible without BIG-IP in place. For example, with 100 ms of round-trip latency, on an OC3, a virtual machine that has one gigabyte of active RAM memory, takes roughly three and a half minutes to migrate across the WAN. If you were to try the same VMotion without BIG-IP in place, it would take more than 13 minutes and only succeed about half the time.

I'm excited about the types of architectures that can be enabled with this kind of solution in place. F5 is laying the ground work to make some exciting infrastructures possible

Have a look at the F5 deployment guide which describes how to set this solution up and how to architect new solutions across your data centers: http://www.f5.com/pdf/deployment-guides/vmware-vmotion-dg.pdff

Technorati Tags: vmware,VMotion,Long Distance,iSessions,GTM,BIGIP-LTM 10.1,BIGIP-WOM

Alice in Wondercloud: The Bidirectional Rabbit Hole

Lori MacVittie — Tue, 02 Feb 2010 11:36:22 GMT

Emerging architectures are conflating responsibilities up and down the application stack. Who is responsible for integration when services reside in the network?

While preparing for an upcoming panel I’m moderating at Cloud Connect (in the “New Infrastructure” track), the panelists and I had a great discussion on the topics we wanted to discuss in the session. During that discussion it became increasingly clear that an interesting phenomenon has been occurring: the conflation of network and application responsibilities in the traditional “stack.”

Much of this inversion is absolutely necessary for emerging models of networking and computing to be successful. Traditional methods of handling QoS (Quality of Service) and identity management, for example, are no longer adequate in the inherently volatile world of cloud computing and dynamic networks. Interestingly, the driver behind the inversion appears to be based largely on the ability of specific layers access to context, which is necessarily replacing IP addresses as a method of client – and server – identification.

CLIMBING UP the RABBIT HOLE

Back in the day, QoS was a class of problem unto itself, with an entire market of products and solutions developed specifically to address the challenge of prioritizing traffic. Initially it was thought that the ToS (Terms of Service) bits in the IP header would suffice, but it quickly became obvious that this required every organization and provider to honor those bits as traffic flowed through and across the Internet.

Didn’t happen.

A market emerged that moved QoS “up the stack” to Layer 4 (transport protocol). A class of devices were deployed that employed either TCP rate shaping or packet queuing technologies to control the amount of bandwidth a given “application” could consume. It quickly became apparent that this method was not robust enough as more and more “applications” began to use the same protocol: TCP. The devices again moved “up the stack” to Layer 7 (application) and began to apply QoS policies based on actually identifying applications based on layer 7 protocols and data characteristics.

In recent years even this has become inadequate because these techniques were all focused on limiting, in some way, total bandwidth for an application. While these solutions were also able to, albeit rudimentarily, accomplish rate shaping on a per-user basis, they still focused on bandwidth as their metric of choice to control. Hence a single user could be limited to X Kbps for all HTTP traffic, and further limited to Y percent for application A and Z percent for application B, but bandwidth as a meter of usage for applications today is not an appropriate measurement.

Hence, QoS has again moved up the stack and is more granular than ever. Rather than worrying about bandwidth, which has grown increasingly cheap and available for both organizations and users, QoS now concerns itself with limiting requests on a per-user basis and, in some cases, a per-client-type basis. Consider Twitter’s rate limiting implementation for its API. This is a modern implementation of QoS that attempts to equalize access to its services for all users, effectively ensuring a consistent quality of service for everyone. Bandwidth is not a factor, because the amount of bandwidth consumed by any given client is highly variable and based on what data is being requested.

Similarly we often see requests for ways in which application usage can be limited based on application layer variables, with nary a mention of bandwidth. It’s always about users and usage patterns of a specific application.

What was once a “network” function, QoS, has moved “up the stack” and is now primarily the responsibility of the “application.”

SLIDING DOWN the RABBIT HOLE

It wouldn’t be an inversion of responsibility if traditionally “application” layer responsibilities weren’t being similarly pushed “down the stack.” A good example of how this is occurring today is in the area of “identity”, which traditionally includes authentication and authorization.

In the early days of web applications, identification was based on a user name and password (sometimes IP address, sometimes a combination thereof) and was expected to be handled by the application. After all, the application knew what users should be allowed and thus is was the demesne of the application to provide those mechanisms. The use of .htaccess files was widespread as a means to achieve this functionality.

But as technology began to merge the world of the web with the internal world of IT, it became increasingly common to leverage external applications as an identity store and the means by which users were authenticated and authorized to access applications. LDAP, Active Directory, RADIUS, DIAMETER. These protocols resided somewhere between the application layer and the transport layer and provide the data necessary for applications to make access decisions.

But again, this method has run into obstacles in adapting to volatile and large environments. Scalability and the need to execute complementary access policies the network layer in authentication and authorization decisions has continued to drive identity and authentication and authorization “down the stack” and into the “network”. In a highly scaled environment, for example, it is often preferable that an intermediary Load balancer authenticate users to an application because it is increasingly painful for developers to tightly integrate application access and security policies into the application. Traditional methods are brittle, static designs that are increasingly tossed out in favor of more policy-based access that resides somewhere “in the network” rather than tightly-coupled with the application.

What was once an “application” function has moved “down the stack” and is now increasingly the responsibility of the “network.”

WHAT DOES IT PORTEND?

The conflation of responsibilities up and down the “stack” point to either an increasingly flattened application architecture comprised of services; services that may reside in the application layer or the network layer, but are leveraged by both in approximately the same way.

This is actually much of the brouhaha behind Infrastructure 2.0; behind the evolution of the network to become “smarter” and more “integrated” with the rest of the infrastructure. As the network takes on more and more responsibility from the applications, especially as is the case in an increasingly cloudy environment, the components in the network must be able to consume services provided by other components and collaborate as a means to ensure the fast and secure delivery of applications to their ultimate consumers.

One of the side-effects is that it will cause some amount of confusion in the organization, at “layer 9”, as it were, regarding what role is responsible for developing and ultimately deploying those policies. Will developers become more network-aware? Will administrators and operators begin to take on a more development-oriented role in order to integrate and orchestrate the data center using the collaborative capabilities of Infrastructure 2.0 services?

Maybe the answer to that depends on where you are, who you are, and whether you’ve drank from the bottle or not.

Related blogs & articles:

Technorati Tags: MacVittie,F5,cloud computing,virtualization,infrastructure 2.0,load balancing,QoS,rate shaping,API,integration,network,development

DevCentral Weekly Roundup Episode 120 - The Thread That Keeps On Giving

DevCentral Weekly Podcast — Mon, 01 Feb 2010 21:46:23 GMT

Welcome to the one hundred and twentieth edition of the DevCentral Weekly Roundtable Podcast! A weekly recap of the interesting things that have been going on within the DevCentral community.

Hosts: Colin Walker, Don MacVittie, Jason Rahm, Scott Koon.

In this weeks podcast the team discussed a cool ECV Pool Member status monitor, ARX Config with Don, Watch Proc, HOST/URI to pool mappings, troubleshooting LTM monitors, VMWare Automation, grep'ng for 'string', and gracefully degrading web 2.0 apps to maintain availability.

All of the links from this recording can be found with the dcpodcast120 tag at Delicious.com.

Get the Flash Player to see this player.

Technorati Tags: F5,DevCentral,ECV,ARX,LTM,iRules,VMWare,Web 2.0,Colin Walker,Don Mac Vittie,Jason Rahm,Scott Koon

Accelerating Your (Secure) Ride to the Cloud: Get in the Fast Lane

F5 Networks News — Mon, 01 Feb 2010 13:22:22 GMT

The challenges associated with cloud computing and remote data center locations are strikingly similar, particularly with regards to performance and transfers of large files across bandwidth constrained, e.g. WAN, links. Congestion, latency, and prioritization of other applications can slow down the transfer of data across data centers, whether traditional or cloud-based. Conversely, the utilization of Internet-connected WAN-speed links to provide both transfer of large data files and application service to myriad remote employees and users via mobile devices, roaming laptops, and dedicated desktops in remote offices, can cause performance challenges.

The challenges are only going to become more complex as more and more users, employees, and customers take advantage of advances in technology to effectively maintain an “always on” connection with the services you provide. This is more challenging than ever before because of the increasingly flexible manner in which applications are accessed across a variety of devices and locations, sometimes simultaneously.

IDC also reported in its new Marketplace Model and Forecast report that more than a quarter of the world's population, or 1.6 billion people, used the Internet in 2009 on a PC, mobile phone, video-game console, or other device. By 2013, that number is expected to rise to 2.2 billion.

-- InformationWeek, “1 Billion Mobile Internet Devices Seen By 2013”

What’s necessary to address the performance challenges of today’s volatile and mutli-faceted network environments is to be able to first identify who is accessing what application from where and what type of device so that the appropriate mechanisms to address the specific issues raised by a given combination of device, user, network, and application can be effectively utilized.

F5 addresses these challenges and provides the means by which organizations can apply optimization, acceleration, and security policies based on a variety of parameters. This allows organizations to specifically target devices, users, locations, applications or any combination thereof with the right set of solutions at the right time to mitigate performance and security challenges introduced by today’s increasingly mobile user-base and application deployment strategies. The solution to these problems does not, surprisingly, introduce more complexity into the data center. On the contrary, F5’s solution to these challenges actually simplifies the application delivery network by unifying many of these highly related functions onto a single, easily managed platform.

F5’s BIG-IP^® Edge Gateway^™ unifies secure SSL VPN access, dynamic access and optimization control, and application acceleration for remote users—regardless of the device or access network—on a single, scalable platform. BIG-IP Edge Gateway accelerates and optimizes applications, offering significant performance gains (up to eight times faster), and ensures that users are always seamlessly connected via F5’s new BIG-IP Edge Client^™ with smart connection technology. Smart connection technology’s unique persistence and access intelligence makes sure that as a user roams networks—wireless (802.11), cell (3G and LTE), or wired—they stay connected, and the integrity of their application session is protected. The client also includes best-in-class endpoint security and unique application and network acceleration.

Reducing the impact on transfer of large data files such as virtual machine images deployed into cloud computing environments or secondary data centers, BIG-IP WAN Optimization Module^™(WOM^™) accelerates data transfer over the network by optimizing traffic and bandwidth through the use of adaptive compression and de-duplication technologies. WOM reduces bandwidth costs while increasing transfer rates up to 70 times over existing methods. This new BIG-IP product module ensures data is replicated quickly and reliably at speeds greater than 1 Gbps—faster and less-expensive than any alternative on the market today.

By leveraging both Edge Gateway and WOM, organizations can reduce bandwidth consumption and ensure no application or data transfer suffers to the benefit of others. Leveraging Edge Gateway organizations can target acceleration policies based on the unique characteristics of the endpoint, the network, and the application and ensure that IT resources are not wasted by being applied to applications and users that will not benefit from them.

You can find more information on Edge Gateway and WOM at F5’s corporate website.

Related resources:

F5 BIG-IP v10.1 Advanced ADC Features [Slideshare Presentation]
BIG-IP Version 10.1: An Integrated Application Delivery Architecture [Whitepaper, PDF]
Geolocation and Application Delivery [Whitepaper, PDF]
Unified Access and Optimization [Whitepaper, PDF]
F5 Delivers Next-Generation Application Delivery Services Giving Enterprises More Control with Context-Aware Networking [Press release]
BIG-IP v10.1 Now Available
Accelerating Your (Secure) Ride to the Cloud: Drive Smart(er)
Accelerating Your (Secure) Ride to the Cloud: Fasten Your Seatbelts
Geolocation Gets More Granular with F5 BIG-IP and Quova
External articles:

Technorati Tags: F5,BIG-IP,v10.1,Edge Gateway,WOM,application delivery,press release

Clouds Are Like Onions

Lori MacVittie — Mon, 01 Feb 2010 11:52:00 GMT

Which of course are like Ogres. They’re big, chaotic, and have lots of layers of virtualization.

In discussions involving cloud it is often the case that someone will remind you that “virtualization” is not required to build a cloud. But that’s only partially true, as some layers of virtualization are, in fact, required to build out a cloud computing environment. It’s only “operating system” virtualization that is not required. Problem is unlike the term “cloud”, “virtualization” has come to be associated with a single, specific kind of virtualization; specifically, it’s almost exclusively used to refer to operating system virtualization, a la Microsoft, VMware, and Citrix. But many kinds of virtualization have existed for much longer than operating system virtualization, and many of them are used extensively in data centers both traditional and cloud-based. Like ogres, the chaotic nature of a dynamic data based on these types of virtualization can be difficult to manage.

Layer upon layer of virtualization within the data center, like the many layers of an onion, are enough to make you cry at the thought of how to control that volatility without sacrificing the flexibility and scalability introduced by the technologies. You can’t get rid of them, however, as some of these types of virtualization are absolutely necessary to the successful implementation of cloud computing. All of them complicate management and make more difficult the task of understanding how data gets from point A to point B within a cloud computing environment.

EIGHT KINDS OF VIRTUALIZATION

Yes, that’s right, eight kinds of virtualization exist though we tend to focus on just the one, operating system virtualization. Some may or may not be leveraged in a cloud computing environment, but at least four of them are almost always found in all data center environments.

Operating System Virtualization is what we tend to think of when we simply say “virtualization.” This is the virtualization of compute resources, the slicing and dicing of a single physical machine into multiple “virtual” machines typically used today to deploy several different applications (or clones of a single application) on the same physical hardware.
Network Virtualization is likely one kind of virtualization many don’t even consider virtualization, but it is and it’s even got standards that help ensure consistency across implementations. The VLAN (Virtual LAN) has existed since the early days of networking and is used in cloud computing environments to isolate customer data. VLANs essentially create a virtual network overlay atop an existing physical network, slicing and dicing the physical connections into multiple virtual (and hopefully smaller) networks that can be configured to provide security and network-layer functions like quality of service and rate shaping peculiar to the applications and users that are directed over the VLAN. VLAN tagging, used to identity traffic as “belonging” to a specific virtual network, is defined by IEEE 802.1q.

Also a form of network virtualization is trunking or link aggregation as defined by IEEE 802.1ad. Trunking aggregates multiple physical ports on a switching device and makes them appear as one logical (virtual) link, providing additional bandwidth to high volume networks as well as load balancing traffic across the physical interconnects in order to maintain consistent network performance. Interestingly enough, VLANs are almost always used when trunking is used in a network.

And of course there is NAT (Network Address Translation), which is also a form of network virtualization. Because of the dearth of IP addresses, most users internal to an organization are directed through a pool of one or more public IP addresses (routable, i.e. accessible by people across the Internet) to access resources external to the organization. The virtualization here again makes many IP addresses (internal, non-routable, private) appear to be one or a small number of IP addresses (public, routable, external). This process is also used on inbound connections, making one or a small number of external, public IP addresses appear to represent multiple, internal, private IP addresses.
Application Server Virtualization occurs when a Load balancer, application delivery controller, or other proxy-based application network device “virtualizes” one or more instances of an application. The process of virtualization an application server makes multiple servers appear to be one ginormous server to clients, and acts in a manner very similar to trunking in that this form of virtualization is about aggregation. When applied to application servers, this virtualization focuses on the aggregation of compute resources.

This form of virtualization is almost always necessary in a data center, whether traditional or cloud-based. Application server virtualization is the foundation on which failover (reliability) and scalability are based, and one would be hard-pressed to find a modern data center in which this form of virtualization – whether provided by software or hardware – is not already implemented.
Storage Virtualization is another form of aggregation-based virtualization. Storage virtualization aggregates multiple sources of storage such as NAS (network attached storage) devices and NFS/CIFS shares hosted on various servers around the data center and “normalizes” them into a single, consistent interface such that users are isolated from the actual implementation and see only the “virtual” namespaces presented by the storage virtualization device.

There are four other “types” of virtualization, but it is these four that are primarily utilized today and with which most folks are already familiar – it just may be that they are using different terminology. Perhaps that’s because virtualization of the network and application server have existed for so long most people do not associate it with virtualization. All four of these kinds of virtualization end up forming layers of abstraction throughout the network, and like operating system virtualization introduce management and architectural challenges that are increasingly difficult to address as environments become more and more dynamic, a la a cloud computing environment.

INFRASTRUCTURE 2.0 to the RESCUE

It is these challenges that Infrastructure 2.0 is attempting to address. The increased strain on networks and infrastructure caused by virtualization of multiple types and the need to dynamically configure and manage all the various components that comprise a cloud computing or highly virtualized environment is enormous. The burden is often placed on the shoulders of operators and administrators who are tasked with keeping straight the myriad processes and tasks that must be complete ere a new resource is added to any one of the “virtual” pools of resources. Whether that’s storage, or application servers, or networks, or applications the challenges are similar in nature.

The ability of network, storage, and application network components to collaborate in a common, standards-based way will be imperative to the long-term success of virtualization and cloud computing. Infrastructure 2.0 enabled components already exist, true, but the means by which they are integrated into the broader data center ecosystem still vary from component to component. While the existence of these dynamic control planes makes it possible to reduce the strain associated with managing and running a dynamic data center, such variations also introduce difficulties and can lead to vendor lock-in. Addressing these concerns is paramount to ensuring the long-term viability of emerging data center models, and to making the introduction of virtualization into the data center a less painful process.

Clouds, like ogres and onions, have layers. Layer upon layer of abstraction through the use of virtualization to provide for scalability and security of the applications that are being delivered. Like onions, attempting to manage such a dynamic, virtual environment could easily make an operator cry. Infrastructure 2.0 is a necessary movement forward to address the challenges that will continue to plague data center architects and operators as they attempt to implement a dynamic data center that achieves IT agility to match the demand for business agility.

If you’re going to Cloud Connect, you may want to sign up for the “New Infrastructure” track and learn more about Infrastructure 2.0 and the challenges it is attempting to address.

Related blogs & articles:

Technorati Tags: MacVittie,F5,virtualization,VLAN,load balancing,file virtualization,storage virtualization,network virtualization,VMware,Microsoft,Citrix,infrastructure 2.0,standards,cloud computing

ARX Config – Week 2

Don MacVittie — Fri, 29 Jan 2010 05:17:39 GMT

I wanted to do at least two updates a week on this series, but circumstances conspired to keep me from an update earlier this week. In case you missed it, we’ve had a release or two going on (that link also has the “F5 joins NetApp Alliance Partner Program” Press Release on it if you missed that one), and I’ve got my bit to play in that. I also inherited a rather large project that I need to drive home, and it took a chunk of time just figuring out where it was and what the next steps were. There all the excuses but the one you came for are done.

Now the one you came for… My network, my devices.

The ARX is up and running beautifully, it behaves as expected except for one niggling bit that I suspect is due to the fact that I’m using SMB class NAS devices, so I’m not going to bring up. If you’ve got a NetApp or EMC NAS, you’re probably not going to see it, so I’ll leave it at that.

My devices on the other hand… Arggghh.

I’ll skip the hoops I jumped through and the number of times I attempted to add shares trying to get my NAS devices to play well with others. One was requiring a login to access a drive marked public, the other was giving me access denied errors. Both of these problems were evident from both servers and the ARX. I’ve changed quite a few settings over the last week, so I went back and started again. It turns out that one NAS device requires the volume in the nfs path, the other does not. Problem one solved. Access wasn’t denied (as the device told me), but the share I was trying to mount didn’t exist. I got the name straight. The other was a setting in the global config that I tracked down – it defaulted to no access for all new nfs shares, and I had created new ones for testing, so I wasn’t messing with production data. A few mouse clicks later, and theoretically both are ready to go. As a bonus, after nearly two weeks of changing things on these boxes to get one of them fully functional – the NetGear was partially functional last week – All of the clients on the network could still get to their shares.

So I go back to the ARX management screen, and attempt to mount a share on my Seagate BlackArmor NAS. This is where owning an SMB NAS really started to hurt. With a fully qualified path, it tried, and it failed because root_squash was turned on. This is a cool protection mechanism of nfs that changes the uid of root to be “nobody” so root has no special privileges and cannot break anything. Fine, I turned it off on the NetGear/Infrant, so I would just turn it off on the Seagate. Remember that the ARX is a file virtualization tool with a lot going on inside. It needs root rights to move things about (particularly files in a tiered environment), manage file access privileges, and to manage the metadata share.

Guess what? After lots of research, I discover that the BlackArmor NAS doesn’t let you turn off root_squash. So I have a solution for this, I have another Namespace (think virtual tree container) on the ARX that I can use that has CIFS enabled. I’ve SMBmounted this box a zillion times, and our XP clients access it fine with CIFS also. So I pop back into the ARX manager, change to that Namespace, and try to add it as CIFS.

“NAS Device does not support Kerberos Authentication” The ARX tells me.

Sigh. So I can’t do NFS because root_squash can’t be disabled, I can’t do SMB without an ADS machine.

The BlackArmor is our primary NAS, so I don’t want to move forward without it, but Lori took down our ADS machine a while back, and it’s physically gone from the building.

That leaves me trying to use SMB PDC functionality (vaguely recall doing that once), or setting up a new ADS server and hoping that the BlackArmor knows how to use that.

So a chunk of the reason I skipped blogging earlier in the week was simple… I had nothing much to report other than the obvious – Seagate BlackArmor isn’t enterprise class NAS. Duh.

And now I have a project for this weekend. Setting up ADS to move this project along, I’m tired of blogging about my network/storage issues and want to move on to actually using the ARX.

I tried to turn this into an excuse to snag a NetApp – something like a FAS2020 would do, but that fell through when a fellow F5er brought reason into the discussion… So that idea is out. For now.

Until next time,

Don.

All Options Were Considered… *

* Photo by Alex Nash and used under the Creative Commons License.

Click the image to view the original picture on Flickr

Audio White Paper - Geolocation and Application Delivery

DevCentral Interview — Fri, 29 Jan 2010 00:14:21 GMT

The data from Geolocation providers offers highly valuable data to a variety of stakeholders and is no longer just for advertising..

You can download the full white paper from f5.com here: geolocation-wp.pdf

Get the Flash Player to see this player.

Technorati Tags: F5,DevCentral,BIG-IP,Application Delivery Controllers,Geolocation,Pete Silva

In 5 Minutes or Less: Consolidate Access with BIG-IP Edge Gateway

DevCentral TV — Thu, 28 Jan 2010 18:04:46 GMT

Unified Access and Optimization with F5 BIG-IP Edge Gateway. Integrating security, availability, and application acceleration services to enable context-aware networking. Watch how to consolidate all your access needs. LAN, Remote and Wireless all on one device.

Technorati Tags: F5,DevCentral,BIG-IP,Edge Gateway,5 Minutes or Less,Pete Silva

How to Make mailto Safe Again

Lori MacVittie — Thu, 28 Jan 2010 11:07:49 GMT

Using HTTP headers and default browser protocol handlers provides an opportunity to rediscover the usability and simplicity of the mailto protocol.

Over the last decade it's become unsafe to use the mailto protocol on a website due to e-mail harvesters and web scraping. No one wants to put their e-mail address out on teh Internets because two minutes after doing so you end up on a trillion SPAM lists and the next thing you know you're changing your e-mail address.

But people still wanted to share contact information, so it became common practice to spell out your e-mail address, such as l.macvittie AT F5 dot com. But e-mail harvesters quickly figured out how to circumvent that practice so people got even more inventive, describing how to type the @ sign instead. For example, you can send me an e-mail at l.macvittie SHIFT 2 f5.com. But that's inconvenient and isn't easily automated, and eventually the e-mail harvesters figure that one out, too.

You could use contact forms instead to hide the e-mail address, but that's not really sharing and it isn't convenient for the person trying to get a hold of you. Like many folks, if I have a need to contact you I’d like a record that I did so and contact forms rarely provide a copy of the message which makes managing communication more difficult. It also affords spammers an easily automated method of submitting spam. What you really want is to be able to share your e-mail address and avoid the automated e-mail harvesters. Some folks suggest using CSS tricks that manipulate selectors to hide the e-mail address, but the problem with this is that it (1) doesn’t automatically launch a mail client and (2) the e-mail address is still in the text of the page, it’s just located in a different place. Some techniques use pure CSS and pseudoclass selectors and others use CSS to expose the actually e-mail address that is “hidden” in one of the HREF attributes, often the title. But in both cases the address is still in the page – or in an external CSS file which bots might pull if they’re following all links - and a simple regular expression search will find it easily enough.

ONE SIMPLE SOLUTION

One solution to this problem lies in leveraging an HTTP redirect and the ubiquitous browser support for the mailto protocol. Another description of this (and simple PHP code) can be found in this extensive reference document listing myriad ways of “hiding” e-mail addresses from harvesters. My only nit is that the author indicates the mailto-redirect method doesn’t work as per a normal mailto link, and I’ve found that’s not the case. A header redirect to a mailto location should automatically launch the mail client with the appropriate e-mail address as expected; at least it has in the testing I’ve done thus far on the iRule code used to accomplish the redirect.

The mailto link in the presentation page is changed to a standard HTTP link which, when clicked, executes logic that sends an HTTP redirect to a mailto location instead of a more standard HTTP location. The reason using this technique works is that the location to which the browser is being redirected is “hidden” in the HTTP headers, which bots and spots rarely interpret or expect to carry pertinent information and it is the browser that must interpret the location, which means any client-side supported protocol – like mailto – will cause the execution of the expected action. In this case it is launching the user’s e-mail client. This technique could, of course, be used to silently launch other client-side applications for which a protocol handler is defined as well.

A traditional HTTP redirect header to a web page would look like this:

   1: Location: http://www.w3.org/pub/WWW/People.html

And what we want is simply to make it look like this:

   1: Location: mailto:myemailaddress@example.com

There are two easy ways to implement this solution: network-side and server-side scripting.

METHOD #1: NETWORK-SIDE SCRIPTING

If you've got an application delivery controller enabled with network-side scripting you easily accomplish this task. You can also do the same with mod_rewrite if you're running Apache, and I'm sure there's a way to do it if you're running IIS, as well. Basically any network-side scripting enabled proxy can accomplish this task. You can also accomplish this via server-side scripts as well, but that requires modification to the application and that may not be desirable, depending on your situation.

First you need a URI which you can map to an e-mail address, e.g. /getmailto. The script needs to (1) look for that URI and (2) respond to the call to that URI with an HTTP redirect containing the appropriate e-mail address.

   1: when HTTP_REQUEST {

   2:    set curr_uri [HTTP::uri]

   3:    if {$curr_uri starts_with "/getmailto"} {

   4:       HTTP::redirect "mailto: <insert e-mail address here>"

   5:    }

   6: }

Now replace your mailto links with a link to the new URL. If your browser and mail client are configured properly, clicking on the link should bring up a new e-mail message with the e-mail address filled in. That supports usability needs (the e-mail address link should launch the user’s mail client) but it also keeps the address out of the page.

You'll probably want to further filter access to the URL by putting some iRule code in to detect bots and spiders and prevent them from exploring this one, but that's pretty easy, too. If you only have to replace one e-mail address, you could probably avoid rewriting the mailto links and simply use an iRule to transform the original mailto links to the new URL. And I'm sure someone out there will figure out how to change any mailto link to a new URL as well.

For example, if all e-mail addresses use the same formula, i.e. first initial, dot, lastname, you could construct a URL that sent the information as the URL, i.e. /lmacvittie. You can use a network-side script to then parse it into the right e-mail address and send the redirect back to the user. Using iRules you could also create a data group that maps URIs to e-mail addresses and do a quick lookup based on the URI to extract the appropriate e-mail address. As mentioned, you can do the redirect using mod_rewrite as well. I think iRules affords more flexibility in dealing with the actual data being manipulated (the e-mail address –> URI mappings), but you should be able to do it using other tools as well. The trick here is in putting the e-mail address in the HTTP header rather than in the body of the page where it is easily discovered by harvesting tools.

METHOD #2: SERVER-SIDE SCRIPTING

If you aren’t lucky enough to have your own personal, private BIG-IP or other network-side scripting enabled solution, you can also accomplish this same functionality in your application code. In a server-side script the trick is to ensure that you’re inserting the HTTP header before any other data is written to the connection. HTTP headers must be received first, before data. It’s like gravity – a law that must be obeyed.

For example, in PHP, all you need to do is call the function header with the appropriate location:

   1: header('Location: mailto: myemailaddress@example.com');

Rather than add this code to every page with an e-mail address it might be advantageous to take a service-based approach and simulate network-side scripting capabilities by creating a single “page” for all mailto redirects and then implementing the lookups and return of the appropriate HTTP redirect in a centralized, more manageable service.

Note that while you could achieve the same effect using custom HTML pages with the appropriate META tag or a small piece of JavaScript, this will result in the e-mail address being in a static page that a bot or spider can find and parse. The best solution will use network or server side executed logic because such code is not generally retrieved and parsed by miscreants. This also allows the integration of lookups dynamically. For example, both server and network-side scripting solutions may integrate with systems such as LDAP or AD and could therefore create a request to lookup an e-mail address dynamically based on the HTTP request.

There are other solutions to prevent this type of web scraping behavior, and of course any solution combined with a good SPAM prevention solution will improve the quality of the e-mail received. SPAM may be a fact of life on the Internet, but anything we can do to preserve the user experience while cutting down on how much SPAM we receive has to be a good thing.

UPDATED NOTE: I just had a thought that because this essentially moves e-mail to a URI-based system, it should be possible to integrate techniques like a CAPTCHA to further secure access to e-mail addresses against bots, spiders, and scripts.

Related blogs & articles:

Technorati Tags: MacVittie,F5,web 2.0,mailto,protocol,HTTP,SPAM,security,network-side scripting,scripting,PHP,redirect,browser,usability

First look: BIG-IP Edge Gateway demo with Nojan Moshiri

DevCentral TV — Wed, 27 Jan 2010 20:30:06 GMT

Nojan talks to Jeff Browning and Colin walker about the new F5 Edge product, a new, faster, secure way to get remote access from F5. Combining acceleration, load balancing, iSessions and remote access to users on any platform, Edge offers a new take on remote access. With traditional DC style tech talk, live demo and white boarding goodness, you don't want to miss this peek at what Edge can do for you.

Technorati Tags: BIG-IP Edge Gateway,F5,SSL,Remote Access,Nojan Moshiri,Jeff Browning,Colin Walkerer

How to Gracefully Degrade Web 2.0 Applications To Maintain Availability

Lori MacVittie — Wed, 27 Jan 2010 10:55:13 GMT

I haven’t heard the term “graceful degradation” in a long time, but as we continue to push the limits of data centers and our budgets to provide capacity it’s a concept we need to revisit.

You might have heard that Twitter was down (again) last week. What you might not have heard (or read) is some interesting crunchy bits about how Twitter attempts to maintain availability by degrading capabilities gracefully when services are over capacity.

“Twitter Down, Overwhelmed by Whales” from Data Center Knowledge offered up the juicy details:

The “whales” comment refers to the “Fail Whale” – the downtime mascot that appears whenever Twitter is unavailable. The appearance of the Fail Whale indicates a server error known as a 503, which then triggers a “Whale Watcher” script that prompts a review of the last 100,000 lines of server logs to sort out what has happened.

When at all possible, Twitter tries to adapt by slowing the site performance as an alternative to a 503. In some cases, this means disabling features like custom searches. In recent weeks Twitter.com users have periodically encountered messages that the service was over capacity, but the condition was usually temporary. At times of heavy load for more on how Twitter manages its capacity challenges, see Using Metrics to Vanquish the Fail Whale.

I found this interesting and refreshing at a time when the answer to capacity problems is to just “go cloud”, primarily because even if (and that’s a big if) “the cloud” was truly capable of “infinite scale” (it is not) it is almost certainly a fact that most organization’s budgets are not capable of “infinite payments” and cloud computing isn’t free.

It’s been many years, in fact, since the phrase “graceful degradation” has been uttered within my hearing, but that’s really what the article is describing and it’s something we don’t talk enough about. Perhaps that’s because it’s difficult to admit that there are limitations – whether technical or financial – on the ability to scale and meet demand. But there are, and if organizations are wise they’ll include in their application delivery strategy the means by which applications and services can “degrade gracefully.”

Twitter’s solution, the disabling of specific features, is a particularly easy way to implement such a strategy for Web 2.0 applications; at least it’s particularly easy if you have a network-side scripting capable solution mediating for the applications.

RACEFUL DEGRADATION

The reason it’s particularly easy to gracefully degrade Web 2.0 applications is that there is generally a 1:1 mapping between “functions” and “URIs.” This is often true for the web-facing interface, almost always true for RESTful APIs, and always true for SOAPy endpoints.

What you need to do is identify those “premium” URIs, i.e. those that can be disabled without negatively impacting core services, so that they can be “degraded” in the face of an overwhelming volume of requests.

You also need an intermediary. This can be a Load balancer, assuming it’s capable of providing the flexibility in configuration necessary to enable and disable service to specific URIs, i.e. it must be layer 7 aware. It has to be an intermediary through which all requests are routed because individual servers do not have the visibility required to be able to “see” the total requests and all responses. The fact that a server is throwing back 503 (Internal Error) errors indicates it doesn’t have the resources available to respond to a request, which means it won’t be able to respond to any requests, including those to disable services. Only an architecture that includes an intermediary of some kind (a reverse proxy) can achieve this solution.

The network-side script, which is deployed on the application delivery platform (load balancer), should implement logic that triggers degradation based on receiving 503 errors. It should probably not trigger on a single 503 or multiple 503s from the same application instance as such behavior could be indicative of a problem with that one instance as opposed to being produced due to a lack of capacity. That means the scripting solution needs to be able to take action based on a pattern of behavior coming from all application instances in conjunction with the total number of requests being received from users.

Yes, it has to be context-aware.

Once it’s determined that the errors are being generated due to a lack of capacity, the scripting solution needs to disable one or more of the specific URIs determined to be “premium” or ancillary. The intermediary can then respond to subsequent requests for the disabled URIs with custom content based on the expected response type. For example, if it’s an API call it might be appropriate to return a pre-formatted response in the appropriate data format indicating service is currently unavailable. Many network-side scripting solutions are capable of returning pre-formatted responses or they can be customized to provide more detail – it’s really up to the implementer to decide what information is included and how.

The premise is that as premium or ancillary services are degraded (disabled) that application instances will be able to focus on servicing core requests and return service to normal for those pieces of the application. When the volume of requests returns to within normal operating parameters for the capacity available, the intermediary can restore service to the previously degraded services.

CALABILITY is NEVER REALLY INFINITE

From a technological point of view “infinite scale” is not possible. At some point the volume of requests will reach boundaries that simply cannot be overcome, be they limitations on the load balancer (there is a limit to how many servers can ultimately be load balanced, and bandwidth is not unlimited) or on the application infrastructure itself. After all, you can’t launch a new instance of an application if there are no physical resources left on which to launch it.

It is almost certainly the case, however, that before reaching the technical limits of an “infinitely scalable” environment that you will hit a financial limitation. Or it may be the case that you haven’t jumped on the “cloud” bandwagon and what you see is what you get: a limited number of physical resources running a finite number of application instances, and that’s it. In either case, there are limitations on capacity and at some point you may reach them. How you respond to those limitations is an organizational decision, but graceful degradation in a controlled manner is probably more desirable than random, uncontrolled service outages.

Graceful degradation is an acceptable strategy for responding to availability issues and is especially easy to implement for a Web 2.0 application or API. It’s certainly more appealing than the alternative, which leaves every user essentially playing a game of Russian Roulette with availability of your web application.

Related blogs & articles:

Technorati Tags: MacVittie,F5,application delivery,network-side scripting,iRules,Twitter,capacity,scalability,cloud computing,Web 2.0,load balancing

The State of My Blog Address

Pete Silva — Wed, 27 Jan 2010 00:01:49 GMT

Readers, distinguished bloggers, various feeds - A year ago this week, I crossed over into double-digit blog entries (a whopping 10 stories at the time but a relative blog newcomer) and was wondering what magical rant would make this Blog Go to Eleven. Fidgeting with the keyboard and watching the blinking curser as nothing came to mind, I decided to dedicate January 30th as ‘Blog About Your Blog Day.’ The day that all bloggers would share stories, tips and other musings about their own blog. Since I don’t see it as a #trendingtopic on Twitter, it might not have stuck. Annual rituals often need a few years to take, so here’s the State of My Blog address in honor of my own made up writing holiday.

Last week, my good buddy Michael Sheehan of GoGrid (@HighTechDad on Twitter) wrote about the detailed process he goes through when creating a blog post. I gotta give him credit for both having a process and actually documenting it since I typically just see a topic/story, fire up Live Writer and tap away. Often stories come to mind while I’m walking the dog the evening before I post. I think it has to do with clearing my mind of all the day’s clutter and suddenly it’s like, ‘There it is!!.’ I’ll get home, quickly jot some notes or create a title, sleep on it and write it the next day. This was one of them. I typically try to post at least once a week and it’s usually around mid-week. This blog talks about how Thursday is the best day to post and this one backs it up with some statistical charts. I’ve read a couple that indicate that Monday’s are not great since everyone is getting back into the work routine, at least for business blogs. And speaking of Personal vs. Business blogs – Michael’s entry describes his method for personal blogs. I really don’t have a ‘personal’ blog since most, if not all, my entries are work related and published on F5’s DevCentral. I do feed WordPress, Ulitzer, Blogger, Posterous and others for greater coverage but our DevCentral community is my main audience. Even with a business blog, I do tend to incorporate personal stories since what I do as a career does mix with who I am as a person. I still remember years ago when I worked at the Milwaukee Repertory Theater an Art Director saying, ‘I am not my art!’ Always thought that was funny but interesting.

Even though this is a F5 branded blog, I do try to keep it focused on technology, trends, ideas and other industry topics instead of a ShamWow ad for BIG-IP. Most of our readers are familiar with BIG-IP (and learning about the new BIG-IP Edge Gateway announced this week) and I just like to compliment what they already know, offer some new ideas or bring attention to market/technology trends and how F5 solves some of these. Nothing too technical, security focused, a bit of humor, some personal insight and our daily lives – that’s the State of My Blog 2010. How about yours?

And here are a few other stories I considered writing about this week:

Until next time…

Technorati Tags: Pete Silva,F5,security,application security,network security,blogging,blogs

I Found the Missing Piece of the Virtualization Puzzle

Lori MacVittie — Tue, 26 Jan 2010 12:02:21 GMT

Nope. Wasn’t under the couch. In fact it turns out it wasn’t even missing, it’s just been overlooked and might already be in your data center.

As more organizations continue to make virtualization a core part of their overall application deployment strategy they are finding challenges associated with managing and, apparently, optimizing their newly created heterogeneous infrastructure. Kevin Fogarty, in “10 Virtualization Vendors to Watch in 2010”, writes of some of the challenges with virtualization to come in the next year. One of those challenges is, apparently, optimization of resources across physical and virtual assets, at least according to Mark Bowker of the Enterprise Strategy Group.

"Anybody who can fill the gaps the big guys don't in helping virtualization admins provision and control their infrastructure is worth a look," adds Mark Bowker, virtualization specialist at Enterprise Strategy Group. "The real missing piece, though, is the ability to optimize performance across both physical and virtual assets."

Later, Kevin addresses the challenges associated with capacity planning in a virtualized environment.

In the virtual world, however, capacity management is something of a black art -- not because few people have thought of it, but because few have built tools to look at both the physical and virtual servers and see how many of one will overwhelm the other. VKernel's product works on both VMware and Microsoft's Hyper-V. Without detailed capacity planning based on real data -- not imagination -- large-scale virtualization of production systems is not practical, according to Chris Wolf, analyst at The Burton Group.

THE MORE THINGS CHANGE, THE MORE THEY STAY THE SAME

One of the easiest and most effective ways to address the challenge of optimizing performance of heterogeneous environments comprising both physical and virtual assets is to optimize the applications being delivered by those assets. Optimizing the application means making more efficient the way the application and its application stack, i.e. the web and/or application server, makes use of compute resources. Doing so means a single virtual machine increases its capacity while lowering its resource requirements, which can translate into a higher VM density.

What seems to be ignored – either purposefully or accidentally – is that applications and their infrastructure stacks are the same whether deployed on a virtual machine or a physical server. For the most part the application doesn’t change, just the number of layers between it and the server. What’s necessary to optimize both resources, then, is to attack performance issues that are common to both models, i.e. those related to TCP, HTTP, and application-specific protocols. If you can optimize connections and application behavior through better resource management at the protocol level, leveraging caching when available, and compressing only when it will be beneficial you can significantly improve the efficiency of the platform – virtual or physical – such that capacity is increased. And because you’re optimizing components common to both deployment models, it doesn’t matter whether those resources are “virtual” or “physical.”

It’s also true that it is possible to instruct application delivery solutions to leverage physical and virtual assets based on their unique properties and capabilities. For example, if a virtual resource has a capacity that is half that of a physical resource, the application delivery solution can certainly be configured in a way to take that into consideration when determining what resource is best suited at the time the request is received to respond. Optimizing the distribution of requests across physical and virtual resources should not be overlooked especially if the environment is heterogeneous both in type of resource (physical or virtual) and capacity (maximum resources available). A poorly chosen load balancing algorithm as a means to distribute requests, for example, can make even more inefficient a heterogeneous environment. But a well-chosen algorithm can significantly increase performance and overall capacity and make more efficient the entire application infrastructure, whether virtual, physical, or both.

VIRTUALIZATION isn’t FREE

Part of the problem with capacity planning for applications is, certainly, ignoring the overhead associated with virtualization. The hypervisor is, after all, still an application and requires resources. It is the “abstraction layer” between the virtual machines and the underlying operating system and hardware and all the “virtual” activity must be channeled through it, which is going to consume resources that will not be available to applications. In other words, a virtualized server will never provide 100% resource capacity for applications, and this needs to be considered when performing capacity planning. Microsoft, for example, notes in a TechEd Europe presentation, “Microsoft Exchange Server Virtualisation: Does It Make Sense? [UNC03-IS]”, that the “hypervisor adds processor overhead” citing the hypervisor consumes an approximate ~12% of processor resources in its own Exchange 2010 testing. Benchmarking tests conducted by VMWare on web servers, too, indicate a varying amount of hypervisor overhead (~16% at its highest) depending on configuration and resources available [Consolidating Web Applications Using VMware Infrastructure, PDF, VMWare].

There’s nothing anyone can really do about hypervisor overhead right now. Virtualization vendors are working with hardware vendors and “virtualization aware” solutions will at some point address much of this overhead. But still, some overhead has to exist when deploying hypervisor-based virtualization solutions because, well, it’s part of the solution. That means when you’re looking at capacity you must take into consideration that overhead and subtract it from the available resources you can provision to your applications. That shouldn’t be a problem as capacity planners have long held that you never plan for 100% capacity anyway. But it still must be accounted for and addressed. One of the ways to offset the loss of resource capacity associated with a hypervisor is to make the applications that will be run atop the hypervisor are as optimized as possible. If you can gain more through optimization than you lose in associated overhead, you win.

Related blogs & articles:

10 Virtualization Vendors to Watch in 2010

It’s 2am: Do You Know What Algorithm Your Load Balancer is Using?

To Take Advantage of Cloud Computing You Must Unlearn, Luke.

Virtual Machine Density as the New Measure of IT Efficiency

The Myth of 100% IT Efficiency

Virtualization Changes Application Deployment But Not Development

Optimize Prime: The Self-Optimizing Application Delivery Network

Pursuit of Intercloud is Practical not Premature

The Application Delivery Deus Ex Machina

Technorati Tags: MacVittie,F5,application delivery,cloud computing,virtualization,optimization,VMware,web applications,infrastructure 2.0,load balancing

Audio White Paper – Unified Access and Optimization with F5 BIG-IP Edge Gateway

DevCentral Interview — Mon, 25 Jan 2010 23:12:57 GMT


Integrating security, availability, and application acceleration services to enable context-aware networking.

You can download the full white paper from f5.com here: unified-access-edge-wp.pdf

Get the Flash Player to see this player.

Technorati Tags: F5,DevCentral,BIG-IP,Application Delivery Controllers,Edge Gateway,Pete Silva

I’m a donkey on the Edge!

Mel Ruby — Mon, 25 Jan 2010 21:52:32 GMT

Not really, I just like that line from “Shreck”. Of course the title of this post has to do with a myriad of announcements F5 Networks made today, and I will try to post some other nuggets in the next few weeks about the important BIG-IP v10.1 news we have been releasing since November 16. Let’s briefly discuss BIG-IP Edge Gateway:

simplifies device management by ensuring access across heterogeneous applications services, access networks, and data resources. With the free BIG-IP Edge Client^™ solution—included with BIG-IP Edge Gateway—customers can combine acceleration, endpoint security, and automatic network roaming persistence for an improved end-user experience and increased workforce efficiency. In addition, web application acceleration yields performance gains for remote and mobile users.

enables organizations to cost-effectively scale their access networks to unprecedented levels. The solution accelerates and optimizes applications, yielding LAN-speed performance gains (up to 8x faster), and ensures that users are always seamlessly connected via F5’s new BIG-IP Edge Client^™ with smart connection technology. Smart connection technology’s persistence and access intelligence makes sure that as a user roams networks—wireless (802.11), cell (3G and LTE), or wired—they stay connected, and the integrity of their application session is protected. The client also includes endpoint security and application and network acceleration.

offers granular access policy enforcement. With features such as Visual Policy Editor (VPE), plus integrated authentication and endpoint inspection, access management becomes simpler, centralized, and more cost-effective. Now, IT staff can provide application access based on context by taking into account the user’s policy, identifying user location for redirection to local resources, and supporting single sign-on for most directory services across all application types, devices, and operating systems.

IT staff can design application prioritization into access policies to ensure that bandwidth is reserved for high-priority applications. For example, VoIP traffic can be managed and prioritized so it is always clear, instead of choppy due to packet loss and latency. Furthermore, with F5’s client-side QoS functions, customers have service levels they need to increase user productivity.

How To Use CoralCDN On-Demand to Keep Your Site Available. For Free.

Lori MacVittie — Mon, 25 Jan 2010 11:55:23 GMT

Cloud computing and content delivery networks (CDN) are both good ways to assist in improving capacity in the face of sudden, high demand for specific content but require preparation and incur operational and often capital expenditures. How about an option that’s free, instead?

While it’s certainly in the best interests of every organization to have a well-thought out application delivery strategy for addressing the various events that can result in downtime for web applications it may be that once in a while a simple, tactical solution will suffice. Even if you’re load balancing already (and you are, of course, aren’t you?) and employing optimization techniques like TCP multiplexing you may find that there are sudden spikes in traffic or maintenance windows during which you simply can’t keep your site available without making a capital investment in more hardware.

Yes, you could certainly use cloud computing to solve the problem, but though it may not be a capital investment it’s still an operational expenditure and thus it incurs costs. Those costs are not only incurred in the event that you need it, but in the time and effort required to prepare and deploy the application(s) in question for that environment.

Consider that you generally serve a fairly consistent patronage, such as would be the case for a local media outlet. No doubt you’ve got the infrastructure in place to handle the thousands of local visitors you receive on a daily basis, but what happens if a blog or editorial or news story is posted that catches someone’s eye? Often it’s relayed to Slashdot, or Digg, or Fark. And if it garners interest there, well, you may in real trouble and have a difficult time maintaining availability. You need a solution that can reliably handle just such a situation, but you can’t predict when that situation may arise. After all, “odd” or breaking news doesn’t often happen with any amount of notice. The budget to build out a larger infrastructure to handle a “could happen, might happen, can’t guarantee will happen” scenario is impossible to justify.

What you need is a down and dirty, inexpensive (as in free) solution as an “insurance” plan against losing availability of your site. If that’s the case, perhaps what you need is to leverage the Coral Content Distribution Network.

WHAT is this CORAL thing?

I could describe it myself, but really the description offered up by the best source (the creators) says it far better than I could:

CoralCDN is a decentralized, self-organizing, peer-to-peer web-content distribution network. CoralCDN leverages the aggregate bandwidth of volunteers running the software to absorb and dissipate most of the traffic for web sites using the system. In so doing, CoralCDN replicates content in proportion to the content's popularity, regardless of the publisher's resources---in effect democratizing content publication.
                                                                                                                                                     -- Coral Content Distribution Network | Overview

According to its Wikipedia entry, it is simplicity itself to take advantage of Coral Cache:

A website can be accessed through the Coral Cache by adding .nyud.net to the hostname in the site's URL, resulting in what is known as a 'coralized link'. So, for example, http://example.com becomes http://example.com.nyud.net. For websites that use a non-standard port for example, http://example.com:8080 becomes http://example.com.8080.nyud.net.

Basically you can leverage Coral to mirror a given host such that your site remains available in the face of an onslaught of traffic, and it’s free. What is not explained is how to get users to access your site via Coral Cache in an on-demand way, such as when a sudden spike in traffic would otherwise make your site inaccessible. Think of Coral as an on-demand, instantly provisioned content distribution network that will mirror your site and keep it available. All you need to do is take advantage of it.

Certainly if you know ahead of time you can create a link as described above and use it instead of your normal link, but it’s not always evident ahead of time that you’ll need the extra bandwidth/capacity and it would be nice if you could leverage such a solution on-demand. So what would be nice is a way to invoke these external services on-demand, in a way that’s not unlike the way in which caching solutions alter URLs, i.e. rewrite them, to take advantage of commercial content delivery networks (CDN).

HOW DO I DO THAT?

There are quite a few ways to leverage such a service on-demand, but all require that you have some amount of visibility into the current operational state of your site and infrastructure. You can’t execute the logic necessary to take advantage of Coral if you don’t know you need it, after all. I’ll offer up three different ways in which you could integrate Coral into your availability strategy; there are many more, I’m sure. The methods included here require that you have a network-side scripting enabled solution at your disposal. If you’ve already got a load balancing solution, check with the vendor; it’s possible that you have the capability. If you don’t, you may want to consider using something like mod_rewrite that gives you similar capabilities, though you’d need to deploy the rules created on every server if you do that unless you create a proxy for your web servers and implement the rules there. That’s one of the advantages of a Load balancer/application delivery controller: it by nature virtualizes multiple servers and acts as a proxy for them, providing a single, centralized location in which to implement these kinds of solutions.

Maintenance Window Redirect
Use case: During specific times of the week/day you’d like the ability to “take down” your servers for maintenance and you’d like to take them all down at the same time to reduce the time required to update/patch them all. In this case you’ll want to codify the times during which your servers will be unavailable and create a redirect (HTTP 302) to the Coral Distribution network as specified above, e.g. www.example.com.nyud.net

Referrer Based Redirect
Use case: Generally speaking the chances of quickly being overwhelmed by traffic are directly related to where the requests are coming from, i.e. Slashdot, Fark, Digg. Thus to handle this scenario you’ll want to create a network-side scripting rule that examines the HTTP_REFERRER header and, if it matches one of the “oh-lord-we’re-about-to-get-hammered” sites, redirect to the Coral Distribution network.

Connection/Request Limit Redirect
Use case: If you have a good idea what the total capacity of your servers is (and you do, because you’ve tested it under load, right?) then you can monitor current load on the load balancer/application delivery controller and upon nearing* those limits begin to redirect subsequent users to Coral. This solution requires a bit more intelligence and flexibility in the network-side scripting capabilities as you’ll need to track statistics, execute redirects based on variables, and end the redirection as requests slow down/decrease.

*The way that Coral works requires that it be able to access your site at least once to mirror it. Thus you cannot simply begin redirecting all requests to Coral without first allowing it to mirror the site by processing a request. This limitation necessarily requires that the network-side scripting solution you employ to implement such a solution be capable of allowing you to codify some amount of logic to allow this process to happen.

Okay, I lied – I’ll offer up a fourth option that requires no scripting and can be utilized without a load balancer:

4. Publish Coralized URI
Use case: If you’re publishing social media quick links on a story/blog/site, use the Coral-enabled URL instead of the origin content as the “link” to share. This won’t stop people from cutting and pasting from the address bar in their browser, but it will make sure that any “sharing” of the content immediately leverages the CoralCDN to distribute.

The reason I was leery of offering up the fourth option is because you lose visibility into statistics when users are directly sent to the CoralCDN. The other three options will be “counted” in logs and in statistics because they first connect to your site (the load balancer/application delivery controller) and then connect to the CoralCDN. Because the load balancer/application delivery controller is almost guaranteed to be able to handle more traffic than your servers, it can easily respond to requests with a redirect. But because it is responding it is counting the connections – and has all the relevant information about the client you might be aggregating - and therefore you don’t lose visibility.

If visibility isn’t an issue, then encouraging users to access the content directly via CoralCDN will certainly be one way to achieve the goal of keeping your content available.

There it is then; a free content distribution network that can be leveraged on-demand. Using CoralCDN is not a panacea and has limitations, of course, in that it’s not as flexible as cloud computing; it essentially mirrors your site, it doesn’t distribute it. But if it’s specific content that’s experiencing high demand and it’s not a normal occurrence, then a limited, tactical solution like CoralCDN may be just what you need to keep your site available and enjoy your 15 megabytes of fame.



Related blogs & articles:

WILS: How can a load balancer keep a single server site available?

How to use CoralCDN to help reduce bandwidth usage / server resources [mod_rewrite / mod_headers solution ]

3 Really good reasons you should use TCP multiplexing

It’s 2am: Do You Know What Algorithm Your Load Balancer is Using?

The Application Delivery Deus Ex Machina

To Take Advantage of Cloud Computing You Must Unlearn, Luke

Vertical Scalability Cloud Computing Style

Putting a Price on Uptime

Cloud Computing: Vertical Scalability is Still Your Problem

Technorati Tags: MacVittie,F5,coralCDN,content delivery network,application delivery network,CDN,mod_rewrite,network-side scripting,load balancer,slashdot,digg,fark,availability

Talk With Evan Loats From CSC About Remote Access

DevCentral Interview — Sat, 23 Jan 2010 01:00:00 GMT


Founded in 1959, Computer Sciences Corporation is a leading global IT services company. CSC provides innovative solutions for industry and government customers around the world by applying leading technologies and CSC's own advanced capabilities. These include systems design and integration; IT and business process outsourcing; applications software development; Web and application hosting; and management consulting.

In this podcast, we talk to Evan Loats, Remote Access Product Manager, Australia and Asia CSC, about the requirements of remote access users in today’s rapidly evolving mobile environment and the solution F5 Networks provides to their customers.

Get the Flash Player to see this player.

Technorati Tags: F5,DevCentral,BIG-IP,FirePass,CSC,Evan Loats

Strap Your Conversion Kit On – Become a Hybrid!

Jason Rahm — Fri, 22 Jan 2010 23:20:59 GMT

No, I’m not talking cars. I’m not convinced (yet) that the total cost of ownership is lower, set aside the performance. So what am I getting at? Skill sets. Jon Olstik wrote today that, well, he said it better than I could summarize, so I’ll quote him:

“IT needs new networking/application specialists. F5 financial results and the whole evolution of ADC functionality suggest the need for a new IT skill set. I believe there is a growing requirement for hybrid IT specialists who understand both networking and application requirements. These people will become architects and application performance gurus — and make a ton of dough. F5 should work with application vendors like Microsoft or Oracle to create a certification program in this area.”

When I first began managing BIG-IP duties back in 2002, I quickly recognized that my skill set was inadequate to do it, the infrastructure, or the applications justice. Its purpose was only load balancing and SSL offload, but the impact to the applications, or really, the potentially positive impact to the applications, was not addressed in design meetings. It just boiled down to make it work. This boded well for me since I didn’t really know squat about anything above layer four (shush all you haters who will contend that’s still the case). As I’ve moved from early exposure to BIG-IP’s full proxy arrival in v9, I’ve also taken interest in understanding the applications. I’m no expert, but I think every network guy that uses application delivery technology owes it to their customer to not just put it on the network and call it good. If you manage dozens of web applications on your BIG-IP, it will serve you well to understand the HTTP protocol. Organizations can make this easier on employees by cross training disciplines. It may make for a slipped deadline or a sluggish development cycle, but rotating your network guys through a month or so of application development shadowing, (and making your application developers field the calls from the users that “the network is slow" with the network guys showing the app developers the traces that prove its not) can only be good long term. As for you, Jon is absolutely correct that if you can marry the wisdom of network and application, you will be well compensated for your services. Get in the lab, get dirty, make mistakes (yes, that’s a Magic School Bus reference) and be humble enough to admit you don’t know it all and ask someone from another discipline to mentor you. You won’t be sorry.

DevCentral Top5 01/22/2010

Colin Walker — Fri, 22 Jan 2010 20:15:50 GMT

Wow! What a whirlwind it's been the past few weeks. Between holidays and vacation and people traveling out of town, it's been an absolute zoo around here. Though I've been out the past week or so there has been an avalanche of content. I've hemmed and hawed and finally managed to slim my picks down to just five, though there are at least a dozen awesome things worth checking out on DevCentral in the past week or so. So don't be shy, get out there and poke around for yourself. For now, though, here are my top 5 picks for the week:

v10.1 - The table Command - The Basics

http://devcentral.f5.com/Default.aspx?tabid=63&articleType=ArticleView&articleId=2375

The new table command introduced in 10.1 is so hawesome and powerful it's hard for me to decide where to even begin describing the grandeur that is the table command. I've decided to begin at the beginning, and point you to the basics first. There are nine (yes, 9) tech tips published in the past week or so having to do with the new table command. They range from this intro doc to some pretty powerful, in depth, well explained examples. They are all penned by the creator of the command and go into amazing detail. This series has instantly become a contender for one of my favorite batches of content ever released on DevCentral, which is saying something. If you're looking for a way to store data, store data in a structured format, perform counting operations or about a bagillion other things dealing with data storage and manipulation in iRules, you must read about the table command. Huge thanks to spark for the work on the command and going above and beyond on the documentation.

TMSH Scripting in v10.1

http://devcentral.f5.com/Default.aspx?tabid=63&articleType=ArticleView&articleId=2374

This week's Top5 has not one, but two awesome docs regarding scripting on your BIG-IP. While iRules are near and dear to my heart, TMSH is quickly catching my interest as well. The new shell along with the powerful new scripting capabilities are wicked cool and have the potential to do some pretty amazing things. TMSH crams a huge amount of utility into an easily approachable package. This great doc Jason wrote up gets you started in style with an excellent description of where to begin, then takes you quite a bit further giving you examples of just how to build your own script. The possibilities seem rather limitless so I'm excited to see what people start doing once they get the hang of it. Check this one out for sure, and if you like what you see I'd recommend taking a look at the TMSH wiki and maybe giving this week's podcast where we spoke with Mark Crosland in depth about TMSH a listen.

ARX Config, Day One

http://devcentral.f5.com/weblogs/dmacvittie/archive/2010/01/18/arx-config-day-one.aspx

In the first installment of what I'm hoping proves to be a long, detailed series describing his experiences with his ARX, Don dishes out a great intro post about getting his ARX out of the box and working. He's honest and gives plenty of details about both what he loved and what he…didn't, which I appreciate. It sounds like he also plans to go into detail about any troubles he's having or things that he finds that stand out to him and the users should know about. With his vast experience in the storage world, getting to see an ARX through his eyes is just about the next best thing to getting to fiddle with one yourself. So if you have any interest in learning what it's like to set up and start using an ARX device, I recommend keeping a keen eye on this series. Having no ARX experience myself I'm quite interested to get his impressions, so I'll be one of the subscribed readers too.

iRule Editor - Offline Editing

http://devcentral.f5.com/Default.aspx?tabid=63&articleType=ArticleView&articleId=2385

Joe's amazing creation, the iRule Editor, just got better. He's released a couple new features for it recently but the one that caught my attention the most is something that people have been asking about for quite some time now: offline editing. The iRule Editor has previously been a 100% online tool. You'd fire it up, connect to your device and start editing away. But what if you're on a plane or just don't have a device to connect to? Well, you were out of luck. Even though you could save the iRules themselves to your on disk archive, the editor wouldn't allow you to edit them offline before. But now, you can. Keep in mind that you won't be able to use any syntax checking because that uses tmm on the BIG-IP to test compile the code, but you can edit to your heart's content along with all the handy features of the iRule Editor you've grown to love. Joe even took the time to go through a walkthrough of how this works and show you how to use the cool new feature in this video. This is a very cool improvement…thanks Joe!

Following Google's Lead on Security? Don't Forget to Encrypt Cookies

http://devcentral.f5.com/weblogs/macvittie/archive/2010/01/15/google-gmail-ssl-cookie-encryption.aspx

Last but certainly not least is Lori's post talking about SSL and why it isn't the only thing you need to think about when working on securing an application. Yes, SSL is an excellent and pretty standard first step to securing an online application these days. I, just like Lori, completely agree that you should be using SSL encryption as a security measure if you're at all concerned about your users or their data. Something Lori mentions though is spot on, "it’s not a panacea, especially where cookies are involved". Just because something is being encrypted across the wire doesn't mean that you can necessarily assume that it's going to be 100% safe once it gets where it's going. Data being stored on a client system, such as cookies that carry auth information, are a prime target for many malicious attacks trying to pry at user info. Cooke Encryption can be a powerful agent in stopping this and stepping up your security one more level. Have a look for yourself for a more detailed description of how this works.

There you have this week's DevCentral Top5. As always, feedback is welcomed and you can check out previous versions of the Top5 here - http://devcentral.f5.com/Default.aspx?tabid=101

Technorati Tags: DevCentral,Top 5,ARX,iRule Editor,iRules,Table,Google,tmsh,Colin Walker

#Colin

DevCentral Weekly Roundup Episode 119 - TMSH And Sed Sledding

DevCentral Weekly Podcast — Thu, 21 Jan 2010 23:21:18 GMT

   Welcome to the one hundred and nineteenth edition of the DevCentral Weekly Roundtable Podcast! A weekly recap of the interesting things that have been going on within the DevCentral community.

Hosts: Colin Walker, Jason Rahm, Joe Pruitt, Scott Koon.
In this weeks podcast we had special guest Mark on the podcast to talk about the new Traffic Management Shell (tmsh). We discussed a little bit about the history surrounding the development of the shell, the choices made regarding the use of TCL, some things you shouldn't do with it, and some things you probably didn't know you could do. We finished up with a few future ideas he's working on now for upcoming releases.

All of the links from this recording can be found with the dcpodcast119 tag at Delicious.com. You can also watch the video recording from our UStream.TV channel: DevCentral Podcast 119.

Get the Flash Player to see this player.

Technorati Tags: F5,DevCentral,tmsh,BIG-IP,TCL,iRules,iControl,Colin Walker,Jason Rahm,Joe Pruitt,Scott Koon

Cloud Balancing, Reverse Cloud Bursting, and Staying PCI-Compliant

Lori MacVittie — Thu, 21 Jan 2010 13:54:28 GMT

One of the concerns with cloud bursting specifically for the use of addressing seasonal scaling needs is that cloud computing environments are not necessarily PCI-friendly. But there may be a solution that allows the application to maintain its PCI-compliance and still make use of cloud computing environments for seasonal scaling efficiency.

Cloud bursting, a.k.a. overdraft protection, is a great concept but in some situations, such as those involving PCI-compliance, it can be difficult if not impossible to actually implement. The financial advantages to cloud bursting for organizations requiring additional capacity on only a seasonal basis are well understood, but the regulatory issues that surround such implementations hinder adoption of this method to address cost-effective capacity increases when necessarily only for short periods of time.

But what if we architected a solution based on cloud bursting that offers the same type of advantages without compromising compliance with regulations and guidelines like PCI-DSS?

REVERSE CLOUD BURSTING and CLOUD BALANCING

The ability to implement such an architecture would require that the PCI-compliant portions of a web application are separated (somehow, perhaps as SOA services or independently accessible RESTful services) from the rest of the application.

The non-PCI related portions of the application are cloned and deployed in a cloud environment. The PCI-related portions stay right where they are. As the PCI related portions are likely less heavily stressed even by seasonal spikes in demand, it is assumed that the available corporate compute resources will suffice to maintain availability during a spike, mainly because the PCI compliant resources have at their disposal all local resources. It is also possible –and likely – that the PCI-related portions of the application will not consume all available corporate compute resources, which means there is some capacity available to essentially reverse cloud burst into the corporate resources if necessary.

In a very simple scenario, the global server load balancer basically “reverses” the priority of data centers when answering queries during the time period in which you expect to see spikes. So all application requests are directed to the cloud computing provider’s instance first except for queries that require the PCI-compliant portion, which are always directed to the corporate (cloud computing perhaps) instance. This is basically a “cloud balancing” scenario: distributing application requests intelligently between two cloud computing environments.

The variations on this theme can become more complex and factor in many more variables. For example, you could set a threshold of capacity on the corporate data center instance that allows enough corporate compute resources available to handle the highest expected transaction rate and only burst into the cloud if the corporate capacity reaches that level. That’s traditional “cloud bursting.” You could also reverse the burst by dipping into corporate compute resources based on thresholds designated at the cloud computing provider’s instance to minimize the financial impact of utilizing a cloud computing provider as the primary delivery mechanism for the application. That would be “reverse cloud bursting.” The key is to ensure that no matter where the compute resources are coming from for the primary application components it does not negatively impact the availability and performance of the PCI-compliant processes executing in the corporate cloud environment.

THE KEY IS FLEXIBILITY IN ARCHITECTURE

Without the flexibility to deploy individual components of an application (a.k.a. services) into different environments these scenarios simply don’t work. Applications developed based on tightly-coupled frameworks and principles will never truly be capable of taking advantage of cloud balancing, bursting, or any architecture that relies upon specific components residing in a specific location because of regulatory issues or other concerns.

This is one of the core principles of SOA – separation of not only interface from implementation, but location-agnosticism. There are many ways to achieve this kind of location-agnosticism including on-demand generation of WSDL for client consumption that specifies end-point location based on the context of the initial request and the use of global server load balancing combined with context-aware application delivery. What’s vitally important, though, is the flexibility of the underlying application architecture and the ability to separate components in a way that makes it possible to distribute across multiple locations in the first place.

If that means SOA is the answer, then SOA is the answer. If that means a well-designed set of RESTful components, so be it. Whatever is going to fit into your organizational development and architectural practices is the right answer, as long as the answer includes “location agnosticism” and loosely-coupled applications. Once you’ve got that down the possibilities for how to leverage external and internal cloud computing environments is limited only by your imagination and, as always, your budget.



Related blogs & articles:

SOA Announces Comeback Tour

Use The Source, Luke!

Cloud is Not a Big Switch

Cloud Computing Makes Servers Obsolete

Governance: Service Catalogs and the Cloud

Have a can of Duh! It’s on me

Forklifts, Rip and Replace, and Other IT Fairy Tales

Intercloud: The Evolution of Global Application Delivery

Cloud Balancing, Cloud Bursting, and Intercloud

The Infrastructure 2.0 Trifecta

The Context-Aware Cloud

Technorati Tags: MacVittie,F5,SOA,REST,architecture,infrastructure 2.0,load balancing,application delivery,PCI,cloud balancing,cloud bursting,intercloud,context-aware

ARX Config, day two (and three, technically)

Don MacVittie — Thu, 21 Jan 2010 06:38:41 GMT

Okay, so I hit a wall and didn’t post yesterday. That is not at all a statement about the ARX, indeed, it was acting as advertised. The problem is our network. It creaks a little bit around the corners.

We’ve got two NAS boxes, a bunch of Linux boxes (all patched, but some OS versions showing their age), a non-public Windows 2000 Server, and a slew of both Linux and WinXP clients. No Windows 7 yet, and we ditched Windows Vista pretty quickly.

Pretty simple setup, right? Yeah, if you’re in IT, you know that the longer a network exists the more weird stuff happens on it. Ours is a hybrid, we use it for testing and for hosting our “production” servers. Several websites, mail, two DNS servers, a box whose job is to present our SAN as a NAS (yeah, we did that)… Apps we installed to test – either for us or for various employers – and a media server.

The first snag I hit was the DNS servers. I set up the base IPs on the switch okay – the management port on one subnet and the data/inband management port on another – and the ARX config to do this is as straight-forward as any I’ve seen. Then I put the new names into DNS (more on names in a minute)… Problem is that our DNS servers have to be restarted in a specific order. I always forget that, so I modded the files and restarted them, and… Nothing. Wasted more time than I should have before I recalled that this happened to me before several years ago because I had restarted the secondary first (IIRC). So I restarted DNS in the opposite order and BAM! Problem solved.

So now I have reliable connectivity other than a serial port, and I pop open the configuration tool in the web browser. I’ve already done the basic config, so now I’m creating the actual virtual directory structure and mapping my drives to it. Or so goes the theory.

ARX again performed exactly as advertised, and the screens are really clear. The logs don’t contain as much information about errors as I’d like, but if I had the network overall configed correctly, that wouldn’t have been a problem.

The only issue when two people with masters degrees in computer science and high-tech jobs share a network is that it changes a lot. We used to have a Windows Domain Controller – ADS on Win2K. We even used to have a pre-ADS PDC… But when I looked, the NAS boxes were in a workgroup, not a domain. Hrmmm. After poking around the network, Lori tweeted that the domain controller has been gone for a while. Doh! Okay, I look at the ARX config, and while it might be possible to run the CIFS portions without a domain controller, it certainly doesn’t look like it. I could have popped off and asked the great people on our ARX Marketing team, or our IT staff who has also offered a hand, but I wanted to work through this to give you all the “starting cold” walk-through, and I knew a secret. I am Storage Guy in the house, and since most of our servers run Linux, all of our NASes support NFS. I don’t create storage without it.

So I checked, and yes, both NAS boxes were configured to run NFS, and ARX has some great NFS support, so I chose this path (as opposed to making our one Windows server into an ADS domain controller).

I was off! Well, kind of. This is the point where I admit that while I set everything up with NFS, I don’t always mount NFS. In fact, it appears that my finely configured NFS interfaces on one NAS box had never been used.

Our primary servers are all Linux. I checked them. They were nearly all mounting the NAS boxes with CIFS. Nearly. All of the ones accessing the primary NAS box were mounting it CIFS.

Sad state of affairs. Now I had NFS configured, and had read up on how to add nfs shares to the ARX (easy as pie, just a few questions like “which file server?” and “What mount point”, etc.)… But my shares were rather stale. So stale in fact that neither machine allowed the ARX access – not with an admin account, not with a user account. The ARX uses the admin task to handle things like moving files between tiers and other non-user activity, while the users just want their files.

Major sidetrack #2. The ARX was talking to both boxes, but wasn’t able to mount them. Either of them, any of the shares. So I go look at the configurations. On the secondary NAS box it was a simple case of mount point permissions. On the primary? I don’t know yet. That’s where I sit. I have a managed volume on the secondary (a 2TB Infrant NAS if you care), and it appears to be loading, but the primary is still not letting me mount via NFS – not from a random Linux box, not from the ARX.

So what’s the point of all of this? Well, you’ve got my “we’ve got a crufty network” update, and Lori and I talked on the phone tonight about how we’re going to rearchitect it after she returns – another fun time for reconfiguring the ARX ;-). And I’ve got at least one filer hooked up. Seems strange to me to call a brick a filer, but it’s equivalent, I still need to get the other going and see what happens when it synchs directories - they’re copied directory structures with some files on both and many others on one but not the other.

And if I can avoid it, I’m not going to take the fine offers of help from fellow F5ers. You are going to have to wade through most of this on your own if you install an ARX, and I want to give you a bit of an overview of one man’s issues as much as I want to do the “look how cool and easy this was!” thing.

Off to get some rest, it’s the 2 year old and I, off on our own tomorrow, I’m going to need that rest!

Tomorrow, we’ll see if I can actually get the basic config together. This sounds bad, but remember that I have other duties, I’ve got about six hands-on hours into this including downloading and reading docs – less than a day of your time, or a day of your time if you hang out at the water cooler a lot. Weeks of your time if you read too much BoFH. ;-)

Until next time,

Don.