iRules

Understanding network-side scripting

Lori MacVittie — Fri, 31 Oct 2008 12:26:49 GMT

We all understand the lines in the sand (or the architectural diagram) that separate client-side scripting from server-side scripting. It's very clear that client-side scripting, e.g. JavaScript, VBScript, ActionScript, executes on the client while server-side scripting, e.g. PHP, ASP, executes on the server. But what about network-side scripting?

"There is no such thing!" might be the first response to this question, but I beg to disagree. Programmable proxies, a la F5's BIG-IP Local Traffic Manager, that provide a scripting language such as iRules, are simultaneously client-side and server-side, with the best definition to describe their placement in architectures being network-side scripting.

That's because the scripting, which is not different at all from client or server side scripting, executes in the network rather than on the client or the server. It has a view of both the client and the server and the data being exchanged between them because of its unique placement in the communication channel.

Network-side scripting essentially gives you a view of both client and server environments simultaneously, and in a single, unified location.

For example, network-side scripting can react to server-focused data like HTTP responses, cookies, and session information while simultaneously taking into consideration client-side information - HTTP requests, cookies, submitted data, and even the network conditions currently being experienced by that specific client. Because a programmable proxy is by necessity a full proxy, it is both client (to your application) and server (to the browser/customer) and can view all interactions between the two as a cohesive unit rather than as disconnected pieces of data.

[Edited to include an example, thanks to a suggestion from Bob in the comments!]

Here's an example of Cookie encryption that uses network-side scripting. The entire script runs in the network (on the proxy) but we've split the code into "client" and "server" side to show how network-side scripting can deal with both sides of the equation. There is additional script that executes when the rule is first initialized. You can check it out in the article that is the source for this code.

"Client side"

"Server side"

when HTTP_REQUEST {
   # If the error cookie exists with any value, for any requested object, try to decrypt it
   if {[string length [HTTP::cookie value $::cookie]]}{

      if {$::cookie_encryption_debug}{log local0. \
         "Original error cookie value: [HTTP::cookie value $::cookie]"}

      # URI decode the value (catching any errors that occur when trying to 
      # decode the cookie value and save the output to cookie_uri_decoded)
      if {not ([catch {URI::decode [HTTP::cookie value $::cookie]} cookie_uri_decoded])}{

         # Log that the cookie was URI decoded
         if {$::cookie_encryption_debug}{log local0. "\$cookie_uri_decoded was set successfully"}

         # Decrypt the value
         if {not ([catch {AES::decrypt $::aes_key $cookie_uri_decoded} cookie_decrypted])}{

            # Log the decrypted cookie value
            if {$::cookie_encryption_debug}{log local0. "\$cookie_decrypted: $cookie_decrypted"}
         } else {

            # URI decoded value couldn't be decrypted.
         }
      } else {
         # Cookie value couldn't be URI decoded
      }
   } else {
      # Cookie wasn't present in the request

when HTTP_RESPONSE {


   # Check if response contains an error cookie with a value
   if {[string length [HTTP::cookie value $::cookie]] > 0}{

      # Log the original error cookie value from the app
      if {$::cookie_encryption_debug}{log local0. \
         "Response from app contained our cookie: [HTTP::cookie value $::cookie]"}

      # Encrypt the cookie value so the client can't change the value
      HTTP::cookie value $::error_cookie [URI::encode [AES::encrypt $::aes_key [HTTP::cookie value $::cookie]]]

      # Log the encoded and encrypted error cookie value
      if {$::cookie_encryption_debug}{log local0. \
        "Encrypted error cookie to: [URI::encode [AES::encrypt $::aes_key [HTTP::cookie value $::cookie]]]"}
   }
}

The "client" side deals with client-specific events, like HTTP_REQUEST, and executes code to find and decrypt the appropriate cookie before sending the request and the now plain-text cookie to the web server.

The "server" side deals with server-side specific events, like HTTP_RESPONSE, and executes code to find and encrypt the cookie before returning the response and the now protected cookie to the client.

Network-side scripting executes in the network, on a programmable proxy such as an application delivery platform. It can store information about client-side activity and environments that can be taken into consideration when processing server responses. It can inspect and modify both client and server side data for myriad purposes including optimization, security, and to assist with implementing application-specific logic.

Network-side scripting allows you to implement application functionality in the network that can provide benefits like improved application performance; it can enable agility in both your network and application infrastructure; it can provide centralization of functionality in a service-oriented manner through reusable network-side scripts that can be applied as necessary to one, two, or all applications with minimal effort.

Network-side scripting is a powerful mechanism through which you can architect more scalable, secure, and peformant infrastructures capable of handling high-volumes of requests while protecting server-side infrastructures from the sometimes adverse affects of sudden spikes in user concurrency.

Network-side scripting uses

Network-side scripting is both client and server-side scripting that lives in the network, mediating between

Cookie Encryption
LDAP Connection Proxy
Authentication
Data input validation
Data scrubbing
Sticky connections
Error handling
Exception handling
URI rewriting

clients and servers to provide a platform on which a wide variety of solutions can be implemented. If you haven't previously considered the potential uses of network-side scripting in your architecture, give it some thought now.

Network-side scripting is one of the ways in which an agile infrastructure can be built that supports both IT and business agility. It offers a unique view into client and server side variables and parameters at a single point in the transaction process, which can be leveraged to implement any number of really cool solutions that span a variety of IT-focused disciplines.

Technorati Tags: MacVittie,F5,BIG-IP,proxy,scripting,server-side,client-side,application delivery,network-side,cookies,persistence,web,internet,applications,architecture,infrastructure,PHP,ASP,JavaScript,VBScript,ActionScript

How to prevent content theft using Apache mod_rewrite or F5 iRules

Lori MacVittie — Tue, 21 Oct 2008 10:31:51 GMT

Over the years imaginative developers have come up with a number of ways through which they hope to stop the pilfering of their images. Whether due to copyright issues or the increased bandwidth and associated costs resulting from "hot linking", site owners have tried a variety of solutions from JavaScript that prevents the ability to right-click and "save as" to watermarking high-resolution versions to make their images less appealing to image thieves.

Regardless of the reason you may want to prevent image theft, there's an easier and more effective method than introducing easily countered JavaScript and costly alternative technology solutions.

HTTP REFERER

Every web request made via HTTP comes with a set of standard HTTP headers. One of those headers is the referer (interesting spelled incorrectly), which indicates the domain and URL from which the request was made. If the request was direct (e.g. typed into the address bar by the user or loaded from a bookmark) then the referer header will be empty and usually displays in logs as "-" (at least they do on Apache). Otherwise, the referer header will have the FQDN (Fully Qualified Domain Name) of the referring page.

The referer header is central to this solution; by checking the referer header you can determine whether the request for your image came from a page on your site or someone else's or was a direct request. If you're trying to prevent theft obviously you only want to allow access to images if the request came from a page hosted on your site. So the referrer must contain your unique domain name (or any domain name you wish to allow access to) or the request should be denied.

Once you've determined that the referrer is not allowed access to the image, you'll want to rewrite the URL (there are caveats with this, so be careful) or respond in such a way as to indicate to the client that the image is not available for viewing.

IMPLEMENTING THE SOLUTION

In order to implement the solution you'll need to be able to intercept the request and examine the headers to determine its validity. We'll look at both mod_rewrite (Apache) and F5 iRules (BIG-IP) as a mechanism to do this.

mod_rewrite

iRules

(mod_rewrite code courtesy of: Debian Administration)

Rewriteengine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://example.com/.*$ [NC]
RewriteRule .*.(gif|GIF|jpg|JPG)$ - [F]

when HTTP_REQUEST {          
   set thief 0        
   set uri [HTTP::uri]          if {[matchclass $uri ends_with $::images] > 0 } {          
      if {[HTTP::header value Referer] contains "example.com"} {
         set thief 0          
      }          
      else {          
         set thief 1          
      }          
   }          
   if {$thief eq 1} {          
      HTTP::respond 200 content ""          
   }          
   else {          
      pool mywebsite_pool           
   }          
}

That's pretty much it. You could get creative and respond with an actual image, or rewrite the URI to be a different image. If you choose the latter, be aware that you'll need to add code to handle that exception case, or you'll put the client into a redirection loop. After all, the referrer is still not your site, so redirecting to another image will fall into the same code unless you specifically catch it. While Firefox will recognize this infinite loop and stop requesting the image, IE 7 just keeps trying, which is somewhat amusing but floods the network with requests that aren't going to get answered and uses up a connection on your web server or BIG-IP.

This solution obviously can be used to stop hotlinking to any type of content: Flash, video, audio, text. You only need change the extensions you are looking for to match those used by the content in question. You could also get more sophisticated and set up a system whereby allowed domains are given a cookie, which you can subsequently check to determine whether access should be allowed. You could also use this logic to stop specific domains from hotlinking to your content by checking the referer header against a list of allowed sites and refusing to serve the content to sites not on the list.

The nature of an intelligent mediator is such that you can pretty much come up with just about any solution involving HTTP headers and implement it fairly easily. There are advantages to using a full-proxy solution over mod_rewrite, but both will definitely provide a platform on which you can deploy a solution that can prevent content theft.

Technorati Tags: MacVittie,F5,iRules,BIG-IP,mod_rewrite,Apache,Flash,images,theft,rewrite,hotlinking,firefox,IE7,referer header,HTTP,internet,web

Automatically detecting client speed

Lori MacVittie — Tue, 09 Sep 2008 10:31:35 GMT

We used to spend a lot of cycles worrying about detecting user agents (i.e. browser) and redirecting clients to the pages written specifically for that browser. You know, back when browser incompatibility was a way of life. Yesterday.

Compatibility is still an issue, but most web developers are either using third-party JavaScript libraries to handle detection and incompatibility issues or don't use those particular features that cause problems.

One thing still seen at times, however, is the "choose high bandwidth or low bandwidth" entry pages, particularly on sites laden with streaming video and audio, whose playback is highly sensitive to the effects of jitter and thus need a fatter pipe over which to stream.

Web site designers necessarily include the "choose your speed" page because they can't reliably determine client speed. Invariably, some user on a poor connection is going to choose high bandwidth anyway, and then e-mail or call to complain about poor service. Because that's how people are.

So obviously we still have a need to detect client speed, but the code and method of doing so in the web application would be prohibitively complex and consume time and resources better spent elsewhere. But we'd still like to direct the client to the appropriate page without asking, because we're nice that way - or more likely we just want to avoid the phone call later. That would be a huge motivator for me, but I'm like that. I hate phones.

Whatever the reason, detecting client speed is valuable for directing users to appropriate content as well as providing other functionality, such as compression. Compression is itself a resource consuming function and applying compression in some situations can actually degrade performance, effectively negating the improvement in response time gained by decreasing the size of the data to be transferred.

If you've got an intelligent application delivery platform in place, you can automatically determine client speed and direct requests based on that speed without needing to ask the client for input. Using iRules, just grab the round-trip time (or bandwidth) and rewrite the URI accordingly:

    when HTTP_REQUEST { 
        if { [TCP::rtt] >= 1000 } { 
            HTTP::uri "/slowsite.html" 
        } 
    }

If you don't want to automatically direct the client, you could use this information to add a message to your normal "choose your bandwidth" page that lets the client know their connection isn't so great and perhaps they should choose the lower-bandwidth option. This is also good for collecting statistics, if you're interested, on the types of connections your customers and users are on. This can help you make a decision regarding whether you even need that choice page, and maybe lead to only supporting one option - making the development and maintenance of your site and video/audio all that much more streamlined.

Technorati Tags: MacVittie,F5,iRules,bandwidth,compression,redirect,http,web,internet,tcp,speed,browser

Layer 7 Switching + Load Balancing = Layer 7 Load Balancing

Lori MacVittie — Tue, 12 Aug 2008 11:44:57 GMT

Modern load balancers (application delivery controllers) blend traditional load-balancing capabilities with advanced, application aware layer 7 switching to support the design of a highly scalable, optimized application delivery network. Here's the difference between the two technologies, and the benefits of combining the two into a single application delivery controller.

LOAD BALANCING

Load balancing is the process of balancing load (application requests) across a number of servers. The load balancer presents to the outside world a "virtual server" that accepts requests on behalf of a pool (also called a cluster or farm) of servers and distributes those requests across all servers based on a load-balancing algorithm. All servers in the pool must contain the same content.

Load balancers generally use one of several industry standard algorithms to distribute request. Some of the most common standard load balancing algorithms are:

round-robin
weighted round-robin
least connections
weighted least connections

Load balancers are used to increase the capacity of a web site or application, ensure availability through failover capabilities, and to improve application performance.

LAYER 7 SWITCHING

Layer 7 switching takes its name from the OSI model, indicating that the device switches requests based on layer 7 (application) data. Layer 7 switching is also known as "request switching", "application switching", and "content based routing".

A layer 7 switch presents to the outside world a "virtual server" that accepts requests on behalf of a number of servers and distributes those requests based on policies that use application data to determine which server should service which request. This allows for the application infrastructure to be specifically tuned/optimized to serve specific types of content. For example, one server can be tuned to serve only images, another for execution of server-side scripting languages like PHP and ASP, and another for static content such as HTML , CSS , and JavaScript.

Unlike load balancing, layer 7 switching does not require that all servers in the pool (farm/cluster) have the same content. In fact, layer 7 switching expects that servers will have different content, thus the need to more deeply inspect requests before determining where they should be directed. Layer 7 switches are capable of directing requests based on URI, host, HTTP headers, and anything in the application message.

The latter capability is what gives layer 7 switches the ability to perform content based routing for ESBs and XML/SOAP services.

LAYER 7 LOAD BALANCING

By combining load balancing with layer 7 switching, we arrive at layer 7 load balancing, a core capability of all modern load balancers (a.k.a. application delivery controllers).

Layer 7 load balancing combines the standard load balancing features of a load balancing to provide failover and improved capacity for specific types of content. This allows the architect to design an application delivery network that is highly optimized to serve specific types of content but is also highly available.

Layer 7 load balancing allows for additional features offered by application delivery controllers to be applied based on content type, which further improves performance by executing only those policies that are applicable to the content. For example, data security in the form of data scrubbing is likely not necessary on JPG or GIF images, so it need only be applied to HTML and PHP.

Layer 7 load balancing also allows for increased efficiency of the application infrastructure. For example, only two highly tuned image servers may be required to meet application performance and user concurrency needs, while three or four optimized servers may be necessary to meet the same requirements for PHP or ASP scripting services. Being able to separate out content based on type, URI, or data allows for better allocation of physical resources in the application infrastructure.

Technorati Tags: MacVittie,F5,load-balancing,layer 7 switching,application delivery,application switching,SOA,XML,SOA delivery,clustering

The Treachery of Hyperlinks

Lori MacVittie — Fri, 08 Aug 2008 10:59:50 GMT

With apologies to René Magritte.

Did you know you could stop the treachery that is Rickrolling hyperlinks with an iRule? Just search your outbound HTML for the appropriate YouTube URLs (you may need a data group to store them all) and strip them out, or search your inbound posts for the URLs and refuse to post them.

Of course you could also write an iRule that automatically changes every submitted URL to be a rickroll, but man, that's evil! Maybe you just want to do it for a specific user. You can do that with iRules if your site uses cookies to identify users by id or name. Just check the cookie and if you find the right user, fire off the appropriate iRule code to replace the URLs before it's posted.

It would still be evil. But it would be funny evil, if you know what I mean.

Technorati Tags: MacVittie,surrealism,art,magritte,iRules,transformation,rickroll

Working around client-side limitations on custom HTTP headers

Lori MacVittie — Wed, 06 Aug 2008 11:07:14 GMT

One of the most well-kept secrets in technology is the extensibility of HTTP. It's one of the reasons it became the de facto application transport protocol and it was instrumental in getting SOAP off the ground before SOAP 1.2 and WS-I Basic Profile made the requirement for the SOAP Action header obsolete.

Web browsers aren't capable of adding custom HTTP headers on their own; that functionality comes from the use of client-side scripting languages such as JavaScript or VBScript. Other RIA (Rich Internet Applications) client platforms such as Adobe AIR and Flash are also capable of adding HTTP headers, though both have limitations on which (if any) custom headers you can use.

There are valid reasons for wanting to set a custom header. The most common use of custom HTTP headers is to preserve in some way the source IP address of the client for logging purposes in a load-balanced environment using the X-Forwarded-For custom header. Custom HTTP headers can be set by the client or set by the server or intermediary (load-balancer, application delivery controller, cache) as well and often are to indicate that the content has passed through a proxy. A quick perusal of the web shows developers desiring to use custom HTTP headers for a variety of reasons including security, SSO (single sign on) functionality, and to transfer data between pages/applications.

Unfortunately, a class of vulnerabilities known as "HTTP header injection" often causes platform providers like Adobe to limit or completely remove the ability to manipulate HTTP headers on the client. And adding custom headers using JavaScript or VBScript may require modification of the application and relies on the user allowing scripts to run in the first place, the consistency of which can no longer be relied upon.

But what if you really need those custom headers to either address a problem or enable some functionality?

All is not lost; you can generally use an intelligent proxy-based load balancer (application delivery controller) to insert the headers for you.If the load balancer/application delivery controller has the ability to inspect requests and modify the requests and responses with a technology like iRules, you can easily add your custom headers at the intermediary without losing the functionality desired or needing to change the request method from GET to POST, as some have done to get around these limitations.

Using your load balancer/application delivery controller to insert, delete, or modify custom HTTP headers has other advantages as well:

You don't need to modify the client or the server-side application or script that served the client
The load balancer can add the required custom HTTP header(s) for all applications at one time in one place
Your application will still work even if the client disables scripting

Custom HTTP headers are often used for valid reasons when developing applications. The inability to manipulate them easily on the client can interfere with the development lifecycle and make it more difficult to address vulnerabilities and quirks with packaged applications and the platforms on which applications are often deployed. Taking advantage of more advanced features available in modern load balancers/application delivery controllers makes implementing such workarounds simple.

Technorati Tags: MacVittie,F5,iRules,HTTP,internet,web,Adobe Flash,JavaScript,VBScript,script,custom,development,architecture,proxy,AIR,http headers

API Request Throttling: A Better Option

Lori MacVittie — Mon, 30 Jun 2008 10:43:11 GMT

This past week there's been some interesting commentary regarding Twitter's change to its API request throttling feature. Request throttling, often used as a method to ensure QoS (Quality of Service) for a variety of network and application uses, is used by Twitter as an attempt to not overwhelm the system such that they are forced to display the now (in)famous Twitter fail whale image.

One of the things you can do with a BIG-IP Local Traffic Manager (LTM) and iRules is request throttling. Why would you want to let a mediating device like an application delivery controller control request throttling? Because request throttling implemented by the server still requires the server to respond to the request. The act of responding wastes some of the resources you're trying to save by request throttling in the first place.

It's like taking two steps forward and one back. By allowing the application delivery controller to manage throttling requests you're relieving the burden on the servers and freeing up resources so the servers can do what they're designed to do: serve content.

Because an intermediary that is also a full proxy (like BIG-IP LTM) terminates the TCP connection on the client side, it does not need to bother the server in the case that a client has exceeded their allotted request usage. Now you might be thinking that such a solution would be fine for an entire site, but Twitter (and others) use request throttling on a per API call basis, not the entire site, and wouldn't a general solution stop people from even connecting to twitter.com in general?

It depends on the implementation. In the case of BIG-IP and iRules, request throttling can be done on a per virtual server (usually corresponding to a single "web site") basis or it can get as granular as specific URIs. In the case of a site with an API like twitter, the URIs generally correspond to their REST-based APIs. That means not only can you throttle requests in general, but you could get even more specific and throttle requests based on specific API calls. If one of the API calls is particularly resource-intensive, you could limit it further than those that are less resource intensive. So while querying may be limited to 40 request per hour, perhaps updating is limited to 30. Or vice-versa. The ability to inspect, detect, and direct messages lets you get as specific as you want - or need - according to the needs of your application and your specific architecture.

It really gets interesting when you consider that you could further make decisions based on parameters, such as a specific user and the application function. Because an intelligent application delivery controller can inspect messages both on request and reply, you can use information that may be returned from a specific request to control the way future requests are handled, whether that's permanently or for a specified time interval.

This kind of functionality is also excellent for service providers moving services to tiers, i.e. "premium (paid) services". By indicating the level of service that should be provided to a given user, usually by setting a cookie, BIG-IP can dynamically apply the appropriate request throttling to that user's service. The reason this is exciting is because it can be done transparently - without modifying the application itself. That means changes in business models can be implemented faster and with less interruption.

As an example, here's a simple iRule that throttles HTTP requests to 3 per second per client. Simple, effective, transparent to the servers. Thanks to our guys in the field for writing this one and sharing!

when HTTP_REQUEST {

    set cur_time [clock seconds]

    if { [HTTP::request_num] > 1 } {

       if { $cur_time == $start_time } {

          if { $reqs_sec > 3 } {

             HTTP::respond 503 Retry-After 2

          }

          incr reqs_sec

          return

       }

    }

    set start_time $cur_time

    set reqs_sec 0

}

It doesn't make sense to implement request throttling inside an application when the reason you're implementing it is because the servers are overwhelmed. Let an intermediary, an application delivery controller, do it for you.

Technorati Tags: MacVittie,F5,BIG-IP,iRules,request throttling,Twitter,QoS

Green IT: 404 Blacklisting

Lori MacVittie — Fri, 27 Jun 2008 11:06:35 GMT

One of the premises of a greener IT is to reduce the number of servers necessary while maintaining performance levels and meeting capacity needs.

Chances are that many of the HTTP requests received that result in a 404 (not found) message are typos, bots, or bad guys attempting to find a way into your web applications. The thing is that the server must respond to these requests, and it often requires some disk I/O to discover the file doesn't exist. That's expensive in terms of resources and can increase the total power consumption of your servers.

If you're finding enough 404 errors in your logs, and you've verified that they're accurate, i.e. the files don't - and shouldn't - exist, then you may want to consider blacklisting those requests. By blacklisting those pesky non-existent files you can obviate the need for the server to look for it, thus reducing the overall burden (and power consumption) on your servers. This equates to a better performing server, and ensures that real requests get the resources they need to be fulfilled in a timely manner.

To accomplish this task, you'll need an iRule that keeps track of URIs resulting in a 404 and ensures that subsequent requests for that URI are immediately "kicked back" to the user instead of passed on to the server. We'll do that dynamically, in real-time, because it's nearly impossible to guess what kind of funky URIs will be requested by users, bots, and bad guys.

We'll also want to log additions to our blacklist so administrators can verify that the files in question really aren't valid files that have somehow been removed/lost.

when RULE_INIT {
   array set ::unknown_pages { }    
}

when HTTP_REQUEST {
   if { [info exists ::unknown_pages([HTTP::uri])] } {
      HTTP::respond 200 content "<html><body>The requested file could not be found</body></html>"

   } else {
   set curr_uri [HTTP::uri]
   pool webpool
}

when HTTP_RESPONSE {
  if { [HTTP::status] == "404"} {
    set ::unknown_pages($curr_uri) 1
    log "Added $curr_uri to the 404 blacklist" 

  }
}

There are myriad other options you could employ to make this iRule even more flexible. You could add a timestamp to any URI added to the 404 blacklist, and revalidate that it is, in fact, still missing after a specified time interval. This allows you to support the case where the file should have existed, but didn't, giving IT time to resolve the problem and automatically removing the URI from the blacklist later. Or you can write another iRule that specifically removes a URI from the list, so you can manually manage the list when you need to.

You could also redirect the user to a prettier "not found" page rather than responding with simple text. Just replace the HTTP::respond line with one that uses HTTP::redirect with the appropriate URL:

HTTP::redirect http://www.example.com/sorry_page.html

In general, removing unnecessary processing from your server infrastructure can reduce costs, improve performance, and increase/maintain capacity. With a flexible platform like BIG-IP Local Traffic Manager and iRules, you can reduce the burden on your servers and get "greener".

Imbibing: Mountain Dew

Technorati Tags: MacVittie,F5,iRules,Green IT,blacklisting,404

Fixing Internet Explorer & AJAX

Lori MacVittie — Thu, 26 Jun 2008 11:41:44 GMT

A few weeks ago, as developers are wont to do, I rewrote our online gameroom. Version 1 was getting crusty, and I'd written all the AJAX handlers manually and wanted to clean up the code by using Prototype and Script.aculo.us. You may recall we discussed using these tools to build a Web 2.0 interface to iControl.

So I rewrote it and was pretty pleased with myself. Until one of our players asked why it wasn't working in Internet Explorer (IE). Now Version 1 hadn't worked in IE either, but because I have a captive set of users I ignored the problem and forced them all to use FireFox instead. But this player's wife will be joining us soon and she's legally blind. She uses a reader to get around the Internet and as luck would have it, the reader only works with IE.

So I started digging into the problem. I had thought it was my code (silly me), and thus moving to prototype would solve the problem. No such luck. Everything but the periodically updated pieces of the application worked fine. The real-time updating components? Broken in IE.

I looked around and found this very interesting article on Wikipedia regarding known problems with IE and XMLHTTPRequest, the core of AJAX.

From the Wikipedia article

Most of the implementations also realize HTTP caching. Internet Explorer and Firefox do, but there is a difference in how and when the cached data is revalidated. Firefox revalidates the cached response every time the page is refreshed, issuing an "If-Modified-Since" header with value set to the value of the "Last-Modified" header of the cached response.

Internet Explorer does so only if the cached response is expired (i.e., after the date of received "Expires" header).

Basically, the problem lies with IE's caching mechanisms. So if you were trying to build an AJAX application with a real-time updating component and it didn't seem to work in IE, now you may know why that is.

There are workarounds:

Modify the AJAX call (within the client-side script) to check the response and, if necessary, make a second call with a Date value in the past to force the call to the server.
Append a unique query string to the call, for example appending a timestamp. This makes the URI unique, ensuring it won't be in the cache and forcing IE to call out to the server to get it.
Change all requests to use POST instead of GET.
Force the "Expires" header to be set in the past (much in the way we expire cookies programmatically). Setting cache control headers may also help force IE to act according to expectations.

I used option #3, because it was a simple, quick fix for me to search the single script using Ajax.PeriodicalUpdater and automatically change all the GETs to POSTs. That may not feasible for everyone, hence the other available options.

Option #4 could easily be achieved using iRules, and could be coded such that only requests sent via IE were modified. In fact, Joe has a great post on how to prevent caching on specific file types that can be easily modified to solve the problem with IE.

First we want to know if the browser is IE, and if so, we want to modify the caching behavior on the response. Don't forget that IE7 is using a slightly different User-Agent header than previous versions of IE. Don't look for specific versions, just try to determine if the browser is a version of IE.

when HTTP_REQUEST {
  if [string tolower [HTTP::header "User-Agent"]] contains "msie"} {
    set foundmatch 1
  }

when HTTP_RESPONSE {
  if {$foundmatch == 1} {
    HTTP::header replace Cache-Control no-cache
    HTTP::header replace Pragma no-cache
    HTTP::header replace Expires -1
  }
}

You could also use an iRule to accomplish #3 dynamically, changing the code only for IE browsers instead of all browsers. This requires a bit more work as you'll have to search through the payload for 'GET' and replace it with 'POST'. It's a good idea to make the search string as specific as possible to ensure that only the HTTP methods are replaced in the Ajax.PeriodicalUpdater calls and not everyplace the letters may appear in the document, hence the inclusion of the quotes around the methods.

Happy Coding!

Imbibing: Coffee

Technorati Tags: MacVittie,iRules,Web 2.0,AJAX,XMLHTTPRequest,caching,Internet Explorer,prototype.js,Ajax.PeriodicalUpdater

Improving Security Through Dynamic Resource Obfuscation

Lori MacVittie — Mon, 16 Jun 2008 14:46:40 GMT

One of the most basic attacks against data-driven sites generated dynamically through scripting languages like PHP and ASP is to use the weaknesses of the language against the developer.

Attacks against sites that make use of scripting languages often attempt to exploit system level calls that can lead to all sorts of nastiness with very little work on the part of the attacker.

One of the ways to guard against this is to write secure code, of course, but we all know that we can only code against known attacks. The unknown is something we just can't always anticipate and that sometimes leaves us open to newly discovered hacks.

Resource obfuscation is an underused, overlooked capability on servers and application delivery platforms that can seriously improve the security of your web applications and decrease the chances of an accidental or automate breach of security. Resource obfuscation can be an added protection against attacks. By hiding or obfuscating the actual file/script being invoked you can prevent a lot of automated exploitation of vulnerabilities in the most popular scripting languages used to build dynamic web sites today.

Resource Obfuscation

Resource obfuscation, called "service virtualization" in the XML world, is really nothing more than rewriting the URI. In many cases you can simply "map" an external URI to an internal URI. For example, "/externalURI.php" will always map to "/internalURI.php".

If your solution is flexible enough, you could set up a map that can translate exceedingly obfuscated URIs such as "xyz123abc.php" into "internalURI.php". If you are really concerned about security and you have a truly dynamic application delivery platform with a feature like iRules you could even dynamically generate the external URIs, creating a per-session map, and replacing all the URIs in the payload with the session-constrained URIs. This is some serious work, but it would certainly keep the bad guys always guessing because the external URIs would never be the same because they would be uniquely generated on a per request basis.

In general, resource obfuscation is a simple, flexible addition to your security toolkit that is easy to implement and adds an additional layer of security around your web applications.

To make your web application even more secure try these additional simple security measures:

1. Make sure the internal URI isn't easily deduced from query parameters or the functionality of the script. For example, if you're calling a PHP script to delete a record, don't make the internal URI "deleteRecord.php", and don't include the name of the file in hidden parameters.

2. Use a WAF (Web Application Firewall) or your chosen solution for rewriting URIs to block external access to internal URIs if possible. This prevents exploitation of scripting level vulnerabilities in the event that an attack deduces the names of your internal scripts and applications.

3. Give your scripts and applications extensions (if applicable) that do not easily (and loudly) proclaim the language and platform on which your site is running. Consider changing PHP and ASP to .abc or .xyz, either on the server itself or using your chosen URI rewrite solution. It won't stop serious attackers, but it might confuse the heck out of automated bots and script kiddies, which can reduce the likelihood of someone discovering some overlooked vulnerability.

Imbibing: Coffee

Technorati Tags: MacVittie,F5,iRules,security,resource obfuscation,rewrite URI,service virtualization