Forum Discussion

Russell_McGinni's avatar
Russell_McGinni
Icon for Nimbostratus rankNimbostratus
Sep 16, 2009

iRule to replace http response data causes havoc

We have a live and beta VIP setup in F5 and in order to keep users in the environment they started in, we would like to use an iRule - we already use an iRule to inject a javascript include on most html pages and thought we could piggy back on that.

 

 

However when we included code to replace "www" with "beta" in our urls our F5's started failing over - not instantly - but more frequently when traffic peeked, less frequently during low traffic usage.

 

 

Any ideas what we are doing wrong ?

 

 

Many thanks in advance

 

 

Russell

 

 

when RULE_INIT {

 

Text to inject at the end of the stream

 

set ::InjectionText ""

 

set ::InjectText 0

 

set ::uri ""

 

}

 

 

when HTTP_REQUEST {

 

Save the URI so we can check it in the response

 

set ::uri [string tolower [HTTP::uri]]

 

set ::InjectText 0

 

}

 

 

when HTTP_RESPONSE {

 

if { [HTTP::status] eq "200" } {

 

set replace 0

 

set ctype [HTTP::header Content-Type]

 

 

First check to see if we are dealing with html or js content

 

switch -glob $ctype {

 

"*html*" -

 

"*javascript*" -

 

"*xml*" {

 

set replace 1

 

}

 

}

 

 

Incase there is no content type set on an htm(l) or js file

 

switch -glob $::uri {

 

"*.htm*" -

 

"*.js" {

 

set replace 1

 

}

 

}

 

 

If we have html content, make sure we add the Injection text later

 

if { $ctype starts_with "text/html" } {

 

set ::InjectText 1

 

}

 

 

But dont add the Injection text if we are dealing with certain files

 

That have a content type of text/html

 

switch -glob $::uri {

 

"*.ashx*" -

 

"*.asmx*" -

 

"*.axd*" -

 

"*.js*" -

 

"/members/login/*" -

 

"/members/uploader*" {

 

set ::InjectText 0

 

}

 

}

 

 

Trigger the event for HTTP_RESPONSE_DATA if needed

 

if { $replace == 1 } {

 

log local0. "Replace value: $replace"

 

if { [HTTP::header exists "Content-Length"] } {

 

set content_length [HTTP::header "Content-Length"]

 

} else {

 

set content_length 2048000

 

}

 

if { $content_length > 0 && $content_length < 2048001} {

 

HTTP::collect $content_length

 

} else {

 

log local0. "Http content greater than 2mb (actual: $content_length) for: $::uri"

 

}

 

}

 

}

 

}

 

 

when HTTP_RESPONSE_DATA {

 

First check to see if we have any text to inject

 

if { $::InjectText == 1 } {

 

set idx [string last "" [HTTP::payload]]

 

if { -1 == $idx } {

 

set idx [string last "" [HTTP::payload]]

 

}

 

 

if { -1 == $idx } {

 

set offset [HTTP::payload length]

 

log local0.debug "Got here but NOT injecting text into $::uri"

 

} else {

 

set offset $idx

 

HTTP::payload replace $offset 0 $::InjectionText

 

log local0.debug "Injecting text into $::uri"

 

}

 

}

 

 

set clen [string length [HTTP::payload]]

 

 

set old_name "www.mydomain.com"

 

set new_name "beta.mydomain.com"

 

set old_name_len [string length $old_name]

 

set new_name_len [string length $new_name]

 

set last [string last $old_name [HTTP::payload $clen]]

 

for {set old_index 0} {$old_index < $last} {incr old_index $new_name_len}{

 

set old_index [string first $old_name [HTTP::payload $clen] $old_index]

 

HTTP::payload replace $old_index $old_name_len $new_name

 

}

 

 

set clen [string length [HTTP::payload]]

 

 

set old_name "www.myotherdomain.org"

 

set new_name "beta.myotherdomain.org"

 

set old_name_len [string length $old_name]

 

set new_name_len [string length $new_name]

 

set last [string last $old_name [HTTP::payload $clen]]

 

for {set old_index 0} {$old_index < $last} {incr old_index $new_name_len} {

 

set old_index [string first $old_name [HTTP::payload $clen] $old_index]

 

HTTP::payload replace $old_index $old_name_len $new_name

 

}

 

 

HTTP::release

 

}

4 Replies

  • Hi,

    There are a few issues. The first one that jumps out is you're using global variables where you should be using locals:

    set ::InjectText 0

    set ::uri ""

    These should both be set as locals and removed from RULE_INIT:

    set InjectText 0

    set uri ""

    Another potential issue is if the server response contains a Content-Length header set to a value greater than 4Mb TMM will crash.

    SOL6578: TMM will crash if an iRule collects more than 4MB of data

    https://support.f5.com/kb/en-us/solutions/public/6000/500/sol6578.html

    You should consider logic like this to limit the payload collection size:

     
            Trigger collection for up to 1MB of data 
           if {([HTTP::header exists "Content-Length"]) && ([HTTP::header "Content-Length"] <= 1024000)}{ 
              set content_length [HTTP::header "Content-Length"] 
           } else { 
              set content_length 1024000 
           } 
           if { [info exists content_length] } { 
              HTTP::collect $content_length 
              if {$::rule_debug}{log local0. "[IP::client_addr]:[TCP::client_port]: Collecting $content_length"} 
           } 
     

    I didn't look too closely at the replacement logic. Does it work after you make the above changes?

    In terms of optimisation, you could eliminate a lot of the intermediate variables like 'set ctype [HTTP::header Content-Type]' and just reference [HTTP::header Content-Type]. The value for most HTTP:: commands is cached in the same event so there isn't a performance hit for re-referencing the command. There is extra memory used when saving the values to a new variable though.

    Aaron
  • Thanks Aaron for the prompt and helpful reply...

     

     

    I am using global variables initialize in RULE_INIT as i thought that was the only way they would be visible across all event handlers - this was a bad assumption?

     

     

    Additionally I thought I was being careful of the 4MB limit, also taking into account Unicode by limiting the payload collection size to 2Mb - however as I am adding content to the payload I can see how that would break through the limits I set.

     

     

    I'll make the changes and test again.

     

     

    Thanks again Aaron
  • Thanks Aaron, is my assumption correct, that if I do not "collect" for payloads greater than a limit (2Mb in my example), the HTTP_RESPONSE_DATA event isnt triggered - and therefore no payload replacement takes place ?

     

     

    Also, "I'd just suggest setting a max and collecting at least that for responses that are bigger than the limit, as you can do partial payload collection." - I'm not really sure I understand what you mean by this.

     

     

    Russell
  • Hi Russell,

     

     

    That's correct. If you don't call HTTP::collect, no payload will be collected and no replacements can be made to that response. You can collect the start (the first x bytes) of the payload and avoid the 4Mb limit even if the payload is larger than x bytes. You could then perform the replacement(s) in that collected portion. If there are strings you wanted to replace past the collected amount, they wouldn't be modified as that portion of the payload wasn't buffered.

     

     

    I have a faint memory of someone discussing the possibility of collecting x bytes, releasing it and then re-collecting more payload, but I can't find the post and haven't ever seen it implemented. Does anyone have more info on this possibility?

     

     

    Aaron