Forum Discussion

Kirk_Bauer_1018's avatar
Kirk_Bauer_1018
Icon for Nimbostratus rankNimbostratus
Mar 31, 2007

Can't get HTTP payload modification iRule to work

I have tried this rule, largely pieced together from sample code on this site:

 

 

when HTTP_REQUEST {

 

set uri [HTTP::uri]

 

}

 

 

when HTTP_RESPONSE {

 

log local0. "$uri: [HTTP::header Content-Type]"

 

Only check responses that are a text content type

 

(text/html, text/xml, text/plain, etc).

 

if { [HTTP::header "Content-Type"] starts_with "text/" } {

 

Get the content length so we can request the data to be

 

processed in the HTTP_RESPONSE_DATA event.

 

if { [HTTP::header exists "Content-Length"] } {

 

set content_length [HTTP::header "Content-Length"]

 

} else {

 

set content_length 4294967295

 

}

 

if { $content_length > 0 } {

 

log local0. "Collecting data"

 

HTTP::collect $content_length

 

}

 

}

 

}

 

 

when HTTP_RESPONSE_DATA {

 

log local0. "Entered HTTP_RESPONSE_DATA"

 

set indices [regexp -all -inline -indices {search_regex} [HTTP::payload]]

 

foreach idx $indices {

 

set start [lindex $idx 0]

 

set end [lindex $idx 1]

 

set len [expr {$end - $start + 1}]

 

 

log local0. "Found String, len=$len"

 

HTTP::payload replace $start $len "replace_string"

 

}

 

}

 

 

Where "search_regex" is my actual search regex taken out here for simplicity. The page in question is very short and does NOT have a content-length header. I consistently have unexpected behavior with this rule.

 

 

1) After saving the rule, the first page load will hang. I'm guessing because we are trying to collect more data than exists -- possibly because the entire page fits in the first response packet? When I look in the log I see "Collecting Data" as the last log line.

 

 

2) When I reload the page in my browser, that causes the log entries "Entered HTTP_RESPONSE_DATA" followed by "Found String, len=34". Then it is immediately followed by a log entry for the new page request, but it never hits the "collecting data" message again. In fact from now on I can reload the page fine (no more hanging) but it never collects data and never changes the page.

 

 

I need to search for a regex so unfortunately stream profile will not work. I hope I'm missing something stupid here?

 

 

Thanks!
application delivery
dev
devops
iRules

5 Replies

Sort By
  • Kirk_Bauer_1018's avatar
    Kirk_Bauer_1018
    Icon for Nimbostratus rankNimbostratus
    Mar 31, 2007
    I did some more testing by adding unique variables to each request and trying multiple parallel requests. Upon changing the rule, the first request to the virtual server hangs, and all other requests will hang, until the first request is aborted. Once the first request is aborted, all other hanging requests to the virtual server complete OK. All future requests work fine without hanging. But the page is never modified, and the HTTP_RESPONSE_DATA code segment is only executed for the first request and only after I abort the request, which means I don't get to see the modified page.

     

     

    As I said before all future requests never collect data because apparently now the Content-Length header exists but is set to 0. If I modify the HTTP::collect line to always collect 10,000 bytes instead of looking at the content-length header, now all requests hang.

     

     

    I don't understand how the first response has no content-length header, and future responses have the header but have a value of 0. I also don't understand how I'm supposed to collect the whole page without knowing how big it is as even a value of 10000 for collecting seems to hang the response.

     

     

    FYI I have chunking set to "rechunk" in the HTTP profile.
  • Kirk_Bauer_1018's avatar
    Kirk_Bauer_1018
    Icon for Nimbostratus rankNimbostratus
    Apr 02, 2007
    Thanks for your help. I'm able to collect 2750 bytes of data, then call TCP::release and the page displays. However the page still breaks if I actually modify anything. If I comment out the actual replacement command then it works OK (but of course the page is not modified). Here is the code fragment:

     

     

     

    when HTTP_RESPONSE_DATA {

     

    log local0. "$uri;$my_id;$time: Entered HTTP_RESPONSE_DATA"

     

    set indices [regexp -all -inline -indices {/intl/en_com/images/logo_plain.png} [HTTP::payload]]

     

    foreach idx $indices {

     

    set start [lindex $idx 0]

     

    set end [lindex $idx 1]

     

    set len [expr {$end - $start + 1}]

     

     

    log local0. "$uri;$my_id;$time: Found String, len=$len"

     

    HTTP::payload replace $start $len "http://kaybee.org:8080/google_f5.png"

     

    }

     

    HTTP::release

     

    }

     

     

    So this code works since the replacement line is commented out. And I see the log entry showing it found a match. So what could be wrong with my replace command?

     

     

    Thanks in advance!
  • Deb_Allen_18's avatar
    Deb_Allen_18
    Historic F5 Account
    Apr 03, 2007
    Hey Kirk --

    I've had good success collecting as much data as necessary, then using regsub against the collected data instead of having to calculate insertion points:
    set payload [HTTP::payload]
set find "/intl/en_com/images/logo_plain.png"
set replace "http://kaybee.org:8080/google_f5.png"
regsub $find $payload $replace payload
HTTP::payload replace 0 [HTTP::payload length] $payload

    In either case, though, if the content length is changing, you'll need to set the Response Chunking option in your http profile to "Re-Chunk" so LTM doesn't send out an incorrect Content-Length header(which can cause the browser to hang or otherwise act funky)

    HTH

    /deb
  • Rom_Ortiz_79128's avatar
    Rom_Ortiz_79128
    Icon for Nimbostratus rankNimbostratus
    Sep 04, 2007
    Hello deb,

     

     

    I tried your script but I don't see that the iRule is replacing the content/ Here is my iRule

     

     

    when HTTP_RESPONSE_DATA {

     

     

    set payload [HTTP::payload]

     

    set find "http://adserver.ticketpop.com"

     

    set replace "http://ads.ticketpop.nre"

     

    regexp -all -inline -indices $find $payload $replace payload

     

    HTTP::payload replace 0 [HTTP::payload length] $payload

     

     

    }

     

     

    I am missing something?

     

     

    Rom
  • hooleylist's avatar
    hooleylist
    Icon for Cirrostratus rankCirrostratus
    Sep 04, 2007
    It would be easier and more efficient to use a stream profile to replace the strings within the data rather than trying to buffer the response data and perform regex operations against it.

     

     

    Just make sure to set Response Chunking to Re-chunk on the HTTP profile for the virtual server.

     

     

    Click here

     

     

    If you want to replace multiple strings, you can use the following format:

     

     

    Name: my_stream

     

    Source:

     

    Target: @123@456@@abc@xyz@

     

     

    Leave the source field blank. 123 will be replaced with 456. abc will be replaced with xyz.

     

     

    [edit: nevermind... it works with a space between the string pairs or without]

     

     

    Aaron