Forum Discussion

TWSS_18275's avatar
TWSS_18275
Icon for Nimbostratus rankNimbostratus
Jun 05, 2008

HTTP::payload Content Limitation

We are using the following to make some on the fly changes, however we are running into a size limitation where we are only getting a small chunk of the payload. Is there anyway to have access to the entire page?

 

 

  
  when HTTP_REQUEST {  
     Don't allow data to be chunked  
    if { [HTTP::version] eq "1.1" } {  
      if { [HTTP::header is_keepalive] } {  
          HTTP::header replace "Connection" "Keep-Alive"  
      }  
      HTTP::version "1.0"  
    }  
  }  
    
  when HTTP_RESPONSE {  
     First, check to see if there's a Content-Length header  
    if {[info exists [HTTP::header Content-Length]] } {  
       If there is, we'll collect that much data  
      set clen [HTTP::header "Content-Length"]  
    } else {  
       Otherwise we collect the max  
      set clen 4294967295  
    }  
    
    if { $clen > 0 } {  
      HTTP::collect $clen  
    }  
  }  
    
  when HTTP_RESPONSE_DATA {  
     Here you define what you want to find in the payload  
    set find "Bad_Data"   
    
     And here's what you'll be replacing it with  
    set replace "==ReWritten_Data_GOOD=="  
    
    set payload [HTTP::payload]  
    
     Run the regsub to make all the replacements (add -nocase for case insensitivity)  
    if {[regsub -all $find $payload $replace new_response] > 0} {  
      HTTP::payload replace 0 [HTTP::payload length] $new_response  
      set payload [HTTP::payload] 
    }  
  }  
  

 

 

This code is from: http://devcentral.f5.com/Default.aspx?tabid=63&articleType=ArticleView&articleId=39

1 Reply

  • First, I'd suggest using a stream profile and STREAM:: commands to replace or remove the offending content, as it wouldn't be subject to the size limitations of HTTP::collect. Also, it should be significantly more efficient, as the HTTP payload isn't being buffered and you're not restricting the server side connection to HTTP v1.0.

     

     

    You can check the STREAM::expression wiki page (Click here) for an example of replacing response content. The stream option is only available in 9.2 or later.

     

     

    If you do need/want to stick with the payload collection method, can you try adding logging to see how much content is being collected? How far into the response is the data being replaced? Also, you might want to add another check before collecting the payload to only collect text-based content-types (you shouldn't bother collecting binary content).

     

     

    Aaron