Forum Discussion

Yoav_Daniely_11's avatar
Yoav_Daniely_11
Icon for Nimbostratus rankNimbostratus
Feb 10, 2008

HTTP Requests are being corrupted

Greetings,

 

 

I have a rule that changes the order of http headers in an http profile based VIP by removing and inserting the headers (pasted below)

 

I use BIG-IP 9.1.2 Build 40.2

 

 

For some reason, sometimes (cannot determine exactly when) the request gets corrupted which caused it to fail.

 

 

An example of such corruption is additional new lines in the HTTP headers and strange values (in the example below the referrer header, notice the if-modified-since header).

 

 

Please advice.

 

 

e.g a corrupted request:

 

 

GET http://wsp.o2active.cz/lava/siteSpecific/TO2/images/1_0_B_O2logo.gif HTTP/1.1

 

Host: wsp.o2active.cz

 

Referer: http://wap.yahoo.com/

 

If-Modified-Since: Mon, 28 Jan 2008 20:18:14 GMT

 

Cookie: LavID=hrMvHvPR6ctMQTV6!-393991407

 

Cookie2: $Version=1

 

TE: deflate, gzip, chunked, identity, trailers

 

X-Nokia-msisdn: +420720676037

 

Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg,

 

image/gif, image/x-xbitmap, */*;q=0.1

 

Accept-Charset: wsp.o2active.cz

 

Referer: http://wap.yahoo.com/

 

 

Accept-Encoding: http://wap.yahoo.com/

 

If-Modified-Since:

 

User-Agent: NokiaN70-1/2.0536.0.2 Series60/2.8 Profile/MIDP-2.0 Configuration/CLDC-1.1

 

 

 

original request

 

 

GET http://wsp.o2active.cz/lava/siteSpecific/TO2/images/1_0_B_O2logo.gif HTTP/1.1

 

User-Agent: NokiaN70-1/2.0536.0.2 Series60/2.8 Profile/MIDP-2.0 Configuration/CLDC-1.1

 

Host: wsp.o2active.cz

 

Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1

 

Accept-Language: en_US,en;q=0.9

 

Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1

 

Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0

 

Referer: http://wap.yahoo.com/

 

If-Modified-Since: Thu, 07 Feb 2008 17:57:10 GMT

 

Cookie: LavID=C8JjHvRP1WJHvT75!725549565

 

Cookie2: $Version=1

 

TE: deflate, gzip, chunked, identity, trailers

 

Connection: keep-alive

 

X-Nokia-msisdn: +420720676837

 

 

 

 

IRULE

 

 

when HTTP_REQUEST {

 

foreach aHeader [HTTP::header names] {

 

if {not ( ( [string tolower $aHeader] eq "x-nokia-msisdn") or ([string tolower $aHeader] eq "x-forwarded-for" ))} {

 

log "before [HTTP::request]"

 

set value [HTTP::header value "Accept"]

 

HTTP::header remove "Accept"

 

HTTP::header insert "lws" "Accept" $value

 

 

set value [HTTP::header value "Accept-Charset"]

 

HTTP::header remove "Accept-Charset"

 

HTTP::header insert "lws" "Accept-Charset" $value

 

 

set value [HTTP::header value "Accept-Encoding"]

 

HTTP::header remove "Accept-Encoding"

 

HTTP::header insert "lws" "Accept-Encoding" $value

 

 

set value [HTTP::header value "User-Agent"]

 

HTTP::header remove "User-Agent"

 

HTTP::header insert "lws" "User-Agent" $value

 

log "after [HTTP::Request]"

 

 

3 Replies

  • Hi,

     

     

    It looks like the rule you posted isn't the same as what generated the corrupted request (based on the order of the headers in the corrupted request). Can you post the exact rule you're using now along with a current example of the failed modifications?

     

     

    Do the failures have a common user-agent string? Are you able to reproduce the failure? If not, how did you get the original headers? Can you post a few examples of corrupted requests along with the original request?

     

     

    Thanks,

     

    Aaron
  • Hi Aaron,

     

     

     

    Thank you.

     

     

    Following your questions:

     

    The rule changes the order of headers, this is the real rule and the real result, I know this output is strange.

     

     

    I was not able to classify for which requests it happens and for which not, it sometimes happens and it always restores for the same request.

     

    User-agent header affects it indeed ( I change the value of the ua and the request somehow gets through).

     

     

    Original headers were produced by Opera+proximitron and some manually added headers.

     

     

    Do you know of any F5 bugs that might relate to that?

     

    Also I will try to send more examples asap.

     

     

     

     

    Please advice and thank you,

     

     

     

     

    Yoav

     

  • Hello All,

     

    I realize this is an older post but it is one that comes up when searching on LWS issues with F5. Therefore I wanted to post an answer here as well. The primary controlling RFC for HTTP over the last 20 years has be RFC 2616. Where it adopted header folding or LWS folding from the SMTP specification. LWS folding places the newline and space "0x000a0x0020" characters every N characters (78 called out by earlier SMTP RFC with a maximum length of 998). The latest RFC group which now obsoletes RFC 2616 and to which newer updates to browsers are written to be compliant with, is RFC 7230. In RFC 2616 Section 4.2 Message Headers it states - "Header fields can be extended over multiple lines by preceding each extra line with at least one SP or HT" but in RFC 7230 Section 3.2.4 Field Parsing:

     

     "Historically, HTTP header field values could be extended over

      multiple lines by preceding each extra line with at least one space

      or horizontal tab (obs-fold). This specification deprecates such

      line folding except within the message/http media type

      (Section 8.3.1). A sender MUST NOT generate a message that includes

      line folding (i.e., that has any field-value that contains a match to

      the obs-fold rule) unless the message is intended for packaging

      within the message/http media type."

     

    Therefore for API "HTTP::header insert" commands should no longer use the LWS qualifier to insert Linear White Space to HTTP headers for newer implementations.

     

    Regards,

     

    Kevin