Forum Discussion

marc_schaer_577's avatar
marc_schaer_577
Icon for Nimbostratus rankNimbostratus
Sep 28, 2009

Content-Length missing in HTTP request

Hi all,

 

i have an issue where i have not yet been able to find a working solution to. Maybe someone came across the same and knows a way on how to solve it with an irule.

 

 

problem:

 

we have an http client that does not send the Content-Length header parameter in its http POST requests. This in turn creates issues on the receiving server. I thought it would be nice if i could simply calculate the Content-Length in an irule and then add it to the request with "HTTP::header insert "Content-Length" $clength". Now the issue is that i have not managed to come fill the clength variable with a value...

 

 

I played with HTTP::collect and HTTP::payload but i have not managed to get a working result.

 

 

did anyone already have a similar issue and maybe also have a working irule to solve this?

 

 

thanks a lot,

 

marc

6 Replies

  • -------->>> this is the rule i am using to test things:

     

     

    when HTTP_REQUEST {

     

    i put 512 here as if i dont put a value the connections start to hang.. How to do this if i dont know the Content-Length?

     

    HTTP::collect 512

     

    }

     

    when HTTP_REQUEST_DATA {

     

    Two different ways of calculating the payload length.

     

    set payload [HTTP::payload]

     

    set payload_a [string length $payload]

     

    set payload_b [HTTP::payload length]

     

    HTTP::header insert "Content-Length-1" $payload_a

     

    HTTP::header insert "Content-Length-2" $payload_b

     

    HTTP::release

     

    }

     

     

     

    -------->>> This is my request:

     

     

    POST http://10.123.60.80/ HTTP/1.1

     

    Accept-Encoding: gzip,deflate

     

    Content-Type: text/xml;charset=UTF-8

     

    SOAPAction: ""

     

    User-Agent: Jakarta Commons-HttpClient/3.1

     

    Host: 10.123.60.80

     

    ---> Content-Length: 666 (only added for testing)

     

     

     

     

     

     

    ComverseVMS

     

    41767778570

     

    41767778570

     

    1

     

    1

     

    1

     

    1

     

    Message

     

     

     

     

     

     

    so if i sent a request that includes Content-Length in the http header i get for both Content-Length-1 and Content-Length-2 the correct value (666).

     

     

    If the request does not include the Content-Length header parameter i dont get anything. Nothing gets added to the outgoing request header.

     

     

    i would appreciate any hints on this.

     

     

    /marc
  • Hi Marc,

     

     

    I assume the client is trying to terminate the HTTP request by closing the TCP connection as it doesn't include a Content-Length header and doesn't set a Transfer-Encoding method of Chunked. An F5 developer should probably be able to provide a more informed suggestion, but in the meantime, you might be able to collect the TCP payload, count the number of bytes in the HTTP headers, calculate the HTTP payload size and then insert an HTTP Content-Length header. I can't see this being very efficient though.

     

     

    Is there any chance of changing the client application to explicitly set a content-length header?

     

     

    Aaron
  • i've run into a very similar issue - our cdn will not cache content without a content-length header. this is an issue with dynamically-generated content, but many of our tomcat apps will not include the header even if it's a static page.

     

     

    to my knowledge the best way to do this is to unchunk first, collect payload, add header, then send it on it's way. it seems like unchunking and converting to http 1.0 would be a performance hit on large files but i haven't tested it yet.
  • nambrosch, your case is slightly different than Marc's in that you're concerned about responses without a content-length header where Marc is having an issue with requests without a content-length header. The HTTP profile option for chunking is only used for responses, so I don't think that will help for Marc.

     

     

    Aaron
  • thanks for the suggestions. And yes i am concerned about the requests not having Content-Length in the header.

     

     

    @Aaron

     

    The client app cannot be changed so easily. And that is why i wnated to come up with an easier solution through irules...

     

    Concerning your suggestion. Did you mean something like this in your previous reply?

     

     

    when CLIENT_ACCEPTED {

     

    i can put here 1, 2, 512, 700 the results dont change...

     

    TCP::collect 2

     

    }

     

    when CLIENT_DATA {

     

    set clength [expr {[TCP::payload length] - 189}]

     

    }

     

    when HTTP_REQUEST {

     

    HTTP::header replace "Content-Length" $clength

     

    }

     

     

    It returns some value now. But i am not sure though this is the way this should be done...

     

    Can i be sure the header is always 189 bytes?

     

    How do i know the value i should provide to TCP::collect. It seems, no matter what value i put, i always get the same result.

     

     

    marc
  • Hi Marc,

     

     

    I think you'd need to search for the first instance of two \r\n (carriage return + line feed, carriage return + line feed) as that is what should delineate the HTTP headers from the payload. I'm not sure if you could use something like 'string first \r\n\r\n [TCP::payload]' or whether you'd need to convert the payload to hex first using 'binary scan [TCP::payload] H* payload_hex (Click here)' and then search for the CR LF CR LF characters using 'string first $payload_hex \x0d\x0a\x0d\x0a'.

     

     

    And unfortunately, I'm not sure if there is a more efficient way to do this. This approach would mean you'd be collecting every TCP payload in order to work around the issue for one client.

     

     

    Aaron