Learn F5 Technologies, Get Answers & Share Community Solutions Join DevCentral

Filter by:
  • Solution
  • Technology
Answers

Problem with iRule that logs excessive HTTP header lengths

Background:

We had been seeing occasional errors in /var/log/ltm that look like this:

HTTP head (33182) exceeded maximum allowed size of 32768

Before increasing MAX HTTP header size across the board, we thought we'd increase the MAX Header size to 64k (doubling it), on one of our VIPs and create an iRule to log HTTP Headers when the total length of them exceeds 32768. 

When the threshold is set much higher than 15000, we see TMM errors associated with the iRule( http-header-size) that look like this:

Oct 19 09:16:31 local/tmm3 err tmm3[6715]: 01220001:3: TCL error: http-header-size - Not found (line 1) invoked from within "HTTP::header names"

When the threshold is set low, the iRule works correctly and we see log entries that look like this:

Oct 19 12:29:17 local/tmm5 info tmm5[6717]: Rule http-header-size : Header length = 1311 Oct 19 12:29:17 local/tmm5 info tmm5[6717]: Rule http-header-size : Accept (164): image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-powerpoint, application/vnd.ms-excel, application/msword, */* Oct 19 12:29:17 local/tmm5 info tmm5[6717]: Rule http-header-size : Referer (33):

and so forth

Can anyone offer any insights as to why the iRule fails when the logging threshold is set above 10000 or so?


Here's the iRule - please forgive the formatting

when HTTP_REQUEST {
    # Loop through each header by name
        set count 0
           foreach aHeader [HTTP::header names] {
                  incr count [string length [HTTP::header value $aHeader]]
           }
                       if {$count > 32768} {
                               # Log details for the request
                                log local0. "Header length = $count" 
                                    foreach aHeader [HTTP::header names] { log local0. "$aHeader ([string length [HTTP::header value $aHeader]]): [HTTP::header value $aHeader]" 
                                    }
                       }
}


0
Rate this Question

Answers to this Question

placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
Hi Evan,

You may want to take a look at this solution:
SOL8482: Error Message: HTTP header exceeded maximum allowed size of
http://support.f5.com/kb/en-us/solutions/public/8000/400/sol8482.html?sr=17162762


Note: F5 recommends that you allocate just enough bytes required to handle the HTTP headers. Since buffer space is allocated to each request handled by the profile, an excessive value may result in memory exhaustion.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
Thanks, Michael. I should have mentioned that the objective was to increase MAX HTTP Header size temporarily so we could capture some of the excessively long headers and have our development team determine if they were valid. If they're valid, we would increase MAX HTTP Header size across the board.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
In F5 land, the HTTP headers also include the initial request line (method and URI). You could do the following
when HTTP_REQUEST {
set hlength [string length [HTTP::query]]
if { $hlength > 32768 } {
log local0.info "Header length = $hlength, [HTTP::query]"
}
}
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
i think HTTP::query does not include method and full uri. do i misunderstand?

HTTP::query wiki
http://devcentral.f5.com/wiki/iRules.HTTP__query.ashx
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
Follow-up

Turns out my iRule did exactly what it was supposed to-namely log data regarding HTTP requests with excessively long HTTP headers. Not sure what those TMM errors were all about. In any case I have the data I need and appreciate everyone's help!
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
Follow-up

Turns out my iRule did exactly what it was supposed to-namely log data regarding HTTP requests with excessively long HTTP headers. Not sure what those TMM errors were all about. In any case I have the data I need and appreciate everyone's help!
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

A lot late to the party, but I needed to do the same.

I noticed one thing, the total header size calculation is slightly wrong. It only counts the total for the header values, but doesn't include the header names themselves.

For instance, the header “Connection: keep-alive”, it was only counting the string length of "keep-alive" it was counting that as 10 bytes and not the correct 22 bytes for that actual line of header, if you have a lot of header with long names and short values it can throw your count by a reasonable amount.

You need to add together the string length of the header name, the string length of the header value, plus 2 (this account for the ": " between the header name and value.

In our case we have upped the max header value to 64k, and we're using this to work out what the actual maximum is.

So to remedy:

when RULE_INIT {
    set static::Header_Alert_Size 32768
    }

when HTTP_REQUEST {
        set header_total 0
        foreach header [HTTP::header names] {
            incr header_total [string length $header]
            incr header_total 2
            incr header_total [string length [HTTP::header value $header]]
        }
        if { ($header_total > $static::Header_Alert_Size) } {
                set LogString "Client [IP::client_addr]:[TCP::client_port] -> [HTTP::host][HTTP::uri]"
                log local0. "$LogString - Header Total $header_total bytes"
        }
    }
0