While using the following monitor, I'm finding that the web servers are keeping TCP sessions in the TIME_WAIT state, rather than closing them outright. GET /serverin.html HTTP/1.1\r\nConnection: Close\r\nHost: \r\n\r\n My understanding was that by specifying HTTP 1.1, with the Close and carriage return, I'd be forcing the FIN and FINACK thereby closing the connection. Unfortunately, at any given time, I've got 3000 plus TIME_WAIT states on my web server courtesy of the F5. Obviously I'm missing something here. Is it possible to use HEAD instead of GET within the send string to correct this? Or is there something else I need to add to the existing string to force an outright closing of the connection? I've checked the archives here and am not finding a whole lot on the use of HEAD within a monitor. Any assistance is greatly appreciated.

I thought the Connection header is a suggestion to the destination host that the sender doesn't intend to reuse the TCP connection. I think it might be up to the recipient to close the TCP connection. RFC2616 doesn't seem to state this clearly, but I've read it other places: http://www.jmarshall.com/easy/http/http1.1s4 If a request includes the "Connection: close" header, that request is the final one for the connection and the server should close the connection after sending the response. Also, the monitoring daemon, bigd, may append one or more CRLF's to the end of the send string in HTTP/S monitors depending which LTM version you're using. This is described in SOL2167: SOL2167: Constructing HTTP requests for use with the HTTP or HTTPS application health monitor https://support.f5.com/kb/en-us/solutions/public/2000/100/sol2167.html If you're on 9.4 or higher, can you remove the two \r\n's from the end of the send string and see if the connections are still left open for the full TIME_WAIT period? I don't think using HEAD instead of GET will change the LTM or server behavior. That would also limit you to checking only the response headers (the server won't respond with a payload to a HEAD request. If this still doesn't work after fixing the CRLF's, can you clarify what type/version of OS the server is? Aaron

Thanks for that advice. I applied your suggestion of removing the two explicit CRLF earlier today, but checking back in three hours later, a "netstat -n" on the IIS server still shows over 3000 connections in TIME_WAIT. GET /serverin.html HTTP/1.1\r\nConnection: Close\r\nHost: The receive string is still simply a 200 OK. HTTP/1.1 200 OK My LTM is currently running 9.4. Our web servers are on Windows 2K3, with IIS 6.0. I've done a TCP dump on the LTM and it looks like the FIN bit is set in the server response. I guess my next step is to do a packet capture with Wireshark for some more detailed output, and see that the Close is in the HTTP header sent by the LTM. If I see anything out of the ordinary, I'll post it here. Thanks again.

I did get output from an ethereal packet capture. The "close\r\n" is included in both the HTTP header request from the LTM and the response from the IIS server. It's obvious however that the IIS box isn't doing what it "agreed" to do and closing the connection outright. Still leaving over 3000 connections in time_wait. Does anyone else have this issue using the built in http monitor template? I'm inclined to ask for help with an advanced monitor, but don't really see where that would help as I believe I have evidence in hand that it's an IIS issue. But I certainly can't be the only one experiencing this? Or, are hundreds, if not thousands, of time_wait connections to be expected between LTM and IIS?

This really isn't a monitor issue, and I always recommend that you add a connection: close, which will notify your web server to close this socket down and be prepared to accept another client request as opposed to maintaining a keep-alive connection (and occupying a socket that could happily service requests) to the LTM even though it won't be re-used. TIME_WAIT status is an artifact of the web server's TCP/IP stack and is common. For single-purposed systems - i.e. a web or app server that does a single task, serving HTTP/S pages - it's common to tune the TIME_WAIT_REUSE and TIME_WAIT_RECYCLE timers on the server's OS to help minimize sockets sitting in this state. I don't know how to do this in Windows, but I'm sure some of the good folks out there will be able to chime in... -Matt

We're having this exact same issue. We did tune our TIME_WAIT timers (and as a good measure increased our max ports), but I'd still love to find a way around this if anyone has one. For record, the Registry changes for Time_Wait and max ports are (obviously use these at your own risk... I place no warranty on registry suggestions) To add and configure the MaxUserPort parameter: 1. Start Registry Editor (Regedt32.exe). 2. Locate the following key in the registry: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters 3. Right-click on Parameters and select New DWORD Value. Type MaxUserPort in the Name data box, type 65534 (Decimal) in the Value data box, and then click OK. NOTE: The default setting for the MaxUserPort value is 5000 (Decimal). 4. Quit Registry Editor. To add and configure the TcpTimedWaitDelay parameter: 1. Start Registry Editor (Regedt32.exe). 2. Locate the following key in the registry: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters 3. Right-click on Parameters and select New DWORD Value. Type TcpTimedWaitDelay in the Name data box, type 60 (Decimal) in the Value data box, and then click OK. NOTE: The default setting for the TcpTimedWaitDelay value is 240 (Decimal), which equals four minutes. Never set this value below 30. 4. Quit Registry Editor.

HTTP monitor with HEAD, not GET

15 Replies

hooleylist
Cirrostratus
Apr 28, 2011
Hi Darren,

Which LTM version are you running?

What's your monitor config look like?

What server OS/version is being monitored?

How long to the connections stay in TIME_WAIT?

If you run a tcpdump between an LTM static self IP and one pool member, how are the connections being handled after a response is received by LTM?

Aaron
Dazzla_20011
Nimbostratus
May 03, 2011
Hi Aaron,

Version 10.1.0

GET /ApplicationFacade.svc HTTP/1.1\r\nHost:\r\nConnection: Close\r\n\r\n

The connections stay in time_wait for 2 minutes on the servers.

I've done a tcpdump is there anything in particular I should be looking for after a response is received by the LTM?

Many Thanks

Darren
Koliper_51942
Nimbostratus
May 05, 2011
tnx for info
Vitaly_96204
Nimbostratus
Jun 24, 2012
If we do not want to have tons of TCP connections in TIME_WAIT state, can we set TIME/WAIT Recycle to disable and/or TIME/WAIT Immediate (system closes the connection immediately after the connection enters the TIME/WAIT state).
L4L7_53191
Nimbostratus
Jun 24, 2012
Yes, although I'm not certain which side of the connection flow you're referring to. I'll assume you're talking about the server side.

In that case, you'll want to keep both time_wait recycle and re_use enabled, as it'll leave those sockets open as candidates for new connections if the stack deems it necessary. TIME_WAIT is something that the stack really should do, although the duration of that interval (by default 2*MSL) is somewhat a matter of debate, especially behind an LTM with a super smart TCP stack...

By the way, not all OS's (linux, for one) allow you to tickle the TIME_WAIT interval - you'll see some bad information out there on the web, so tread lightly (e.g. setting the FIN_WAIT2 timer). So it's best to leave recycle/reuse enabled and let the stack sort it out as it needs to.

HTH,

--Matt Cauthorn

Forum Discussion

HTTP monitor with HEAD, not GET

15 Replies

Recent Discussions

Wildcard SSL Certificate Deployment on F5 LTM

Migration from i series 10200 with 1 child VCMP to r series 10900 series

Open Redirection Mitigation

GTM Packet Capture command

How to Implement 2 Way SSL in F5 LTM

Related Content

Cannot modify Alias Service Port of a HTTP monitor

HTTP Health Monitor Failing, Not Sure Why

Monitor problem

HTTP Monitor

Custom Header Monitor UDP