Forum Discussion

Ed_Summers's avatar
Ed_Summers
Icon for Nimbostratus rankNimbostratus
Jul 31, 2013

Monitor not pollling all pool members

I created a simple http monitor that sends a GET request and expects a 200 OK response. This monitor uses the alias port 8080 as it is intended for a pool whose members listen on port 443. The client has a dependency service on the server that will respond on port 8080. This way, if the dependency server is down, the pool member will be marked down even if the https service is up.

 

I created the health monitor as such:

 

myMonitorName {

 

defaults-from http

 

destination *:8080

 

interval 5

 

recv "HTTP/1.1 200 OK"

 

send "GET /something/something HTTP/1.1\r\nAccept:*/*\r\nHOST:fqdn.is.here:8080\r\nConnection: close\r\n\r\n"

 

time-until-up 0

 

timeout 16

 

}

 

 

I applied the monitor to the pool, which already has the "gateway_ICMP" and "https" monitors applied. Based on the output of tcpdump, polling is only being done on the first listed of two pool members. I see the polls for a.a.a.1, but not for a.a.a.2. However, both pool members are marked as AVAILABLE.

 

I first noticed this when I was troubleshooting the monitor. I had some issues with my SEND string, and when applied to the pool only the first member (a.a.a.1) was marked down. Once I fixed the monitor the first member was marked as up, but the monitor never appeared to affect the second member.

 

tcpdump from the LTM appears to confirm this. I see the traffic when filtering to host a.a.a.1, but no traffic when filtering to host a.a.a.2 (port 8080). Note that if I remove the port restriction and view all traffic to host a.a.a.2, I do see monitor traffic coming from the gateway_icmp and https monitors.

 

 

I cannot figure out why the LTM is not applying the monitor/polling the second pool member. Any suggestions appreciated.

 

F5 BigIP LTM ver 10.2.3 Build 123.0 Hotfix HF1

 

 

Thanks,

 

Ed

 

4 Replies

  • not sure if you can restart bigd and whether it is helpful.

     

     

    tmsh restart sys service bigd
  • Nice dog, btw! Reminds me of mine.

     

     

    I'll do some research into restarting the bigd service. I did review top to see if there were any suspicious memory or CPU usage figures. Odd that statsd shows a high CPU usage (110% or above), but overall CPU utilization lists quite low when viewed on the GUI (<20% peak for all CPUs). It's an LTM-8900 in case that is of interest.

     

     

    I need to research how disruptive a restart of the bigd service would be before attempting. As this is a production device I can't trade downtime of some other systems to troubleshoot this monitor. Thanks for the feedback! I'll start looking into that today.
  • I did review top to see if there were any suspicious memory or CPU usage figures. Odd that statsd shows a high CPU usage (110% or above), but overall CPU utilization lists quite low when viewed on the GUI (<20% peak for all CPUs).sol3242: Overview of BIG-IP Traffic Management Microkernel (TMM) CPU and RAM usage

     

    http://support.f5.com/kb/en-us/solutions/public/3000/200/sol3242.html

     

     

    I need to research how disruptive a restart of the bigd service would be before attempting.sol6967: When the BIG-IP LTM bigd daemon fails, the health check status of pool members, nodes, and services remain unchanged until the bigd daemon is restarted

     

    http://support.f5.com/kb/en-us/solutions/public/6000/900/sol6967.html

     

     

    Nice dogdefinitely he is. :D

     

     

    hope this helps.
  • Colleague found the issue. He was right when he told me I'd laugh when I heard the problem.

     

     

    The second pool member (a.a.a.2) had the option "Member specific" selected for health monitors. I had (attempted) to look for this before, but apparently looked in the wrong place. Returning this value to "Inherit from Pool" got things back on track. We're not sure who would have set the value, probably by mistake, but good lesson for the future.

     

     

    As you can see I'm still learning quite a bit about the more detailed features. Thank you sincerely for your help!