Forum Discussion

CraigM_17826's avatar
CraigM_17826
Icon for Altostratus rankAltostratus
Mar 16, 2010

External Monitor not working

Hi all,

I am having some odd issues with a monitor. I have a web application I need to monitor and for whatever reason a HTPS monitor does not seem to work. The status of the pool always stays as "Unknown (Enabled)"

In desperation I am now trying to use a external monitor. Using the sample_monitor.sh script in /usr/bin/monitors as a basis for my monitor. The script uses three arguments, hostname/ip, URI and expected string. If the output of curl contains the extected string then it returns UP, else it returns DOWN.

Now the script runs and works as expected when I execute it from the command line, but when I create a external monitor to run this script with the required parameters the status of the pool remains as "Unknown". I decided to put in a debug line in the script that creates a logfile and writes to the logfile at certain points within the script. This logfile is never created which seems to inidicate the monitor is not being run.

Any ideas on why this would be the case? All I see is this in the LTM logfile once I apply the monitor to my test pool

Mar 16 15:59:12 local/bigip-u1 notice mcpd[1657]: 01070638:5: Pool member nnn.nnn.nnn.nnn:443 monitor status unchecked.

Here is the monitor shell script

   
   !/bin/sh   
      
   LOGFN="/tmp/exam-services.log"   
   touch $LOGFN   
   echo "Starting exam-services monitor" >>$LOGFN   
      
    remove IPv6/IPv4 compatibility prefix (LTM passes addresses in IPv6 format)   
   IP=`echo ${1} | sed 's/::ffff://'`   
      
   PIDFILE="/var/run/`basename ${0}`.${IP}.pid"   
    kill of the last instance of this monitor if hung and log current pid   
   if [ -f $PIDFILE ]   
   then   
      kill -9 `cat $PIDFILE` > /dev/null 2>&1   
   fi   
   echo "$$" > $PIDFILE   
   echo "PIDFILE $PIDFILE created" >>$LOGFN   
      
    send request & check for expected response   
    using -k as Monash use a self signed certificate   
      
      
   echo "Executing CURL command" >>$LOGFN   
   curl -fNsvk https://${IP}/exam-services/student 2>&1 | grep -i "/exam-services" 2>&1 > /dev/null   
    mark node UP if expected response was received   
   if [ $? -eq 0 ]   
   then   
       echo "UP"   
       echo "Exam Services Website is up." >>$LOGFN   
   else   
       echo "DOWN"   
       echo "Exam Servicews Website is down." >>$LOGFN   
   fi   
      
   rm -f $PIDFILE   
   echo "Exiting exam-services monitor" >>$LOGFN   
   exit   
   

and in the health monitor GUI setup screen I have

External Program: /usr/bin/monitors/examservices_monitor.sh

Arguments:

I have actually tried just specifying the scipt name unqualified in the External Program field as well and there was no difference.

Everything else is left at their default values.

As I mentioned, it all works when I run from the command line.

Regards

Craig

p.s. Sorry, omitted to mention we are running 10.0.1 Build 378.0 with HF3.

20 Replies

  • Hi,

    the two boxes are identical models running the same software release (10.0.1 B378.0 with HF3). As I understand it we have a fairly bog standard high availability setup (active/standby).

    I was just about to force a failover to the standby unit and thought I would just check to see if it all looked ok and now the pool that was using the external monitor is flagged as down. If I run the script manually it reports the remote system as up. This is very confusing.

    I tried restarting the bigd service on the standby unit and it made no difference. The LTM log is reporting that it thinks the members in the pool are down

     
     Mar 18 10:38:52 local/bigip-u2 notice mcpd[1672]: 01070638:5: Pool member nnn.nnn.nnn.nnn:443 monitor status down. 
     Mar 18 10:38:52 local/tmm err tmm[2485]: 01010028:3: No members available for pool Test_HM 
     Mar 18 10:43:20 local/bigip-u2 notice root: -bash  ==> /usr/bin/bigstart restart bigd 
     Mar 18 10:43:20 local/bigip-u2 notice mcpd[1672]: 01070410:5: Removed subscription with subscriber id BIGD_Subscriber 
     

    I'll reboot the standby unit and see if there is any change from a fresh reboot.

    Craig
  • Ok, after a clean reboot the pool the monitor is applied to is flagged as down now, depsite the fact when I run the script from the command line it works and reports the remote server as up.

     

     

    This is all very confusing and somewhat frustrating. Might have to log a call over this one as I am running out of ideas.

     

     

    Regards,

     

     

    Craig
  • If you still want to experiment, try creating a simple monitor, eg.

      
     !/bin/sh   
     echo $1 $2 >> /tmp/test.log   
     echo UP  
     

    Do you see /tmp/test.log file increasing and the pool member UP?
  • Hi Hum,

     

     

    I'l give it a go, but to add to the confusion, the orginal internal health monitor I had which was NOT working has now started to work on the standby unit. There aren't any issues having monitors querying the same pool members is there? It almost seems to be when one monitor works the other fails.

     

     

    The internal monitor was just doing a

     

     

    GET /exam-services/student HTTP/1.1\r\nHost: \r\nConnection:Close\r\n

     

     

    and expecting HTTP/1.1 200 OK

     

     

    Although this is a HTTPS monitor it was suggested I didn't need to specify a client certificate or client key because the monitors don't use them.

     

     

    Regards,

     

     

    Craig
  • Hi Hum,

     

     

    just tried creating that simple monitor and assigning it to a poo. They pool is flagged as down and no log file is created.

     

     

    I'm getting quite concerned that something is not right with our units.

     

     

    Regards,

     

     

    Craig
  • Hi Hum,

     

     

    Will do. It's getting beyond me. Thanks for all you help all the same.

     

     

    Craig
  • Don't mention it .. It would be my pleasure if I could help you. Feel free to let us know if you need help ...
  • Hello, I've got a similar issue: http://devcentral-sea.f5.com/Forums/tabid/53/afv/topic/aff/32/aft/1172321/Default.aspx Have you had any luck? I'm about to raise a support call with F5.

     

     

    Running BIG-IP 9.4.7 Build 320.1 Final
  • Hi There,

     

     

    I have the very same problem. My script works just fine from the command line but F5 seems to not be able to run the script. Nothing is logged in the script debug file and the server is marked as down. Is there any update on this bug?
  • Nadim's avatar
    Nadim
    Icon for Nimbostratus rankNimbostratus

    I had a similar issue and found that converting the fileformat=unix, within vi editor, then save resolved issue.