External Monitor Causing CPU to Spike

We run a set of HTTP streaming servers behind an F5 Virtual Server/Pool. Due to the HTTP streaming specification, we have to send video in 30 seconds increments. As such we need a monitor on the pool to disable a member node instead of marking it as down to allow for current streams to continue but not allow new connections to be made. We use an external monitor to check a web page that is hosted on each of the pool members. Basically it's a binary check of text on a webpage to return a 0 (means enable the pool member) or 1 (disable the pool member). However when we run the monitor we notice via top command I see load on the F5 spike up to 6 or 7 and as high as 12-15. This causes connections to be dropped frequently from the F5. There are roughly 25 members in the pool this monitor is assigned to. We run the monitor every 30 seconds with a timeout of 90 seconds. Does anyone have any idea as to what we can do to less the CPU load on our F5? Below is the code written for our external monitor.

-------------------------------------------------------BEGIN MONITOR----------------------------------------------------------------------

!/bin/bash

=================================DESCRIPTION=======================================

This monitor scans for the HTTP Streaming Server Shutdown flag, and if found, it

disables the node. If not found, it re-enables the mode.

This is DIFFERENT than saying the node is UP or DOWN.

A "down" node will have persistence table entries erased, but a "disabled" node will

not. So this "monitor" always reports a node as being "up", which means it should

NOT be used as a health check.

This script needs to be installed in /usr/bin/monitors and a health monitor

of type EAV configured to use it, with the following arguments set:

NODE_ADDR - Implicit first parameter

NODE_PORT - Implicit second parameter

POOL_NAME - The name of the pool that will be checked.

=================================START OF SCRIPT=======================================

---------------------------NON-USER-MODIFIABLE VARIABLES-------------------------------

STARTTIME=`date +%s%N`

PIDFILE="/var/run/monitor_$1..$2.pid"

NODE_ADDR=`echo $1 | sed 's/::ffff://'`

NODE_PORT=`echo $2`

POOL_NAME=`echo $3`

if [ "$POOL_NAME" == "" ]

then

POOL_NAME=`echo all`

fi

------------------------END OF NON-USER-MODIFIABLE VARIABLES---------------------------

----------------------------------HOUSEKEEPING-----------------------------------------

PIDFILE stores the process id of this script to avoid multiple runs.

Kill last instance if still running, and update so script can be killed on hang.

[ -f ${PIDFILE} ] && kill -9 `cat $PIDFILE` 2> /dev/null

echo "$$" > $PIDFILE

-------------------------------END OF HOUSEKEEPING-------------------------------------

-------------------------------BEGIN SCRIPT LOGIC--------------------------------------

Send a GET request to the server and parse the response for the Shutdown=0

RESPONSE=`echo "GET /mcdata HTTP/1.1\r\nHost: \r\nConnection: Close" | /usr/bin/nc $NODE_ADDR 5700`

RESPONSETIME=`date +%s%N`

TIMETORESPONSE=$((( $RESPONSETIME - $STARTTIME ) / 1000000 ))

ALIVE=`echo $RESPONSE | grep "DVC Server"`

SHUTDOWN=`echo $RESPONSE | grep "Shutdown=1"`

QUERYTIME=$RESPONSETIME

output=""

if [ "$ALIVE" != "" ]

then

cmd=""

ISENABLED=`b pool $POOL_NAME member $NODE_ADDR:$NODE_PORT show | grep enabled`

QUERYTIME=`date +%s%N`

if [ "$SHUTDOWN" == "" ]

then

if [ "$ISENABLED" == "" ]

then

output=`echo "up - $NODE_ADDR:$NODE_PORT should be enabled"`

cmd=`echo "b pool $POOL_NAME member $NODE_ADDR:$NODE_PORT session enable"`

else

output=`echo "up - $NODE_ADDR:$NODE_PORT is already enabled"`

fi

else

if [ "$ISENABLED" != "" ]

then

output=`echo "up - $NODE_ADDR:$NODE_PORT should be disabled"`

cmd=`echo "b pool $POOL_NAME member $NODE_ADDR:$NODE_PORT session disable"`

else

output=`echo "up - $NODE_ADDR:$NODE_PORT is already disabled"`

fi

fi

Issue the pool member command

if [ "$cmd" != "" ]

then

`$cmd`

fi

fi

COMMANDTIME=`date +%s%N`

TIMETOQUERY=$((( $QUERYTIME - $RESPONSETIME ) / 1000000 ))

TIMETORUN=$((( $COMMANDTIME - $QUERYTIME ) / 1000000 ))

TIMETOTAL=$((( $COMMANDTIME - $STARTTIME ) / 1000000 ))

DEBUG CODE

echo Inputs: NodeAddr: $NODE_ADDR, NodePort: $NODE_PORT, PoolName: $POOL_NAME >> /tmp/disable_shutdown_mon.log

echo Response: == $RESPONSE == >> /tmp/disable_shutdown_mon.log

echo Alive: == $ALIVE == >> /tmp/disable_shutdown_mon.log

echo Shutdown: == $SHUTDOWN == >> /tmp/disable_shutdown_mon.log

echo Enabled: == $ISENABLED == >> /tmp/disable_shutdown_mon.log

echo Command: $cmd >> /tmp/disable_shutdown_mon.log

echo Output: $output >> /tmp/disable_shutdown_mon.log

echo Time\(ms\): resp=$TIMETORESPONSE, qry=$TIMETOQUERY, cmd=$TIMETORUN, tot=$TIMETOTAL >> /tmp/disable_shutdown_mon.log

END DEBUG CODE

if [ "$output" != "" ]

then

Now, echo up command to standard output

echo $output

fi

Clean up the working files and exit cleanly.

rm -f $PIDFILE

exit

-------------------------------END SCRIPT LOGIC----------------------------------------

================================END OF SCRIPT==========================================

-------------------------------------------------------END MONITOR-------------------------------------------------------------------------