External monitor "state returned down" response from incorrect IP
I have a combined GTM/LTM system running BIG-IP Version 11.4.1 Build 608.0 Final (Let's call this Device 1).
I have several pools of virtual servers in the Global Traffic component, and I have created an external monitor to check the availability of these virtual servers, and hence marking them online/offline accordingly. The external monitor currently checks a static web page hosted on a web server via port 80. The monitor greps for the string "=up" in order to mark the virtual server as available.
This works as expected on a BIG-IP GTM Virtual Edition running v11.3.0 Build 2806.0 Final (no LTM component, let's call this Device 2). I can run a tcpdump packet capture on the external interface and see the device sending HTTP requests to the IP address of the web server, as expected.
However, on Device 1, the same tcpdump returns nothing. Not even for any of the other interfaces. I can run a cURL request from the console successfully, which verifies that there are no firewall issues - but I can't work out why I'm not seeing the GTM try to reach the web server.
On Device 1, when I go to Global Traffic > Pools > Pool List and check out the members of a pool, I can mouse over the red icon in the Status column, and get one of the two below tooltips:
Offline (Enabled) - Monitor /Common/HTTPMonitorcURLGET from 192.168.1.125 : state returned down
Offline (Enabled) - Monitor /Common/HTTPMonitorcURLGET from 192.168.0.251 : state returned down
These are not the IP addresses the monitor should be using! As far as I understand, from GTM, Know Thyself, the self-IP of the GTM's internal interface should be displayed here (xxx.xxx.xx.126 = 192.168.0.240). The correct IP is shown for Device 2.
This is the Listener and Server Object configuration for Device 1:
Device 1 External UDP listener Destination: 192.168.0.240
Device 1 Internal UDP listener Destination: 192.168.1.125
Device 1 GTM Self-IPs: 192.168.1.125, xxx.xxx.xx.126 = 192.168.0.240
Device 1 LTM Self-IPs: 192.168.0.251
Note that the GTM has both the Internal and External interfaces set as self-IPs.
My question is: how do I force the monitor HTTPMonitorcURLGET to use the 192.168.0.240 (external) interface, instead of the internal GTM or LTM interfaces! Why does a packet capture not even show packets going to the web server from the internal interface? Device 1 is a live system, so I can't change things on a whim without knowing (a) it won't break anything else and (b) that it will work, beyond reasonable doubt!
HTTPMonitorcURLGET script:
!/bin/sh
These arguments are supplied automatically for all external monitors:
$1 = IP address of the virtual server (IPV6 notation. IPv4 addresses are
passed in the form ::ffff:w.x.y.z, where "w.x.y.z" is the IPv4 address)
$2 = port of the virtual server (decimal, host byte order)
Name/Value pairs may also be specified in the monitor template.
This example expects the following Name/Value pairs:
URI = the URI to request from the server (e.g. "/testpath/testfile.html")
RECV = the expected response (not case sensitive) e.g. "UP"
B = web server IPv4 addresses, separated by a ":"
DEBUG= enable logging to local0.debug if set to 1, no logging if set to 0
Log debug to local0.debug
if [-n "$DEBUG" ]
then
if [ $DEBUG -eq 1]
then
echo "EAV `basename $0`: \$DEBUG: $DEBUG" | logger -p local0.debug
fi
else
If the monitor configuration didn't specify debug, enable/disable it here
DEBUG=0
fi
Remove IPv6/IPv4 compatibility prefix (GTM passes addresses in IPv6 format)
IP=`echo $1 | sed 's/::ffff://'`
Save the port for use in the shell command
PORT=$2
Check if there is a prior instance of the monitor running
PIDFILE="/var/run/`basename $0`.$IP.$PORT.pid"
kill off the last instance of this monitor if hung
if [ -f $PIDFILE ] if file $PIDFILE exists and is not a directory
then
echo "EAV `basename $0`: exceeded monitor interval, needed to kill"\
"${IP}:${PORT} with PID `cat $PIDFILE`" | logger -p local0.error
interface with syslog module with priority (-p) "error" in local0 facility
kill -9 `cat $PIDFILE` > /dev/null 2>&1 9 is the signal number
fi
Add the current PID to file
echo "$$" > $PIDFILE $$ is the PID of the current process
Debug
if [ $DEBUG -eq 1 ]
then
echo "EAV `basename $0`: Running for ${IP}:${PORT} using custom command" \
| logger -p local0.debug
fi
Split the B value into individual IP addresses
bld=$(echo $B | tr ":" "\n")
for i in $bld cycle through each web page, exit if "UP" is found
do
curl --connect-timeout 30 -fNs http://${i}:80${URI} | grep -i \
"${IP}=${RECV}" 2>&1 > /dev/null
if [ $? -eq 0 ] $? is the return code of the last executed command
then
rm -f $PIDFILE
if [ $DEBUG -eq 1 ]
then
echo "EAV `basename $0`: Succeeded for ${IP}:${PORT}" | logger -p \
local0.debug
fi
echo "UP"
exit exit from the script early if an "UP" is found
else
if [ $DEBUG -eq 1 ]
then
echo "EAV `basename $0`: Failed for ${IP}:${PORT}" | logger -p \
local0.debug
fi
fi
done
rm -f $PIDFILE