user_alert.conf and automation - good stuff!

For a description on how user_alert.conf works & syntax rules, see

http://devcentral.f5.com/Community/GroupDetails/tabid/1082223/asg/44/aft/1178752/showtab/groupforums/Default.aspx

http://devcentral.f5.com/Community/GroupDetails/tabid/1082223/asg/44/aft/55956/showtab/groupforums/Default.aspx

http://support.f5.com/kb/en-us/solutions/public/3000/600/sol3667.html

http://support.f5.com/kb/en-us/solutions/public/3000/700/sol3727.html

http://support.f5.com/kb/en-us/solutions/public/12000/400/sol12428.html

With new messages available in 10.2 such as:

Aug 17 10:34:31 local/localhost notice mcpd[5571]: 01070727:5: Pool member 10.10.10.1:80 monitor status up.

Aug 17 10:34:31 local/tmm7 err tmm7[7389]: 01010221:3: Pool your-cool-pool now has available members

combined with the older and very useful messages:

Aug 17 10:34:22 local/localhost notice mcpd[5571]: 01070638:5: Pool member 10.10.10.1:80 monitor status down.

Aug 17 10:34:22 local/tmm9 err tmm9[7404]: 01010028:3: No members available for pool your-cool-pool

you can now perform actions on the F5 itself based on a pool becoming totally unavailable.

In the /config/user_alert.conf file, you can include statements like:

/*  This works with my cool pool */
alert your-cool-pool-DOWN "No members available for pool your-cool-pool" {
   exec command="/config/monitors/MyScript 'your-cool-pool' 'down'"
}
alert your-cool-pool-UP "Pool your-cool-pool now has available members" {
   exec command="/config/monitors/MyScript 'your-cool-pool' 'up'"
}

This provides a complete solution to taking an action when a pool goes up or down.

WARNING: Since alertd only examines the /var/log/ltm syslog-ng message body, you can not detect which TMM instance (CPU) cut the message. Since the script has no access to the complete message that triggered the execution, you are left with the challenge to develop your own concurrency collision race condition handler through a lock file of some sort.