Cookie persistence and disabled servers

We’ve made ourselves a relatively interesting problem.

We run a fair few java containers from tomcat. They supply their connection to the world via the http connector using the apache portable runtime for speed. This is then load balanced and accelerated with our F5 load balancers.

We have created a custom monitor that looks at a URL of /monitor/ in the tomcat container which provides a status fragment in XML. This is so that developers can disable a pool member by logging into a custom app and marking the container as unavailable.

Normally the tag is OK and it changes to DISABLED This works really well and lets us perform maintenance etc on the tomcat container that is marked as done. For example, they can mark the node as disabled, undeploy an application, deploy the new version, check that everything is OK, reenable the container and then repeat the process on the other nodes. Giving us a very seamless upgrade process.

The problem is that cookie persistence still sends traffic to a node even if it is marked down.

e.g. Cookie says current server is server A, server A is marked as disabled so is disabled in the pool, traffic still goes to server A while the tomcat is running. If we turn off the tomcat so that a TCP reject is received this actually forces a cookie reset.

My question is, how do I get the cookie to reset when a server is marked down in the pool? I want it to reselect a new node for our clients at this point, not when the tomcat engine is actually shutdown.

I'm thinking an iRule might be the only way to do this. Surely it isn't an uncommon problem though?

We're using 10.0.1 on some 6900's.

Any hints, tips, suggestions or otherwise would be much appreciated,

Dave