Forum Discussion

Tim_126716's avatar
Tim_126716
Icon for Nimbostratus rankNimbostratus
May 03, 2013

Load balancing by custom queue length

Hi all,

 

For my current project we'd like to have the F5 Load Balancer give more traffic to our less busy machines. The load is determined by the number of files in a directory, as each file is a batch task. Our servers are Windows Server 2008 R2, and we can install or create any kind of software required.

 

We've considered SNMP DCA monitors, but CPU and memory aren't good indicators of system performance, as each system will be identically loaded if it has even one batch job to process. SNMP DCA Base monitoring may work, we just have to work out how create a custom SNMP agent (likely easy enough with a little custom application) and then how to get the F5 to balance from that data. Our trained F5 guys are super busy so we have to muddle through ourselves.

 

Does anyone have any tips, references, or search terms that would point us in the right direction?

 

Many thanks,

 

Tim

 

7 Replies

  • i am not familiar with icontrol but i think it is correct way to go.

     

     

    Can we change the Weigth Ratio of a pool memeber via an Irule

     

    https://devcentral.f5.com/community/group/aft/21550/asg/50

     

     

    Controlling a Pool Members Ratio and Priority Group with iControl by Joe Pruitt

     

    https://devcentral.f5.com/tech-tips/articles/controlling-a-pool-members-ratio-and-priority-group-with-icontrol
  • I believe iControl is certainly a great option, with a few thoughts:

    1. You would have to have an iControl script running on each server that can make (iControl) calls to the BIG-IP and alter its own weight, or a central server that polls the individual servers and makes a global weight adjustment.

    2. If the former option above, because each server doesn't have knowledge of the other servers' loads, you'd have to have some static gauge by which to assign weight based on this server's number of files (ie. 2 files = weight 1, 3 files = weight 2, etc.). This of course wouldn't be a completely accurate scale.

    3. If the latter option above, you could query all of the servers, take an average, and assign weights accordingly. This option can also be performed in two places: an iControl running on a central server, or if you can place a little scripted page in your web servers that simply returns the count of files, you can do all of this from an external monitor on the BIG-IP. For example, here's a PHP script that returns the file count:

     
    
    

    Once you have all of the numbers in house, simply perform some local TMSH calls to adjust the weights of the individual members.
  • nitass, Kevin, thank you both for your replies. The PHP script idea sounds ideal. I'll do some reading on iControl and hopefully can work it out from here, though I'll post again if we need a little more direction help. Thanks again.
  • Tim, just to be clear, the PHP script example could be used from either side (central Windows server and iControl script, or BIG-IP external monitor and TMSH script). For my money I'd go with the external monitor script, but I'm prone to lean in the Linux direction more often than not. ;)

     

  • Thanks Kevin. Can you expand on the two options a little please, in phrases that an F5 beginner might understand? I understand you're saying the monitoring and control can be done from different locations, but I don't understand the F5 well enough yet to work out what's technically best, and which would take less effort. We want to load balance across the dozens of windows servers based purely on the size of their queues, over a certain threshold we probably want to remove it from the load balancer completely.
  • This is one of those moments when I wish the BIG-IP had a monitor that could reach into Windows and read the file system or see if services are running, but unfortunately those sorts of things take a little extra work to accomplish. In any case, you need to be able make load balancing decisions based on the number of files in some arbitrary folder across multiple machines, and then base the (total) load balancing ratio on the proportions of files up to some specific threshold. In order to make that calculation you need to first know how many files each server has, and that can be done from two places - a central Windows server or the BIG-IP via external monitor script.

     

     

    If using a central Windows server, you would probably only need to reach across the file system natively, using shares or perhaps a Powershell script, get the numbers of files to make your calculation, and then use that same Powershell script to make an iControl call out to the BIG-IP to alter the load balancing ratio of a set of pool members.

     

     

    If using the BIG-IP and external monitor, you'd need something that the monitor could query, perhaps a web server script (like the above PHP example) to get the number of files to make the calculation, and then make a local TMSH call to alter the load balancing ratio for a set of pool members. So if you think about it, each option is making a local call and a remote call, but just opposite of each other.

     

     

    Which version you think is better depends on a few things, like your knowledge of Windows or Linux scripting, and perhaps your ability to create a file-counting script to run in your web server. The calculation itself should/would be the same in either place.
  • Thanks for explaining that Kevin, it's really helpful. I'll look into iControl and the external monitor to see which looks easier to use. If I come up with anything useful or interesting I'll post back to the thread so others can learn from it.