We are currently experiencing an issue in our data center. We have two F5 LTM’s with VIPs for our database cluster and webservers. In the data center, we run HP ProLiant Servers with Fault Tolerant Load Balancing (TLB) NIC teams and two Cisco 3560E switches with Etherchannel Layer 2 trunks. We have two F5 LTMs that are currently in an Active/Standby configuration. On the first LTM, we have interface 1.1 going to switch 1 and interface 1.2 going to switch 2. On the second LTM, we have the same configuration. Please see the simplified topology concerning the connection to the switches. Over the weekend, we removed foundry switches and replaced those with the Cisco 3560E’s. Since this change over we have had a few issues with no resolution to date. First, when we try to access the VIP for our webservers on the F5 LTM by HTTP/HTTPS it does not resolve. If we try to access the LTMs HTTPS web address, it does not resolve either. However, we can access all servers using their physical address with HTTP/HTTPS. What is really weird is that we can ping the VIP and LTM address. We do not currently have an access-list on any device denying this traffic. Also, when we removed a NIC from the team, we could resolve the VIP and LTM by HTTP/HTTPs. The second issue is that spanning-tree is blocking the redundant interfaces on our second switch. Not sure why this is happening if the LTM is in an Active/Standby state and it must be noted that we are using STP pass through. Hopefully someone reading this has experienced this before or has an idea/suggestion for a resolution. We have opened a ticket with F5, but no resolution yet. We opened a case with Cisco TAC and they have reviewed the switch configuration and everything looks good.

Can you post your switch config and your bigip_base.conf file? Where do the web servers plug in here? Not being able to resolve a hostname doesn't really fit into the picture you sent.. Do you have GTM(s) running anywhere? If it truly is a DNS resolution issue, what's your DNS architecture like? I think you might be using the word "resolve", but really meaning you can't bring up the content you're looking for when hitting the VIP address.. Correct?

DNS is working like is should, so no complaints here. For example, I cannot go to http://172.23.10.10 (VIP) or https://172.23.10.8 (F5 LTM web management). No GMT running in our network Ports 1.1 & 1.1 from both F5 LTM connect to SW1 Ports 1.2 & 1.2 from both F5 LTM connect to SW2 See the attached switch configs SW1-SW2 Etherchannel are ports 49-50 & F5 LTMs connect to ports 37-38 on both switches I will work on getting you those config files from the F5s. Issues: 1) When NICs are teamed with TLB, I cannot access the server VIPs or either LTM by HTTP/HTTPS. (Remove one NIC from the team and it works...also works with NFT) I can ping when teamed though. 2) Spanning-tree has disabled ports 37 & 38 on SW2 for connections from 1.2 on both LTMs (not sure if it's suppose to do this or not when LTMs are in Active/Standby) Additional Information: Nothing has changed on the LTMs since the change or prior to. I have not rebooted either LTM, but have cleared ARP

What type of NIC teaming configuration are your servers using? I believe in the HP teaming setup you can see this by selecting the team in the main screen and clicking "Properties". The two I'm interested in are "Team Type Selection" and "Transmit Load Balancing Method". I'm not saying this is necessarily incorrect, but it may contribute depending on the setting.

Please depict your servers on your drawing, and please post the config of the switches the servers are plugged into... Thanks

Cisco's example documentation relating to NIC teaming compatibity with Etherchannel actually uses "802.3ad Dynamic with Fault Tolerance" as the Selection and "Destination IP" as the Method:See Here The document above covers Catalyst switch configuration with HP NIC teaming, but you get the idea... I've not gone through your switch configurations yet, but this is something I've seen in other environments. When I get some extra time I'll scope out those configs.

F5 LTM VIP/STP Problem

25 Replies

Austin_Geraci_3
Cirrus
Apr 01, 2011
Can you post your switch config and your bigip_base.conf file? Where do the web servers plug in here?

Not being able to resolve a hostname doesn't really fit into the picture you sent.. Do you have GTM(s) running anywhere? If it truly is a DNS resolution issue, what's your DNS architecture like? I think you might be using the word "resolve", but really meaning you can't bring up the content you're looking for when hitting the VIP address.. Correct?
jfrizzell_43066
Nimbostratus
Apr 01, 2011
DNS is working like is should, so no complaints here. For example, I cannot go to http://172.23.10.10 (VIP) or https://172.23.10.8 (F5 LTM web management).

No GMT running in our network

Ports 1.1 & 1.1 from both F5 LTM connect to SW1

Ports 1.2 & 1.2 from both F5 LTM connect to SW2

See the attached switch configs

SW1-SW2 Etherchannel are ports 49-50 & F5 LTMs connect to ports 37-38 on both switches

I will work on getting you those config files from the F5s.

Issues:

1) When NICs are teamed with TLB, I cannot access the server VIPs or either LTM by HTTP/HTTPS. (Remove one NIC from the team and it works...also works with NFT) I can ping when teamed though.

2) Spanning-tree has disabled ports 37 & 38 on SW2 for connections from 1.2 on both LTMs (not sure if it's suppose to do this or not when LTMs are in Active/Standby)

Additional Information:

Nothing has changed on the LTMs since the change or prior to. I have not rebooted either LTM, but have cleared ARP
Joel_Moses
Nimbostratus
Apr 01, 2011
What type of NIC teaming configuration are your servers using? I believe in the HP teaming setup you can see this by selecting the team in the main screen and clicking "Properties". The two I'm interested in are "Team Type Selection" and "Transmit Load Balancing Method".

I'm not saying this is necessarily incorrect, but it may contribute depending on the setting.
Austin_Geraci_3
Cirrus
Apr 01, 2011
Please depict your servers on your drawing, and please post the config of the switches the servers are plugged into...

Thanks
Joel_Moses
Nimbostratus
Apr 01, 2011
Cisco's example documentation relating to NIC teaming compatibity with Etherchannel actually uses "802.3ad Dynamic with Fault Tolerance" as the Selection and "Destination IP" as the Method:See Here

The document above covers Catalyst switch configuration with HP NIC teaming, but you get the idea... I've not gone through your switch configurations yet, but this is something I've seen in other environments. When I get some extra time I'll scope out those configs.
jfrizzell_43066
Nimbostratus
Apr 01, 2011
iRuleYou,

I have attached a simple document showing the server connection to the switches. We have one NIC attached to SW1 and the second NIC attached to SW2. For example, the Management Server connects to both switches on Port 12. What is interesting about the NIC teams is that I can access all resources including HTTP/HTTPs for all servers, but VIP or F5 Web management cannot be accessed.

Joel,

Thanks for the link, but we are not using any type of 802.3ad or Dual Channeling. We are using a simple one NIC connection to our redundant switches. I have provided a link to the document about HP NIC Teaming.

ftp://ftp.compaq.com/pub/products/servers/networking/TeamingWP.pdf

Another interesting item to note was this was all functioning properly prior to the switch change over. F5 support thinks that rebooting the LTMs will fix the issue, but I am not sure about that.
jfrizzell_43066
Nimbostratus
Apr 01, 2011
Just received word from F5 support that HP NIC teams that consist of TLB and SLB are not supported by F5 LTM. Also, it looks as if the Cisco switches are doing their job by blocking the ports connected to the second switch. Although it seems a bit weird that this would occur with the LTMs in pass through mode, but I guess the Cisco switch sees it differently. I will team the NICs using Network Fault Tolerance and this should solve my issue with access to the VIPs.

If anyone has any other suggestions or believes otherwise, please let me know. Thank you to all that have helped with this issue thus far.
Hamish
Cirrocumulus
Apr 02, 2011
OK...

Firstly... I'm assuming the etherchannel you're referring to in your original post is the one BETWEEN the pair of switches only. Because 3560's don't do VPC's. (You need 6500 VSS or Nexus for that kind of stuff).

Second... Your F5 interfaces are bridged? That's why your redundant interfaces are blocking... Because you have a bridging loop. So the switches HAVE to block one of them (Because you're passing through the STP or PVST packets - As you need to if you're bridging other packets).

Hmm.. Thinking... You're not trying to do a trunk WITHOUT signalling are you? That won't work. It never does. (Well, OK, it CAN but any failures or mis-cabling generally lead to the problems you describe. So I tend to discourage actively any attempts to do so (i.e. won't support it and tell people to fix their end - You can only support stuff you have faith in. Unless you like to make great hourly rates fixing other peoples mistakes).

TLB & SLB... Hmm... I'm not a great fan of HP's NIC teaming... Most of the options don't actually work very well in a multi-switch environment. Switches are simple beasts. They expect IP's to map to consistent MAC's. And they expect MAC's to be on ONE switchport, and not move around. The only really useful NIC teaming mode is 802.3ad (LACP). It works fine, as long as the load-balancing is set correctly. If it's not, then again, you get the problems you're describing.

In summary.

Could be trunking/etherchannel problem without signalling

Or incorrect load balancing modes and MAC addresses moving between switches.

Or something else... The full config and diagram would need to be provided to give much more info.
Hamish
Cirrocumulus
Apr 02, 2011
Edited... re-read the config. What's E01? And why do you have it hardset to 1000FDX? Cisco won't advertise the speed & duplex when you do that, you might find you're getting half-duplex at the far end with this...

H
Hamish
Cirrocumulus
Apr 02, 2011
Posted By jfrizzell on 04/01/2011 03:54 PM

Just received word from F5 support that HP NIC teams that consist of TLB and SLB are not supported by F5 LTM. Also, it looks as if the Cisco switches are doing their job by blocking the ports connected to the second switch. Although it seems a bit weird that this would occur with the LTMs in pass through mode, but I guess the Cisco switch sees it differently. I will team the NICs using Network Fault Tolerance and this should solve my issue with access to the VIPs.

If anyone has any other suggestions or believes otherwise, please let me know. Thank you to all that have helped with this issue thus far.

Ah... Not sure the F5 will care whether the HP NIC's are doing TLB/SLB or anything else. That's purely between the host and the switches. The F5 just sees an IP address, and (If they're on the same VLAN) a MAC address. Anything else (OK. MTU as well) is irrelevant.

On the port blocking. If you're re-sending traffic down one link fine an other, then the switch is doing it's job BECAUSE you're in pass-though mode. In this mode, the F5 re-sends the BPDU (SPanning Tree) packets back to the switch. So the switch knows that you've looped the traffic back on itself. Therefore it blocks one of the links, because otherwise you get broadcast storms as broadcast packets loop around & around...

I think you just have (Had? If it's now working) a problem with HP NIC teaming (Because it's not really good with multiple switches), and trying to aggregate links without using signalling.

I'm still a bit suspicious about the early statement that you couldn't resolve addresses with the previous config... But you don't mention where your client and DNS servers were... So that may simply be lack of info.

H