Servers using LTM as its gateway can ping but can't connect on 443
I have a situation where a group of servers that use LTM as their gateway can successfully ping and tracert to servers in a totally different subnet, but are unable to connect on port 443.
Specifically, servers in 10.54.13.x use gateway IP address 10.54.13.1 which is an LTM traffic group address. These servers are part of a load balanced group, no SNAT is in use for the virtual server, and this all works fine. A tracert from server 10.54.13.101 to 172.22.0.100 completes in 6 hops. Ping succeeds. I cannot, however, telnet to 172.22.0.100:443, or open https://172.22.0.100 from the 10.54.13.101 server (or any server in that subnet).
The successful tracert from server 10.54.13.101 to 172.22.0.100 shows the first hop as the LTM self-IP address 10.54.13.21. The second hop is 10.54.12.1 which is a core router.
If I do the same tracert from the LTM to 172.22.0.100, it succeeds in 1 hop. Also from the LTM, I can connect to https://172.22.0.100 without problem. So I do not believe this is a firewall issue. Because pings and tracerts are OK, I tend to think the routing is all correct.
https connections to 172.22.0.100 from anywhere else on the network all work fine. Only from these servers using LTM as their gateway have this problem.
I do not believe it's a problem with the 172.22.0.100 host because I have the same exact symptoms with other servers in the 172.22.0.x subnet.
On LTM, I do have the "RoutedForwarding_IP_Forwarding" iapp installed which performs IP forwarding on all sources, destinations, and VLANs (except the sync VLAN)
So, somehow the LTM is blocking this 443 connection from the server to the other server at 172.22.0.100. Why would tracerts and pings succeed, but https fail? Does the LTM allow ICMP by default somehow? Short of opening a case with F5 support, I thought I'd post here and see if anyone had any ideas about the cause, or what steps I can take to troubleshoot further? Probably a packet capture is needed here.
thanks!
I keep thinking asymmetric routing... but your ports match up. a tcpdump on the F5 may give you more info where you could see all the MAC addresses.