Forum Discussion

Tabish_Mirza_12's avatar
Tabish_Mirza_12
Icon for Nimbostratus rankNimbostratus
Mar 12, 2014

unable to ping backend servers through F5 BIG-IP LTM inline mode

We have pair of BIG-IP LTM running 11.4 HF3 with multiple route domain. Route Domain 0 & 1. In route domain 1 we have two VLAN (External & Internal). We have servers connected behind BIG-IP using BIG-IP internal interface floating IP as a gateway & Internal Vlan. We do not have out of band management interface on back-end servers to do management task on it, so we have to use back-end servers communication interface means the interface which is using Internal VLAN IP to do managment task on servers but the problem is BIG-IP is not allowing us to access back-end servers. We do have default route on BIG-IP. We can ping BIG-IP external interface IP(self & floating) but we can not ping BIG-IP internal interface IP (self & floating). below is the tcpdump output.

 

15:27:07.989673 IP 10.10.94.200 > 10.11.27.200: ICMP echo request, id 3319, seq 0, length 80 15:27:08.054649 01:00:0c:cc:cc:cd (oui Unknown) > 00:19:30:4a:f0:6d (oui Unknown) SNAP Unnumbered, ui, Flags [Command], length 50 0x0000: 0100 0ccc cccd 0019 304a f06d 8100 e000 ........0J.m.... 0x0010: 0032 aaaa 0300 000c 010b 0000 0000 0081 .2.............. 0x0020: dc00 0a8b 2528 0000 0000 0381 dc00 0af3 ....%(.......... 0x0030: 9045 4080 0601 .E@... 15:27:08.421438 arp who-has 10.11.27.201 tell 10.11.27.100 15:27:08.421522 arp who-has 10.11.27.201 tell 10.11.27.100 15:27:08.443560 IP 10.11.31.200.hsrp > all-routers.mcast.net.hsrp: HSRPv0-hello 20: state=active group=253 addr=10.11.31.202 15:27:08.443629 IP 10.11.31.200.hsrp > all-routers.mcast.net.hsrp: HSRPv0-hello 20: state=active group=253 addr=10.11.31.202 15:27:08.566989 IP 10.11.25.201.hsrp > all-routers.mcast.net.hsrp: HSRPv0-unknown (3) 16: state=initial group=2 [|hsrp] 15:27:08.725659 01:00:0c:cc:cc:cd (oui Unknown) > 00:19:30:4a:f0:6d (oui Unknown) SNAP Unnumbered, ui, Flags [Command], length 50 0x0000: 0100 0ccc cccd 0019 304a f06d 8100 e000 ........0J.m.... 0x0010: 0032 aaaa 0300 000c 010b 0000 0000 0081 .2.............. 0x0020: d900 0a8b 2528 0000 0000 0381 d900 0af3 ....%(.......... 0x0030: 9045 4080 0601 .E@... 15:27:08.727743 01:00:0c:cc:cc:cd (oui Unknown) > 00:19:30:4a:f0:6d (oui Unknown) SNAP Unnumbered, ui, Flags [Command], length 50 0x0000: 0100 0ccc cccd 0019 304a f06d 8100 e000 ........0J.m.... 0x0010: 0032 aaaa 0300 000c 010b 0000 0000 0081 .2.............. 0x0020: d400 0a8b 2528 0000 0000 0381 d400 0af3 ....%(.......... 0x0030: 9045 4080 0601 .E@... 15:27:08.731838 01:00:0c:cc:cc:cd (oui Unknown) > 00:19:30:4a:f0:6d (oui Unknown) SNAP Unnumbered, ui, Flags [Command], length 50 0x0000: 0100 0ccc cccd 0019 304a f06d 8100 e1d3 ........0J.m.... 0x0010: 0032 aaaa 0300 000c 010b 0000 0000 0081 .2.............. 0x0020: d300 0a8b 2528 0000 0000 0381 d300 0af3 ....%(.......... 0x0030: 9045 4080 0601 .E@... 15:27:08.901896 01:00:0c:cc:cc:cd (oui Unknown) > 00:19:30:4a:f0:6d (oui Unknown) SNAP Unnumbered, ui, Flags [Command], length 50 0x0000: 0100 0ccc cccd 0019 304a f06d 8100 e000 ........0J.m.... 0x0010: 0032 aaaa 0300 000c 010b 0000 0000 0081 .2.............. 0x0020: da00 0a8b 2528 0000 0000 0381 da00 0af3 ....%(.......... 0x0030: 9045 4080 0601 .E@... 15:27:08.903064 IP 10.11.25.201.hsrp > all-routers.mcast.net.hsrp: HSRPv0-hello 20: state=standby group=249 addr=10.11.25.202 15:27:08.904082 01:00:0c:cc:cc:cd (oui Unknown) > 00:19:30:4a:f0:6d (oui Unknown) SNAP Unnumbered, ui, Flags [Command], length 50 0x0000: 0100 0ccc cccd 0019 304a f06d 8100 e000 ........0J.m.... 0x0010: 0032 aaaa 0300 000c 010b 0000 0000 0081 .2.............. 0x0020: db00 0a8b 2528 0000 0000 0381 db00 0af3 ....%(.......... 0x0030: 9045 4080 0601 .E@... 15:27:08.904256 IP 10.11.29.249.23735 > 10.43.5.2.8005: S 2700772237:2700772237(0) win 14600 15:27:08.904340 IP 10.11.29.249 > 10.43.5.1: ICMP echo request, id 8787, seq 11547, length 20

 

APM
application delivery
deployment
LTM
security

10 Replies

Sort By
  • Moe_Jartin's avatar
    Moe_Jartin
    Icon for Cirrus rankCirrus
    Mar 12, 2014

    Have you configured an IP Forwarding virtual server for the subnet that the servers are on? LTM is a default deny box. It is not going to forward traffic for which it does not have a "listener".

     

    • Tabish_Mirza_12's avatar
      Tabish_Mirza_12
      Icon for Nimbostratus rankNimbostratus
      Mar 12, 2014
      Hi Martin, Thanks for your response indeed. Yes I created two Wildcard IP Forwarding Virtual Server. One for default route domain means 0 & one for route domain 1 with %1 notation. But no luck. Do you have any idea that why I am getting below while running tcpdump on Interface 1.3 which is tag on External VLAN ? 15:27:08.054649 01:00:0c:cc:cc:cd (oui Unknown) > 00:19:30:4a:f0:6d (oui Unknown) SNAP Unnumbered, ui, Flags [Command], length 50 0x0000: 0100 0ccc cccd 0019 304a f06d 8100 e000 ........0J.m.... 0x0010: 0032 aaaa 0300 000c 010b 0000 0000 0081 .2.............. 0x0020: dc00 0a8b 2528 0000 0000 0381 dc00 0af3 ....%(.......... 0x0030: 9045 4080 0601 .E@... Thanks Tabish Moreover, Le
  • Moe_Jartin's avatar
    Moe_Jartin
    Icon for Cirrus rankCirrus
    Mar 12, 2014

    That looks like some Cisco switches running STP. That 00:0c:cc:cc:cd is the L2 multicast address for spanning tree and the 00:19:30:4a:f0:6d is definitely a Cisco MAC. I would ignore those. Not part of the problem.

     

    Can you ping both the client IP address and the server IP address in question from the BIG IP device? Just want to make sure you have connectivity to hosts before going any farther.

     

    Also, is there a route for the server subnet on the upstream router pointing to the F5 external floating IP address?

     

    Joe

     

    • Tabish_Mirza_12's avatar
      Tabish_Mirza_12
      Icon for Nimbostratus rankNimbostratus
      Mar 12, 2014
      Hi Joe, Yes we do have the reverse route configured for the subnet which is running behind BIG-IP 10.11.27.0/24 in route domain 1 on our upstream Cisco Switch 6509 running HSRP. ip route 10.11.27.0 255.255.255.0 10.11.29.251 (10.11.29.251 is the floating IP of BIG-IP external internal means 1.3 in route-domain 1) Yes I can ping both the back-end servers which is 10.11.27.100 & the client IP which is 10.10.94.200 from the BIG-IP from the route domain 1 by the following command. [root@CIODCGDNF5LTM01:Active:In Sync] config rdsh 1 [root@CIODCGDNF5LTM01:Active:In Sync] config ping 10.11.27.100 PING 10.11.27.100 (10.11.27.100) 56(84) bytes of data. 64 bytes from 10.11.27.100: icmp_seq=1 ttl=64 time=3.07 ms 64 bytes from 10.11.27.100: icmp_seq=2 ttl=64 time=0.958 ms 64 bytes from 10.11.27.100: icmp_seq=3 ttl=64 time=0.841 ms ^C --- 10.11.27.100 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.841/1.626/3.079/1.028 ms Following is the interface details of route-domain 1 [root@CIODCGDNF5LTM01:Active:In Sync] config rdsh 1 [root@CIODCGDNF5LTM01:Active:In Sync] config ifconfig External-VIP Link encap:Ethernet HWaddr 00:23:E9:68:73:09 inet addr:10.11.29.249 Bcast:10.11.29.255 Mask:255.255.255.0 inet6 addr: fe80::223:e9ff:fe68:7309/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:33796 errors:0 dropped:0 overruns:0 frame:0 TX packets:7060 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:3561469 (3.3 MiB) TX bytes:436990 (426.7 KiB) STG-Web-IN Link encap:Ethernet HWaddr 00:23:E9:68:73:04 inet addr:10.11.27.200 Bcast:10.11.27.255 Mask:255.255.255.0 inet6 addr: fe80::223:e9ff:fe68:7304/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2133 errors:0 dropped:0 overruns:0 frame:0 TX packets:2 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:142462 (139.1 KiB) TX bytes:180 (180.0 b) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.255.255.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:504 (504.0 b) TX bytes:504 (504.0 b) tmm0 Link encap:Ethernet HWaddr 00:98:76:54:32:10 inet addr:127.1.1.1 Bcast:127.1.1.255 Mask:255.255.255.0 inet6 addr: fe80::298:76ff:fe54:3210/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Thanks Tabish
    • Tabish_Mirza_12's avatar
      Tabish_Mirza_12
      Icon for Nimbostratus rankNimbostratus
      Mar 12, 2014
      One more thing I just want to tell you when I do tcpdump on route-domain 1 with the interface number or vlan ID. I do not see any packet but when I do tcpdump on default route domain with the interface number . I can see packets. Below is the output. Route Domain 1 [root@CIODCGDNF5LTM01:Active:In Sync] config rdsh 1 [root@CIODCGDNF5LTM01:Active:In Sync] config tcpdump -i External-VIP tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on External-VIP, link-type EN10MB (Ethernet), capture size 96 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel [root@CIODCGDNF5LTM01:Active:In Sync] config tcpdump -i 1.3 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on 1.3, link-type EN10MB (Ethernet), capture size 96 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel
  • Moe_Jartin's avatar
    Moe_Jartin
    Icon for Cirrus rankCirrus
    Mar 12, 2014

    What version are you running? which route domain/partiton/traffci group were those VLANs and Sefl IPs created in?

     

    Joe

     

    • Tabish_Mirza_12's avatar
      Tabish_Mirza_12
      Icon for Nimbostratus rankNimbostratus
      Mar 12, 2014
      We are running 11.4.1 HF2. Common partition running both route domains (default route-domain 0 & non default route-domain1). VLANs and Self IPs (External-VIP & STG-Web-IN) created in route domain 1. We have two default route (one for route-domain 0 & one for route domain 1). Thanks
  • Moe_Jartin's avatar
    Moe_Jartin
    Icon for Cirrus rankCirrus
    Mar 12, 2014

    You said, "when I do tcpdump on route-domain 1 with the interface number or vlan ID. I do not see any packet but when I do tcpdump on default route domain with the interface number . I can see packets."

     

    So are both VLANs coming the the same physical interface? if so, do you have them set as tagged or untagged in the VLAN config?

     

    Also, has this ever worked? or is this new install that has never worked yet?

     

    • Tabish_Mirza_12's avatar
      Tabish_Mirza_12
      Icon for Nimbostratus rankNimbostratus
      Mar 12, 2014
      Both VLANs running on different interfaces (Interface 1.3 running VLAN External-VIP subnet 10.11.29.0/24 & Interface 1.2 running VLAN STG-Web-IN subnet 10.11.27.0/24(Back-end servers using this subnet)). I have tagged VLAN External-VIP on 1.3 & VLAN STG-Web-IN tagged on 1.2 interface. The reason I used tagged because we do have others vlan running on these interfaces. Thanks
  • Moe_Jartin's avatar
    Moe_Jartin
    Icon for Cirrus rankCirrus
    Mar 12, 2014

    hmmm??? you can ping each of the hosts in question from the BIGIP from RD1; you have forwarding virtual configured in RD 1 associated with all/both VLANs; you have routes to/thru the BIGIP in both directions. I am stumped without seeing more details.

     

    what protocol is on the forwarding virtual? TCP? UDP? All protocols? should be all protocols.

     

    The fact that you can ping the devices in question from the BIGIP from RD1 leads me to believe the RD/VLAN/Self IP config is correct. I am guessing it has to do with the forwarding to/through the BIGIP from the next hop devices in one or both directions. OR something to do with the forwarding virtual config.

     

    Joe

     

    Joe