Forum Discussion

Jonathan_Perrig's avatar
Jonathan_Perrig
Icon for Nimbostratus rankNimbostratus
Apr 28, 2014

Trouble with pool member server recursively calling a web reference through the Big IP

I am planning to contact support on this as well but thought I would see if anyone has run into this before. I am still very green to BigIP. My development group is in the process of deploying a set of .Net web services. I am setting up a basic LTM pool and VIP in Big IP. This particular application is a little different than the previous (legacy) ones I've set up as this application has web reference calls to itself, meaning that in order for it to complete it's transaction from the client, it will make sub-calls to other web services that are all reached by using the same VIP on the BigIP. Now to my problem...these sub-calls are failing through the BigIP, however they work fine when called directly, such as by using a local host file entry to avoid going back through the VIP.

The BigIP is set up in a on-arm configuration where it does not serve as a gateway between the client and the pool member servers. I routinely set Source Address Translation to auto-map to avoid asymmetric routing. However, that setting did not work in this case. Researching the site, I tried creating a SNAT pool with the VIP as the only address in it and assigning that pool for Source Address Translation. Interestingly, this allows one of the two test servers I am working with to now work, but the other still does not. This is consistent. The same server always works, the same one always fails.

I ran a TCPDump to capture traffic from the server that is failing (below, some formatted was included to enhance readability): .169 is the VIP, .170 is the member server. It appears that the server (.170) is not sending ACKs back to the BigIP.

I'm at a loss as to where to troubleshoot from here. I'm not understanding why AutoMap shouldn't work (the Self-IP shares the same network as the member servers) and why implementing an explicit SNAT pool provides partial functionality.

Any help or possible explanation is appreciated.

Thank you,

Jonathan

Record  Time        Source IP/Port      Dest IP/Port    TCP Flag    Packet Sequence ack ack number      source tcp window       
1   14:01:18    IP  10.60.110.170.netboot-pxe   >   10.60.111.169.http: S   4284183473:4284183473(0)            win 65535   "
2   14:01:18    IP  10.60.111.169.http  >   10.60.110.170.netboot-pxe:  S   66770417:66770417(0)    ack 4284183474  win 4380    "
3   14:01:21    IP  10.60.111.169.http  >   10.60.110.170.netboot-pxe:  S   66770417:66770417(0)    ack 4284183474  win 4380    "
4   14:01:21    IP  10.60.110.170.netboot-pxe   >   10.60.111.169.http: S   4284183473:4284183473(0)            win 65535   "
5   14:01:27    IP  10.60.111.169.http  >   10.60.110.170.netboot-pxe:  S   66770417:66770417(0)    ack 4284183474  win 4380    "
6   14:01:27    IP  10.60.110.170.netboot-pxe   >   10.60.111.169.http: S   4284183473:4284183473(0)            win 65535   "
7   14:01:39    IP  10.60.110.170.sunfm-port    >   10.60.111.169.http: S   3111077506:3111077506(0)            win 65535   "
8   14:01:39    IP  10.60.111.169.http  >   10.60.110.170.sunfm-port:   S   1974346658:1974346658(0)    ack 3111077507  win 4380    "
9   14:01:39    IP  10.60.111.169.http  >   10.60.110.170.netboot-pxe:  S   66770417:66770417(0)    ack 4284183474  win 4380    "
10  14:01:42    IP  10.60.111.169.http  >   10.60.110.170.sunfm-port:   S   1974346658:1974346658(0)    ack 3111077507  win 4380    "
11  14:01:42    IP  10.60.110.170.sunfm-port    >   10.60.111.169.http: S   3111077506:3111077506(0)            win 65535   "
12  14:01:48    IP  10.60.111.169.http  >   10.60.110.170.sunfm-port:   S   1974346658:1974346658(0)    ack 3111077507  win 4380    "
13  14:01:48    IP  10.60.110.170.sunfm-port    >   10.60.111.169.http: S   3111077506:3111077506(0)            win 65535   "
14  14:02:00    IP  10.60.111.169.http  >   10.60.110.170.sunfm-port:   S   1974346658:1974346658(0)    ack 3111077507  win 4380    "
15  14:02:15    IP  10.60.110.170.sops  >   10.60.111.169.http: S   2951809440:2951809440(0)            win 65535   "
16  14:02:15    IP  10.60.111.169.http  >   10.60.110.170.sops: S   174329152:174329152(0)  ack 2951809441  win 4380    "
17  14:02:18    IP  10.60.110.170.sops  >   10.60.111.169.http: S   2951809440:2951809440(0)            win 65535   "
18  14:02:18    IP  10.60.111.169.http  >   10.60.110.170.sops: S   174329152:174329152(0)  ack 2951809441  win 4380    "
19  14:02:24    IP  10.60.110.170.sops  >   10.60.111.169.http: S   2951809440:2951809440(0)            win 65535   "
20  14:02:24    IP  10.60.111.169.http  >   10.60.110.170.sops: S   174329152:174329152(0)  ack 2951809441  win 4380    "
application delivery
availability
deployment
LTM

2 Replies

Sort By
  • nitass's avatar
    nitass
    Icon for Employee rankEmployee
    Apr 28, 2014

    I ran a TCPDump to capture traffic from the server that is failing (below, some formatted was included to enhance readability): .169 is the VIP, .170 is the member server. It appears that the server (.170) is not sending ACKs back to the BigIP.

     

    is arp entry correct on bigip and server?

     

    is it possible to capture packet on server (i.e. running tcpdump on server)?

     

  • Cory_50405's avatar
    Cory_50405
    Icon for Noctilucent rankNoctilucent
    Apr 28, 2014

    The SYN packet is making it from the server to the BIG-IP, so it makes sense that the ACK from the server to the BIG-IP would too, if the server were actually receiving the SYN ACK. The SYN ACK from BIG-IP to your server appears to be being lost, hence you are not seeing the final ACK of the three-way handshake.

     

    As Nitass asked, check your ARP table on the BIG-IP and ensure you are seeing what you expect to (arp -a).