Forum Discussion

kash_49328's avatar
kash_49328
Icon for Nimbostratus rankNimbostratus
Dec 16, 2014

Big IP LTM sending tcp Resets due to SSL handshake time out ?

Hi F5 gurus, We have a https file transfer going on a daily basis and we are experiencing a big problem here. Client is a java program and server is behind the F5. We are offloading ssl on F5 so we use client ssl profile with default settings ( Version 11.2.1 LTM, ssl handshake time out = 10 sec ) . Tcpdump is saying that RST are generated from F5. As per F5 Handshake time more than 10sec will make system vulnerable to DoS attack. Also client route through many network devices before hitting the G5 big ip. I have enabled the rstcause.log and rstcause.pkt which gives me below logs.

 

ltm 12-15 17:00:13 err lb1 tmm1[9547]: RST sent from server ip :443 to client ip :14720, [0x147b9e1:962] SSL handshake timeout exceeded

 

ltm 12-15 17:00:13 err lb1 tmm1[9547]: RST sent from server ip :443 to client ip :14716, [0x147b9e1:962] SSL handshake timeout exceeded

 

ltm 12-15 17:00:13 err lb1 tmm1[9547]: RST sent from server ip :443 to client ip :14718, [0x147b9e1:962] SSL handshake timeout exceeded

 

ltm 12-15 17:00:13 err lb1 tmm[9546]: RST sent from server ip :443 to client ip :14719, [0x147b9e1:962] SSL handshake timeout exceeded

 

ltm 12-15 17:00:14 err lb1 tmm1[9547]: RST sent from server ip :443 to client ip :14716, [0x147db9a:4315] TCP 3WHS rejected

 

ltm 12-15 17:00:14 err lb1 tmm1[9547]: RST sent from server ip :443 to client ip :14718, [0x147db9a:4315] TCP 3WHS rejected

 

ltm 12-15 17:00:14 err lb1 tmm[9546]: RST sent from server ip :443 to client ip :14719, [0x147db9a:4315] TCP 3WHS rejected

 

Please help me if you have seen this problem before.

 

Thanks in advance

 

application delivery
cisco
LTM
management
performance
TMSH

10 Replies

Sort By
  • nitass's avatar
    nitass
    Icon for Employee rankEmployee
    Dec 16, 2014

    isn't 10 seconds for ssl handshake enough? it is only handshake (not data transfer), isn't it?

     

  • afedden_1985's avatar
    afedden_1985
    Icon for Cirrus rankCirrus
    Dec 16, 2014

    You might try using ssldump to capture the connection between the F5 and the Server. It might help find root cause. https://support.f5.com/kb/en-us/solutions/public/10000/200/sol10209.html

     

    • kash_49328's avatar
      kash_49328
      Icon for Nimbostratus rankNimbostratus
      Dec 17, 2014
      Yeah i did that but its not actually giving me any specific reason why BIG ip is sending RST. 2 4.7914 (0.0000) S>C TCP FIN 2 258 4.8733 (0.0818) C>S Alert 2 4.8733 (0.0000) C>S TCP FIN 6 5 9.3710 (9.2912) C>S Handshake ClientKeyExchange 6 6 9.3710 (0.0000) C>S ChangeCipherSpec 6 7 9.3710 (0.0000) C>S Handshake 6 8 9.3757 (0.0046) S>C ChangeCipherSpec 6 9 9.3757 (0.0000) S>C Handshake 6 10 9.4564 (0.0806) C>S application_data 6 11 9.9540 (0.4976) S>C application_data 6 12 9.9546 (0.0005) S>C application_data 6 13 10.0347 (0.0801) S>C application_data 1 10.0793 (9.9991) S>C TCP RST 3 10.0793 (9.9993) S>C TCP RST 5 10.0799 (10.0001) S>C TCP RST 4 10.0800 (10.0000) S>C TCP RST
  • afedden_1985's avatar
    afedden_1985
    Icon for Cirrus rankCirrus
    Dec 17, 2014

    Your last update didn't get added to the conversation? Are the resets happening all the time or are some flows working? Is it possible the servers SSL ciphers were updated? With all the SSL issues lately a lot in our shop has been changing to mitigate SSL issues. here are a few commands to check ssl cipher compatibility, you can try this on each of the servers in the pool to see if one of them causing a problem. This curl will test a vip or server to see if it allows sslv3 just use the correct address curl -k -v -3 https://IP.addr/test to test for tls support replace the -3 in the above command with –tls1.2 ex: * curl -k -v -3 https://10.10.34.141/test * and each of the tls versions

     

    * curl -k -v -tlsv1.0 https://10.50.2.48/test * curl -k -v -tlsv1.1 https://10.50.2.48/test * curl -k -v -tlsv1.2 https://10.50.2.48/test

     

    • kash_49328's avatar
      kash_49328
      Icon for Nimbostratus rankNimbostratus
      Dec 17, 2014
      We are doing SSL offloading at F5 LB. Both client and F5 use TLSv1 for SSL and communication between F5 and actual server is http.
  • Mike_Sullivan_2's avatar
    Mike_Sullivan_2
    Icon for Nimbostratus rankNimbostratus
    Dec 17, 2014

    You mentioned ssl offloading. That implies you are not re-encrypting on the backend (If you are encrypting the backend connection, you'll need a server side ssl profile). For offloading, your pool members ought to be setup for port 80 and the VIP needs to have Address Translation and Port Translation enabled.

     

    If the pool members are not using the LTM as the default gateway, enable Source Address Translation Auto Map. Get a dump on the backend to see what's happening there.

     

    Cheers, Mike

     

    • kash_49328's avatar
      kash_49328
      Icon for Nimbostratus rankNimbostratus
      Dec 17, 2014
      You are right. No backend encryption, client profile is applied. Address translation, port translation and Automap is already enabled. I am pretty sure its F5 not the server because communication starts at 1700 GMT and first RST i see in the F5 logs ( after enabling the rstcause.log/rstcause.pkt ) is from VIP to client. I do see server's private IP sending RSTs but first its after VIP RSTs. thanks for your reply
    • kash_49328's avatar
      kash_49328
      Icon for Nimbostratus rankNimbostratus
      Dec 17, 2014
      Also this VIP works well with other clients. This is only for particular one client.
  • Mike_Sullivan_2's avatar
    Mike_Sullivan_2
    Icon for Nimbostratus rankNimbostratus
    Dec 19, 2014

    What message did the rstcause.log have? SOL13223 has the breakdown on the messages. The fact that it works for all but one client makes my think about local firewalls on the members or even on the client.

     

    Cheers, Mike