Forum Discussion

Bhargav_9588's avatar
Bhargav_9588
Icon for Nimbostratus rankNimbostratus
Oct 24, 2008

handshake_failure

Hello,

 

 

After upgrading LTM to 9.3.1 we started noticing "handshake_failure" errors with java clients that are trying to connect to https://www.***.com/ (virtual server in LTM with Client SSL Profile and Client Certificate set to "request").

 

 

It use to work with out any issues before upgrade with same settings. After upgrade, when java client tries to connect to same webservice with https getting following error:

 

 

main, WRITE: TLSv1 Handshake, length = 32

 

main, received EOFException: error

 

main, handling exception: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake

 

main, SEND TLSv1 ALERT: fatal, description = handshake_failure

 

main, WRITE: TLSv1 Alert, length = 18

 

main, called closeSocket()

 

Exception :javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake

 

Authorization: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake

 

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:801)

 

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1089)

 

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1116)

 

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1100)

 

at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:402)

 

at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:170)

 

at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:857)

 

at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:230)

 

at gateway.login(gateway.java:64)

 

at createCollection.main(createCollection.java:31)

 

Caused by: java.io.EOFException: SSL peer shut down incorrectly

 

at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)

 

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:782)

 

 

 

 

Any help is appreciated and thanks in advance.

 

 

--Thanks

7 Replies

  • It looks like an SSL handshake error, but the client program isn't giving details on the actual SSL error. It could be a cipher issue. Can you use openssl s_client to test? That should give you more debug information. Else, you could use ssldump on the BIG-IP to capture a trace of the problem.

     

     

    You can use tcpdump to capture the encrypted traffic failure:

     

     

    tcpdump -i0.0 -s0 -w/var/tmp/encrypted.dmp host 1.1.1.1

     

     

    -i 0.0 captures on all switch interfaces

     

    -w writes the output to a binary file

     

    host 1.1.1.1 filters on traffic to/from 1.1.1.1

     

     

    Make sure to start the tcpdump before the client establishes a connection to the VIP. SOL411 (Click here) has more detail on using tcpdump.

     

     

    You can then use ssldump to decrypt the tcpdump file.

     

     

     

    https://support.f5.com/kb/en-us/solutions/public/7000/800/sol7823.html

     

     

    ssldump -AdneN -r/var/tmp/encrypted.dmp -k/path/to/server.key >/var/tmp/decrypted.txt

     

     

     

     

    Where /var/tmp/encrypted.dmp is the tcpdump file, /path/to/server.key is the path to the server's (VIP) SSL key, and /var/tmp/decrypted.txt is the cleartext output file. You can run ssldump on the BIG-IP or a windows host. For details on the command, check the man page: man ssldump.

     

     

    Also, there is a forum for Advanced Configuration (Click here) where you might get more relevant eyes on your post.

     

     

    Aaron
  • Thanks Aaron!

     

     

    I have tried s_client and I see the following (handshake failure):

     

     

    openssl s_client -connect www.xyz.com:443 -CAfile ../../IssuingCA.cer

     

    CONNECTED(00000004)

     

    depth=1 /emailAddress=email@xyz.com/C=**/ST=**/L=***/O=Corp./OU=Corporate/CN=Corp. CA

     

    verify error:num=2:unable to get issuer certificate

     

    issuer= /emailAddress=cisinfrastructure@aes.com/C=**/ST=**/L=***/O=The XYZ Corp./OU=Corporate/CN=Root CA

     

    verify return:0

     

    1618:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:226:

     

     

    It works fine with no issues from browser. We started seeing issues with java clients after upgrading LTM.

     

     

    I have not made success yet to decrypt tcpdump file using ssldump.

     

     

    Thanks in advance.
  • Just for information....Our CA is not a public CA and we use Windows Certificate Services
  • Hi bhargav,

     

     

    I'd suggest using ssldump to decrypt the trace. It should give you a more specific error.

     

     

    Why do you have the client SSL profile set to request a client cert? Is the application expecting the client to send a client cert? If so, how is it getting the details of the client cert if you're decrypting the traffic on LTM?

     

     

    You should be able to find more info on the error strings through any search engine as they're standard openssl error messages. For details on the SSL profile options, you can check the LTM config guide for your version. Here is a link to the latest 9.4.x guide (Click here)

     

     

    Aaron
  • We use SSL Client authentication and certificate is being stored in an http header using the following iRule:

     

     

    when CLIENTSSL_CLIENTCERT {

     

    set ssl_stuff [list cert cert_status]

     

    set ssl_cert_status [X509::verify_cert_error_string [SSL::verify_result]]

     

    lset ssl_stuff 0 [SSL::cert 0]

     

    lset ssl_stuff 1 $ssl_cert_status

     

    session add ssl [SSL::sessionid] $ssl_stuff 3600

     

    }

     

    when HTTP_REQUEST {

     

    set ssl_stuff [session lookup ssl [SSL::sessionid]]

     

    set the_cert [lindex $ssl_stuff 0]

     

    set ssl_cert_status [lindex $ssl_stuff 1]

     

    if { $ssl_cert_status eq "ok"}

     

    {

     

    if { $the_cert != ""}

     

    {

     

    HTTP::header insert SSLClientCert [ join [string trim [string map { "-----BEGIN CERTIFICATE-----" "" "-----END CERTIFICATE-----" ""} [X509::whole $the_cert ] ] ] "" ]

     

    }

     

    }

     

    }

     

     

    Thanks,

     

    Bhargav
  • Here is the decrypted string with ssldump:

     

     

    New TCP connection 1: 10.1.129.4(57430) <-> 10.1.4.22(443)

     

    1 1 1224858753.8851 (0.2500) C>S SSLv2 compatible client hello

     

    Version 3.1

     

    cipher suites

     

    TLS_RSA_WITH_RC4_128_MD5

     

    SSL2_CK_RC4

     

    TLS_RSA_WITH_RC4_128_SHA

     

    Unknown value 0x2f

     

    Unknown value 0x35

     

    Unknown value 0x33

     

    Unknown value 0x39

     

    Unknown value 0x32

     

    Unknown value 0x38

     

    TLS_RSA_WITH_3DES_EDE_CBC_SHA

     

    SSL2_CK_3DES

     

    TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA

     

    TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA

     

    TLS_RSA_WITH_DES_CBC_SHA

     

    SSL2_CK_DES

     

    TLS_DHE_RSA_WITH_DES_CBC_SHA

     

    TLS_DHE_DSS_WITH_DES_CBC_SHA

     

    TLS_RSA_EXPORT_WITH_RC4_40_MD5

     

    SSL2_CK_RC4_EXPORT40

     

    TLS_RSA_EXPORT_WITH_DES40_CBC_SHA

     

    TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA

     

    TLS_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA

     

    1 2 1224858753.8905 (0.0054) S>CV3.1(74) Handshake

     

    ServerHello

     

    Version 3.1

     

    random[32]=

     

    f1 62 06 51 60 fe 2b b9 b3 bc 01 e0 40 48 0c c4

     

    bb 8e 97 4f 77 c6 27 01 f2 d1 90 fc 8a 85 16 e4

     

    session_id[32]=

     

    60 4c f0 c7 96 0a 2e d0 85 17 36 65 d4 99 93 ca

     

    18 3b e4 1a 6e 47 d2 93 93 d9 03 b7 77 50 49 57

     

    cipherSuite TLS_RSA_WITH_RC4_128_MD5

     

    compressionMethod NULL

     

    1 3 1224858753.8910 (0.0004) S>CV3.1(1730) Handshake

     

    Certificate

     

    Subject

     

    C=US

     

    ST=***

     

    L=asb

     

    O=Corporation

     

    OU=Corporate IT

     

    CN=www.xyz.com

     

    Issuer

     

    emailAddress=abc@xyz.com

     

    C=US

     

    ST=***

     

    L=***

     

    O=Corp.

     

    OU=Corporate IT

     

    CN=Corp. Issuing CA

     

    Serial 68 9f 1e a4 00 00 00 00 00 5c

     

    Extensions

     

    Extension: X509v3 Subject Key Identifier

     

    Extension: X509v3 Authority Key Identifier

     

    Extension: X509v3 CRL Distribution Points

     

    Extension: Authority Information Access

     

    Extension: X509v3 Basic Constraints

     

    Critical

     

    Extension: X509v3 Key Usage

     

    Extension: X509v3 Extended Key Usage

     

    Extension: 1.3.6.1.4.1.311.20.2

     

    1 4 1224858753.8910 (0.0000) S>CV3.1(170) Handshake

     

    CertificateRequest

     

    certificate_types rsa_sign

     

    certificate_authority

     

    emailAddress=abc@xyz.com

     

    C=US

     

    ST=***

     

    L=***

     

    O=Corp.

     

    OU=Corporate IT

     

    CN=Root CA

     

    1 5 1224858753.8910 (0.0000) S>CV3.1(4) Handshake

     

    ServerHelloDone

     

    1 6 1224858753.9199 (0.0288) C>SV3.1(269) Handshake

     

    Certificate

     

    ClientKeyExchange

     

    EncryptedPreMasterSecret[256]=

     

    45 be b7 09 1b 7b bb 1a 80 e7 cc cd b2 5f 28 0e

     

    cd e0 c9 40 44 33 34 c0 a0 a7 87 82 b6 14 c7 f9

     

    3a 74 28 ce 0f de 13 ce ad 43 10 b9 15 07 2d 1e

     

    68 ed 11 0b 90 66 ad b4 05 a8 1c 15 49 7f d9 c5

     

    1b 00 a2 36 69 76 95 bb 5d 90 da 75 10 dc a8 87

     

    20 1e 6c 33 83 be f7 25 b2 e1 4b fa 5e 05 d9 a5

     

    b2 10 af 2b f2 e3 fe 64 da 3e 39 15 92 65 ba d6

     

    40 19 87 96 77 8d b7 65 11 84 95 a3 48 f6 8c 5b

     

    c2 18 9a 11 8b d7 41 79 75 b3 21 3c 5b b3 51 72

     

    8f 04 c3 98 15 e9 19 db 15 16 01 73 e1 cf ec de

     

    2a 5a 23 9a a9 ca 10 23 4f 74 ae a6 bc b7 0b ee

     

    0b 9c a3 d6 7e fd 86 38 04 8e 58 96 66 82 25 a7

     

    55 b0 46 43 81 17 4c f0 c6 17 08 c1 b0 d4 3c 74

     

    43 ab d3 ea ef cb 4d 13 ee 5b 9f 7f c4 83 4c 0e

     

    2a c6 e2 e6 1a 20 54 29 e7 82 d5 08 75 12 44 1b

     

    6d b7 0b 2b c3 76 b1 1e 88 53 2f 37 2a 6b f8 f3

     

    1 7 1224858753.9216 (0.0017) C>SV3.1(1) ChangeCipherSpec

     

    1 8 1224858753.9235 (0.0018) C>SV3.1(32) Handshake

     

    1 1224858753.9374 (0.0139) S>C TCP FIN

     

    1 9 1224858753.9386 (0.0012) C>SV3.1(18) Alert

     

    1 1224858753.9387 (0.0001) C>S TCP FIN
  • Steve_Scott_873's avatar
    Steve_Scott_873
    Historic F5 Account
    Could i suggest wireshark as an easier to use alternative to SSLDump.

     

     

    Also, this is invaluable: http://en.wikipedia.org/wiki/Secure_Sockets_Layer (Don't laugh, Wiki is pretty good for this sort of info and a lot easier to read than an RFC!)