TCP-Out-Of-Order Packets on new sha 2 certificates

i really doubt this has anything to do with the SHA2 certificate. where are you capturing?

I'm capturing using the tcpdump with the client, VIP and node IP

tcpdump on the big-ip? no irules or any traffic redirection in place? what kind of virtual server? standard? all browsers?

could you do an

openssl s_client -connect ip:port

Yes tcpdump on big-ip and no irules with default pool. 
Virtual server is standard and i tested with IE and chrome.
For SNAT, I have a SNAT list linked to this Virtual Server.

I see the result for openssl below and see that it couldn't detect or pick up the cert. But the certificates created are Web certs and their root certs already in F5. 

[adm@Host:Active:Changes Pending] ~  openssl s_client -connect 172.20.50.20:443
CONNECTED(00000003)
47898972639784:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:184:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 277 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

something must be wrong with it then. i would recheck the SSL client profile, make sure the certificate and key match.

Yes the SSL client profile is correct. I have also create a few more test certs and so far only certificates from Old CA (Windows 2003) works but those from the new CA (Windows 2012) don't seem to work.

Any special requirements from F5 for Windows 2012 CA ?

there really shouldn't be, i kinda suspect you are doing something differently on windows 2012 which causes the certificates to be invalid for some reason. different template perhaps?

you can check if the key and cert match for one thing

https://support.comodo.com/index.php?/Default/Knowledgebase/Article/View/684/17/how-do-i-verify-that-a-private-key-matches-a-certificate-openssl

I get the error below when i run the command. Do I need to specify the path ? I also was unable to find the key when i run find / -name *.key`

Error opening Certificate Test.pfx
47358422848040:error:02001002:system library:fopen:No such file or directory:bss_file.c:398:fopen('Test.pfx','r')
47358422848040:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:400:

yeah you need to add the path or go to the directory where the key and cert are. these are usually around here

/config/filestore/files_d/Common_d/...

a pfx won't work by the way, that should contain the cert and key. but those should be split at the f5, you use the import pkcs 12 (iis) option? there isn't a password on the key by chance?

Yes correct, when i import the certificate, I use the pkcs12(iis) option and put the password of the certificate.

How do i check if there's a password on the key ?

On the CA, I usually export the certs together with the private key and put in the password. It will then be exported as pfx format.

I also just check on the cert and key. Both matches as below

[adm@Host:Active:Changes Pending] certificate_d  openssl x509 -noout -modulus -in /config/filestore/files_d/Common_d/certificate_d/:Common:Test.crt_77214_1 | openssl md5
(stdin)= 5773260e200ee58e7c89ae5a374d9a64

[adm@Host:Active:Changes Pending] certificate_key_d  openssl rsa -noout -modulus -in /config/filestore/files_d/Common_d/certificate_key_d/:Common:Test.key_77211_1 | openssl md5
(stdin)= 5773260e200ee58e7c89ae5a374d9a64