Learn F5 Technologies, Get Answers & Share Community Solutions Join DevCentral

Filter by:
  • Solution
  • Technology
Answers

OpenSSL and Heart Bleed Vuln

Get the latest updates on how F5 mitigates Heartbleed

Hi Team,

I know this question is eventually going to be asked - I may as well do it.

With the news today about the Heartbleed OpenSSL Vulnerability (http://heartbleed.com) I wanted to confirm if we are at any risk. All of my LTM V11 and V10 instances are running OpenSSL 0.9.8x which does not appear to be a vulnerable version of OpenSSL... Does the F5 hook into this when we Sign/Request SSL Certs? If so we're sitting pretty, right?

Thanks.

Updates based on feedback:

ul

Update 2: F5 have published a security advisory on this issue - http://support.f5.com/kb/en-us/solutions/public/15000/100/sol15159.html

21
Rate this Question
Comments on this Question
Comment made 07-Apr-2014 by Kip Young DL 1
I'm also interested in F5's official stance on this.
8
Comment made 08-Apr-2014 by Dale 0
I'm particularly interested in whether Virtual Edition is vulnerable, as it does not have recourse to the Cavium offload.
0
Comment made 08-Apr-2014 by BinaryCanary
I think, but haven't confirmed, that the platform is irrelevant. The NATIVE stack provides an API that offloads to hardware when it is available, but will always do what you ask of it, falling back to software if no hardware is present. The same thing happens with TCP Segment Offload. The TSO subsystem will always transmit your packets, whether or not there is physical hardware available to offload the packet processing.
0
Comment made 08-Apr-2014 by MegaZone
The official AskF5 Solution is out: http://support.f5.com/kb/en-us/solutions/public/15000/100/sol15159.html See also: https://devcentral.f5.com/articles/openssl-heartbleed-cve-2014-0160
3

Answers to this Question

placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Whether OpenSSL is used or not will depend on the cipher string in your SSL profiles. If you're using the DEFAULT or NATIVE strings or something else that only uses native ciphers then OpenSSL is NOT used and the Cavium offload card (and related proprietary SSL/TLS software) is. If you're using something else (a compat cipher) then OpenSSL is. A cipher string specifying any compat cipher, even if the rest are native, will result in OpenSSL being used. <<CORRECTED: As discussed (and proven) below, if you mix native and compat ciphers in a cipher string and a native cipher is negotiated OpenSSL is NOT used. Thanks aFanen01!

Find the definitive list here and modify your profiles accordingly: http://support.f5.com/kb/en-us/solutions/public/13000/100/sol13163.html.

Also remember that you'll get much better performance if you only use native ciphers.

3
Comments on this Answer
Comment made 08-Apr-2014 by squip 122
Thanks very much for that - that's exactly the answer I was looking for. It seems like I am unaffected :)
0
Comment made 13-Apr-2014 by Mahmoud Eldeeb 780
Excellent
0
Comment made 4 months ago by Philip Lim 1

Excellent list of SSL ciphers supported in the NATIVE SSL stack in BIG-IP. Wondering if there is a more recent list?

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

What versions of the OpenSSL are affected?

Status of different versions:

OpenSSL 1.0.1 through 1.0.1f (inclusive) are vulnerable
OpenSSL 1.0.1g is NOT vulnerable
OpenSSL 1.0.0 branch is NOT vulnerable
OpenSSL 0.9.8 branch is NOT vulnerable

veriosn 11.3/11.4 uses openssl 0.9.8, run openssl version on cli to confirm

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

My 11.5.0 comes back as not vulnerable.

0
Comments on this Answer
Comment made 08-Apr-2014 by BinaryCanary
By default, only native ciphers are used.
0
Comment made 10-Apr-2014 by Kiozs 31
how did you test it out ? we did test last 2 days ago, V11.5.0 management access is vulnerable.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

You can check if you are retuning the TLS heartbeat extension using the following from bash:

 openssl s_client -connect google.com:443  -tlsextdebug 2>&1| grep 'server extension "heartbeat" (id=15)' || echo safe 

It with return the extension if affected, otherwise "Safe". If it does return the extension, check your version of OpenSSL ...This should be no-one running an F5 I don't think.

0
Comments on this Answer
Comment made 08-Apr-2014 by wbbigdave 0
From what are you running this bash?
0
Comment made 08-Apr-2014 by squip 122
From my workstation
0
Comment made 08-Apr-2014 by Frank @ UWM 0
The string just before the pipe symbol is translated. It should be 2>&1 to tie stderr to stdout.
0
Comment made 08-Apr-2014 by ionepoch 0
squip ... this is a nice command you pasted warning... for clarification though... i think the source workstation you are testing must not be running 0.9.8 ... i.e. ... i think your testing workstation should ideally be running OpenSSL 1.0.1 ... otherwise your script will report safe when you are not running out of time patching stuff... but, the command you provided could probably be altered to check the local version of openssl first... then test the remote site to check
0
Comment made 08-Apr-2014 by squip 122
Thanks for that Frank - Looks the forums translated as literal HTML, I have corrected it (the code block isn't that user friendly). @ionepoch - I don't think it should matter, it's only checking for the TLS server extension so it should be agnostic of the version running locally I would have thought...
0
Comment made 08-Apr-2014 by Alois 1
I tried openssl s_client -connect google.com:443 -tlsextdebug -> got "TLS server extension "heartbeat" (id=15), len=1" I have done a doublecheck with http://filippo.io/Heartbleed/#google.com:443 -> got "All good, google.com:443 seems not affected!" So I am not sure which tool I should trust? Any further ideas ?
0
Comment made 08-Apr-2014 by squip 122
It just means that they are returning the Heartbeat TLS Extension, but are not necessarily vulnerable... It was just a quick check prior to these PoC sites turning up. I used it to scan through all my sites to check if I return that extension, if I did I did more investigation to see if I need to Patch OpenSSL :)
0
Comment made 08-Apr-2014 by ionepoch 0
@squip Here's a test from a linux box with openssl 1.0.1 (i.e. shows that google could have a problem if they haven't upgraded openssl): # openssl s_client -connect google.com:443 -tlsextdebug 2>&1 | less CONNECTED(00000003) TLS server extension "renegotiation info" (id=65281), len=1 0001 - <SPACES/NULS> TLS server extension "EC point formats" (id=11), len=4 0000 - 03 00 01 02 .... TLS server extension "session ticket" (id=35), len=0 TLS server extension "heartbeat" (id=15), len=1 0000 - 01 . depth=2 C = US, O = GeoTrust Inc., CN = GeoTrust Global CA verify error:num=20:unable to get local issuer certificate verify return:0 --- Certificate chain 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=*.google.com i:/C=US/O=Google Inc/CN=Google Internet Authority G2 1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2 i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA 2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority --- Server certificate ... And here's from a Mac OSX Mavericks 0.9.8 (i.e. does not show the problem): # openssl s_client -connect google.com:443 -tlsextdebug 2>&1 | less CONNECTED(00000003) TLS server extension "renegotiate" (id=65281), len=1 0001 - <SPACES/NULS> depth=2 /C=US/O=GeoTrust Inc./CN=GeoTrust Global CA verify error:num=20:unable to get local issuer certificate verify return:0 --- Certificate chain 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=*.google.com i:/C=US/O=Google Inc/CN=Google Internet Authority G2 1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2 i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA 2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority --- Server certificate ... Long story short... I really think you need to test with openssl 1.0.1 to detect the problem (or... perhaps there are some extra command line arguments to throw at openssl to force the communication to use a certain protocol / cipher etc)
0
Comment made 08-Apr-2014 by squip 122
Very interesting, thanks for the update - I appreciate it! I'm using this now https://gist.github.com/takeshixx/10107280
0
Comment made 09-Apr-2014 by DR A.
To better automate, the following alteration to the above s_client command may be useful: [code]echo "QUIT"|openssl s_client -connect google.com:443 -tlsextdebug 2>&1|grep 'server extension "heartbeat" (id=15)' || echo safe[/code] Or, if you're like me, and want to check all the places you frequent: [code]for s in amazon.com google.com gmail.com secure.supersecret.com:4443; do pt=`echo "$s"|cut -d: -sf2`; [ -z "$pt" ] && pa=':443' || pa=''; echo "QUIT"|openssl s_client -connect $s$pa -tlsextdebug 2>&1|grep 'server extension "heartbeat" (id=15)' >/dev/null && echo -e "VULN\t$s" || echo -e "safe\t$s"; done[/code] Obviously the domains are just for example. Make sure the domains used are those actually in use on the secure version of the website (it isn't always www.mydomain.com, sometimes the secure site is on, or login is handled via, another domain). Lastly, it worked for me. Hope it works for you. Provided as-is.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Hi Guys, I have one question, whether the F5s use Openssl for SSL offloading functionality? Or is it just a tool installed on the F5s to use for SSL testing.

Thanks

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

To the question about offload, it is my understanding that the openssl is used for csr and cert creation but not for the offloading functionality.

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

According to SOL14457 11.5.0 uses 1.0.1e which is vulnerable http://support.f5.com/kb/en-us/solutions/public/14000/400/sol14457.html

Apparently 1.0.1g or greater addresses the vuln.

I would also like to know (for sure) if OpenSSL is used for offloading, James comment above that his 11.5.0 appears not subject to the vulnerability suggests it's not ... regardless I imagine a patch is needed.

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

It's important to note that the management console is also vulnerable, making the question of whether your individual hosts are exposed moot for some installations.

Hopefully F5 will give us some good news this morning.

0
Comments on this Answer
Comment made 08-Apr-2014 by What Lies Beneath 6527
True, but you'd hope the management interface is on a pretty secure, private network rather than a public facing one.
1
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Okay still struggling to get my head around this.

We use 11.4.2 HF2.

We get connections using TLS 1.2 (and some old) using a bespoke set of ciphers including "NATIVE"

I assume the TLS 1.2 connection must be using the NATIVE code, but that we may fall back to older ciphers, and that since the SSL library mod_ssl.so depends on is libssl.so.0.9.8 (readelf -d mod_ssl.so) then this F5 pair are not affected, but that people with 11.5 may be. And I nearly upgraded BECAUSE of the better SSL cipher support in 11.5......

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Seems you are not.

I'd like some clarity around the compat, native mixed cipher strings myself. I assume worse case and that if a compat cipher is specified in the cipher string then regardless of whether a native cipher is actually negotiated and used everything drops down to OpenSSL. Perhaps that's not the case.

0
Comments on this Answer
Comment made 08-Apr-2014 by BinaryCanary
Hmmm, that's an interesting observation. I would think myself that whether or not Openssl is used is decided on a per-connection basis.
0
Comment made 08-Apr-2014 by What Lies Beneath 6527
Shame I can't find anything that confirms things either way.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

tmm --clientciphers NATIVE

will give you the list of the NATIVE ssl ciphers on your bigip Version. For all ciphers that appear on that list, you're not using openssl.

Also, the only BigIP version that is using a vulerable openssl in the wild is v11.5.0.

0
Comments on this Answer
Comment made 08-Apr-2014 by What Lies Beneath 6527
I'm not convinced that's the case when using a mixed cipher string. OpenSSL supports the native ciphers too. I'll try and do some testing and see what I find. You're correct that only v11.5 is vulnerable. As noted earlier, the management GUI is also vulnerable but again only on v11.5.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

It's quite possible to work this out logically: Ref: http://support.f5.com/kb/en-us/solutions/public/13000/100/sol13163.html

  1. There are ciphers which exist in Native but not in Compat.

  2. YOu can specify "NATIVE:COMPAT" as a cipher string

  3. Therefore, the stack used depends on which cipher is in use, with the natural preference being for NATIVE.

0
Comments on this Answer
Comment made 08-Apr-2014 by BinaryCanary
Tested this just now on 11.2.1 using "NATIVE:COMPAT" as my cipher string on ssl profile. I can connect just fine using ciphers that are only present in NATIVE, and I can do the same using ciphers only present in COMPAT list. So that should confirm that the stack used is decided on a per-connection basis.
0
Comment made 08-Apr-2014 by BinaryCanary
One thing I find curious though, is that if I run "openssl ciphers" on my machine, I get a longer list of ciphers than is listed in Sol13163 in the compat list for my version (11.2.1).
0
Comment made 08-Apr-2014 by What Lies Beneath 6527
Yes, OpenSSL supports ALL the NATIVE ciphers as well as the compat ones but obviously those ciphers are handled by TMM and the offload hardware. Your test isn't definitive unless you check the stats and confirm that your connection shows as native, not compat.
0
Comment made 08-Apr-2014 by What Lies Beneath 6527
tmsh show ltm profile client-ssl 'name' should show you what's what if you use it to display stats for a profile with a mix of native and compat.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
ltm profile client-ssl ciphertest {
    app-service none
    ciphers NATIVE:COMPAT
    defaults-from clientssl
}


-----------------------------------------------------------------
Ltm::ClientSSL Profile: ciphertest
-----------------------------------------------------------------
Virtual Server Name                          N/A

Bytes                                    Inbound  Outbound
  Encrypted                                 1.7K      4.7K
  Decrypted                                    0         0

Connections                                 Open   Maximum  Total
  Native                                       0         2      3
  Compatibility                                0         1      1
  Total                                        0         2      4
0
Comments on this Answer
Comment made 08-Apr-2014 by What Lies Beneath 6527
Excellent, thank you. That proves it! :-)
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

so is 11.5.0 safe or do we need to update?

0
Comments on this Answer
Comment made 08-Apr-2014 by What Lies Beneath 6527
There is no update. v11.5 is vulnerable to this issue in two respects: 1) Where the Management Web GUI is concerned and 2) If any of your SSL Profiles contain cipher strings which contain compat ciphers but only if the client negotiates a compat cipher (which a hacker clearly would ensure happens).
1
Comment made 08-Apr-2014 by HR 2
oke waiting for a update then, had an upgrade scheduled for 11.5, but actually our 10.2.4 version is actually safer then :D
0
Comment made 08-Apr-2014 by BinaryCanary
I'm not a security expert, so you may want to verify this independently: 1. The Native Stack on the Bigip v11.5.0 is not vulnerable. 2. You can force use of the native stack by specifying "NATIVE" as your cipher suites list on your SSL profiles. On 11.5, there is a long list of ciphers supported by NATIVE, so this should not cause any significant loss of options for connecting clients.
0
Comment made 08-Apr-2014 by BinaryCanary
That said, I wouldn't update my production boxes to 11.5 just yet, unless there is a pressing need for a feature there. Usually good to give software a few months for other brave souls to test before you upgrade your critical devices.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

This discussion seems to be around BIG-IP hw appliances with offloading ssl acceleration to Cavium card. What about Virtual Editions, they do all offload on sw - are they vulnerable?

0
Comments on this Answer
Comment made 08-Apr-2014 by What Lies Beneath 6527
Interesting point. I think it's no different as there is still a proprietary stack used by TMM for native ciphers even if there's no hardware acceleration but I can't be 100% sure.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

A solution article has just been released regarding this issue.

http://support.f5.com/kb/en-us/solutions/public/15000/100/sol15159.html

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

For those using Edge Client you should know about its vulnerability: https://devcentral.f5.com/questions/edge-client-and-cve-2014-0160-heartbleed

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Virtual servers using an SSL profile configured with the default Native SSL ciphers are not vulnerable. Only virtual servers using an SSL profile configured to use ciphers from the COMPAT SSL stack are vulnerable in BIG-IP 11.5.0 and 11.5.1. In addition, virtual servers that do not use SSL profiles and pass SSL traffic to the back-end web servers will not protect the back-end resource servers.

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

OpenSSL 1.0.1 through 1.0.1f (inclusive) are vulnerable OpenSSL 1.0.1g is NOT vulnerable OpenSSL 1.0.0 branch is NOT vulnerable OpenSSL 0.9.8 branch is NOT vulnerable

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

if you want to test you could use

http://filippo.io/Heartbleed/

-1
Comments on this Answer
Comment made 08-Apr-2014 by Beinhard 10
Some other testtool that you run by yourself. https://gist.github.com/sh1n0b1/10100394 http://s3.jspenguin.org/ssltest.py
0
Comment made 08-Apr-2014 by BinaryCanary
It's easy to know if you're vulnerable to this: You are running an affected openssl version. You can simply run "openssl version" on the CLI.
0
Comment made 08-Apr-2014 by boneyard 4561
that onyl shows half of the picture in some cases and isnt always possible with appliances.
0
Comment made 08-Apr-2014 by cquick11 9
I guess this applies to server-side ssl too, as i did that command and we are using 9.8 and native.
0
Comment made 08-Apr-2014 by Lask1235 0
Something i found to be helpful. The BIG-IP SSL profiles can use ciphers from two different SSL stacks; the NATIVE stack is built into the Traffic Management Microkernel (TMM), and the COMPAT stack, is based on the OpenSSL library. The NATIVE stack is an optimized SSL stack which can be used by the BIG-IP system to leverage hardware acceleration. F5 recommends using the NATIVE stack because it is suitable for most SSL connections. In BIG-IP 11.x, the SSL profiles only use ciphers from the NATIVE SSL stack. To use SSL ciphers from the COMPAT stack, you must manually configure the cipher string for the profile to COMPAT.
0