You may have heard of the term "truthiness" coined by Stephen Colbert, host of the fake news show "The Colbert Report".

I read a blog entry on Network Computing's site that uses that term in reference to vendor performance claims. At the heart of the discussion that ensues with the vendor is the inability to reproduce the performance claims that the vendor makes in a real environment.

At F5 we take that very seriously.  Over time we have been accused of being too conservative with how we characterize the performance of our devices.  Some of the vendors that we compete against make their performance claims based on truthiness.  A couple of examples are using stats like TCP connection setups per second or what we term infinite to infinite connections. 

The TCP setup per second stat ignores the fact that there is also work to be done to tear the connection down let alone actually send data across the connection - both of which dramatically reduce the original number claimed.  Typically vendors that have some type of hardware acceleration that is good at setting up connections will cite that stat.  Several years ago, Alteon Websystems, now owned by Nortel, used that form of measurement extensively to make big performance claims.  More recenly, we've seen the TCP setup per second claim resurface in the launch materials surrounding Cisco's ACE.  Kudos to Cisco for bringing back a classic.

Citrix's Netscaler product performance claims use what we call infinite to infinite connections.  Basically that is multiplexing as many individual requests as possible over relatively few connections between the client and the Netscaler box as well as between the Netscaler box and the server.  That's something you can do in the lab but that you would never actually see in the field.  We're seeing more customers who were initially lured by the high performance claims switching to F5 when reality did not match the expectations that were set.

We don't use those tricks when we come up with our performance claims I would rather provide data that is useful to customers to make an informed sizing decision.  Will we always win at every possible performance metric that can be cooked up in a lab? No. But I believe that customers appreciate truth over truthiness and that will serve the market better in the long term.