I’ve written several articles on the TCP profile (click here) and enjoy digging into TCP.  It’s a beast, and I am constantly re-learning the inner workings.  Still etched in my visual memory map, however, is the TCP header format, shown in Figure 1 below.

Since 9.0 was released, TCP payload data (that which comes after the header) has been consumable in iRules via the TCP::payload and the port information has been available in the contextual commands TCP::local_port/TCP::remote_port and of course TCP::client_port/TCP::server_port.  Options, however, have been inaccessible.  However, beginning with version 10.2.0-HF2, it is now possible to retrieve data from the options fields.

Preparing the BIG-IP

Currently, it is necessary to set a bigpipe database key with the option (or options) of interest:

bigpipe db Rules.Tcpoption.settings [option, first|last], [option, first|last]

The option is an integer between 2 and 255, and the first/last setting indicates whether the system will retain the first or last instance of the specified option.  Once that key is set, you’ll need to do a bigstart restart for it to take (warning: service impacting).  Note also that the LTM only collects option data starting with the ACK of a connection.  The initial SYN is ignored even if you select the first keyword.  This is done to prevent a SYN flood attack (in keeping with SYN-cookies).

A New iRules Command: TCP::option

The TCP::option command has the following syntax:

TCP::option get <option>

Pretty simple, no? So now that you can access them, what fun can be had? 

Real World Scenario: Akamai

In Akamai’s IPA and SXL product lines, they support client IP visibility by embedding a version number (one byte) and an IPv4 address (four bytes) as part of their overlay path feature in tcp option number 28.  To access this data, we first set the database key:

b db Rules.Tcpoption.settings [28,first]

Now, the iRule utilizing the TCP::option command:

   1: when CLIENT_ACCEPTED {
   2:     set opt28 [TCP::option get 28]
   3:     if { [string length $opt28] == 5 } {
   4:         binary scan $opt28 cH8 ver addr
   5:         if { $ver != 1 } {
   6:             log local0. "Unsupported Akamai version: $ver"
   7:         } else {
   8:               scan $addr "%2x%2x%2x%2x" ip1 ip2 ip3 ip4
   9:                 set optaddr "$ip1.$ip2.$ip3.$ip4"
  10:         }
  11:     }
  12: }
  13: when HTTP_REQUEST {
  14:     if { [info exists optaddr] } {
  15:         HTTP::header insert "X-Forwarded-For" $optaddr
  16:     }
  17: }

The Akamai version should be one, so we log if not.  Otherwise, we take the address (stored in the variable addr in hex) and scan it to get the decimal equivalents to build the address for inserting in the X-Forwarded-For header.  Cool, right?  Also cool—along with the new TCP::option command, an extension was made to the IP::addr command to parse binary fields into a dotted decimal IP address.  This extension is also available beginning in 10.2.0-HF2.  Here’s the syntax:

IP::addr parse [-swap] <binary field> [<offset>]

So in the context of our TCP option, we have 5-bytes of data with the first byte not mattering in the context of an address, so we get at the address with this:

set optaddr [IP::addr parse [TCP::option get 28] 1]

This cleans up the rule a bit:

   1: when CLIENT_ACCEPTED {
   2:     set opt28 [TCP::option get 28]
   3:     if { [string length $opt28] == 5 } {
   4:         binary scan $opt c ver
   5:         if { $ver != 1 } {
   6:             log local0. "Unsupported Akamai version: $ver"
   7:         } else {
   8:                 set optaddr [IP::addr parse $opt28 1]
   9:         }
  10:     }
  11: }
  12: when HTTP_REQUEST {
  13:     if { [info exists optaddr] } {
  14:         HTTP::header insert "X-Forwarded-For" $optaddr
  15:     }
  16: }

No need to store the address in the first binary scan and no need for the scan command at all so I eliminated those.  Setting a forwarding header is not the only thing we can do with this data.  It could also be shipped off to a logging server, or used as a snat address (assuming the server had either a default route to the BIG-IP, or specific routes for the customer destinations, which is doubtful).  Logging is trivial, shown below with the log command.  The HSL commands could be used in lieu of log if sending off-box to a log server.

   1: when CLIENT_ACCEPTED {
   2:     set opt28 [TCP::option get 28]
   3:     if { [string length $opt28] == 5 } {
   4:         binary scan $opt c ver
   5:         if { $ver != 1 } {
   6:             log local0. "Unsupported Akamai version: $ver"
   7:         } else {
   8:                 set optaddr [IP::addr parse $opt28 1]
   9:                 log local0. "Client IP extracted from Akamai TCP option is $optaddr"
  10:         }
  11:     }
  12: }

If setting the provided IP as a snat address, you’ll want to make sure it’s a valid IP address before doing so.  You can use the TCL catch command and IP::addr to perform this check as seen in the iRule below:

   1: when CLIENT_ACCEPTED {
   2:     set addrs [list \
   3:         "192.168.1.1" \
   4:         "256.168.1.1" \
   5:         "192.256.1.1" \
   6:         "192.168.256.1" \
   7:         "192.168.1.256" \
   8:         ]
   9:     foreach x $addrs {
  10:         if { [catch {IP::addr $x mask 255.255.255.255}] } {
  11:             log local0. "IP $x is invalid"
  12:         } else { log local0. "IP $x is valid" }
  13:     }
  14: }

The output of this iRule:

<CLIENT_ACCEPTED>: IP 192.168.1.1 is valid
<CLIENT_ACCEPTED>: IP 256.168.1.1 is invalid
<CLIENT_ACCEPTED>: IP 192.256.1.1 is invalid
<CLIENT_ACCEPTED>: IP 192.168.256.1 is invalid
<CLIENT_ACCEPTED>: IP 192.168.1.256 is invalid

Adding this logic into a functional rule with snat:

   1: when CLIENT_ACCEPTED {
   2:     set opt28 [TCP::option get 28]
   3:     if { [string length $opt28] == 5 } {
   4:         binary scan $opt c ver
   5:         if { $ver != 1 } {
   6:             log local0. "Unsupported Akamai version: $ver"
   7:         } else {
   8:                 set optaddr [IP::addr parse $opt28 1]
   9:                 if { [catch {IP::addr $x mask 255.255.255.255}] } {
  10:                     log local0. "$optaddr is not a valid address"
  11:                     snat automap
  12:                 } else {
  13:                         log local0. "Akamai inserted Client IP is $optaddr.  Setting as snat address."
  14:                         snat $optaddr 
  15:                 }
  16:         }
  17:     }
  18: }

Alternative TCP Option Use Cases

The Akamai solution shows an application implementation taking advantage of normally unused space in TCP headers.    There are, however, defined uses for several option “kind” numbers.  The list is available here: http://www.iana.org/assignments/tcp-parameters/tcp-parameters.xml.  Some options that might be useful in troubleshooting efforts:

  • Opkind 2 – Max Segment Size
  • Opkind 3 – Window Scaling
  • Opkind 5 – Selective Acknowledgements
  • Opkind 8 – Timestamps

Of course, with tcpdump you get all this plus the context of other header information and data, but hey, another tool in the toolbox, right?

Related Articles