Forum Discussion

beefy80's avatar
beefy80
Icon for Nimbostratus rankNimbostratus
Apr 03, 2014

Calculating a two byte header in message payload

Hello I am currently writing a logging event to capture a message but I am finding that by enabling the TCP::collect event the processing is being slowed down by as much as 2 seconds. As the messages are variable length we have a two byte prefix in front of the message that gives the payload length minus the two byte header e.g. <0x00 0x04> <\x02 12 \x03> I want to try capturing the prefix and calculate the message length from this and then apply a check to see once the message length is reached call TCP::release to try and speed this process up. I am struggling to calculate the two byte header and hoping someone may be able to assist me. I have been dabbling with Binary Scan but cannot get the correct result.

 

Thank you James

 

9 Replies

  • John_Alam_45640's avatar
    John_Alam_45640
    Historic F5 Account

    James

    I would start by collecting only 2 bytes. Convert those from hex to decimal and then collect that number of bytes before you release.

    when CLIENT_ACCEPTED {
       TCP::collect 2
       set header_collected "false"
    }
    
    when CLIENT_DATA {
    
       if { not ($header_collected) } {
           Comment out one of the two binary scan lines below.
           use this form if lower order byte is last (big endian order)
          binary scan [TCP::payload] S message_length
           use this form if lower order byte is first (little endian order)
          binary scan [TCP::payload] s message_length
    
          if { $message_length > 0 } {
             set header_collected "true"
             log local0. "Collecting message with length $message_length"
             TCP::collect $message_length
           }
        } else {
           if we are here the actual message is collected.
          set actual_message_text [TCP::payload]
          TCP::release
        }
     }
    

    Feel free to post what you have and we take a look.

    Examples of binary scan:

    % binary scan \x02\x00 s var1
    1
    % puts $var1
    2
    % binary scan \x00\x03 S var1
    1
    % puts $var1
    3
    %
    

    http://tmml.sourceforge.net/doc/tcl/binary.html

    HTH.
    • beefy80's avatar
      beefy80
      Icon for Nimbostratus rankNimbostratus
      Thank you for this John. I am going to look at this further and will post back how I get on.
    • beefy80's avatar
      beefy80
      Icon for Nimbostratus rankNimbostratus
      Please see enhancement to the code here below.
  • John

     

    I had to enhance your code slightly as the payload messages are quite small and therefore client_data event was not getting triggered again as the LTM has already buffered the rest of the data. I have therefore now added a check to see if we already have the data or not and if we do TCP::release. I cannot see a more affiant way of doing this, what do you think?

     

    when CLIENT_ACCEPTED {
        TCP::collect 2
        set header_collected "false"
        }
    }
    when CLIENT_DATA {
    
        if { not ($header_collected) } {
         Comment out one of the two binary scan lines below.
         use this form if lower order byte is last (big endian order)
        binary scan [TCP::payload] S message_length
         use this form if lower order byte is first (little endian order)
         binary scan [TCP::payload] s message_length
    
         as the message length collected does not include itself add 2 more onto the result
        set message_length [expr { $message_length + 2 }]
    
        if { $message_length > 0 } {
            set header_collected "true"
             collect data with the required length
            TCP::collect $message_length
             check if we already have the required data in the buffer if so 
             capture the payload in vars and release it
            if {[TCP::payload length] == $message_length } {
                 set vars with payload info for HSL logging later
                set my_req_msg_payload [TCP::payload]
                set my_req_msg_length [TCP::payload length]
                 release the data already collected
                TCP::release
            }
           }
        } else {
           if we are here the actual message is collected.
            set my_req_msg_payload [TCP::payload]
            set my_req_msg_length [TCP::payload length]
            TCP::release
        }
    }
  • John_Alam_45640's avatar
    John_Alam_45640
    Historic F5 Account

    Good catch.

    Your modification look great. Only a couple suggestions:

    1) don't add the 2 to the message_length variable because it is used for the TCP::collect later, instead add it within the comparison operation.

    2) Perform the second the TCP::collect in the else statement so that it can be avoided if not needed.

    Here is what it would look like:

    when CLIENT_ACCEPTED {
        TCP::collect 2
        set header_collected "false"
        }
    }
    when CLIENT_DATA {
    
        if { not ($header_collected) } {
         Comment out one of the two binary scan lines below.
         use this form if lower order byte is last (big endian order)
        binary scan [TCP::payload] S message_length
         use this form if lower order byte is first (little endian order)
         binary scan [TCP::payload] s message_length
    
         as the message length collected does not include itself add 2 more onto the result
        
    
        if { $message_length > 0 } {
            set header_collected "true"
    
             check if we already have the required data in the buffer if so 
             capture the payload in vars and release it
            if {[TCP::payload length] == [expr { $message_length + 2 }] } {
                 set vars with payload info for HSL logging later
                set my_req_msg_payload [TCP::payload]
                set my_req_msg_length [TCP::payload length]
                 release the data already collected
                TCP::release
            } else {
                 collect data with the required length
                TCP::collect $message_length            
            }
           }
        } else {
           if we are here the actual message is collected.
            set my_req_msg_payload [TCP::payload]
            set my_req_msg_length [TCP::payload length]
            TCP::release
        }
    }
    
    • beefy80's avatar
      beefy80
      Icon for Nimbostratus rankNimbostratus
      John, I have just tried your enhancement but this does not work for me until our software sends the request again then it works. I have reverted to the previous version of code that I included. I like the code on 11.4 not aware of the procedures function however I am on 11.3 at the moment but will keep that in mind for the future.
  • John_Alam_45640's avatar
    John_Alam_45640
    Historic F5 Account

    if you are on 11.4 or above, here is a slight variation in order to demonstrate iRule procedures "proc".

    proc save_n_let_it_go args {
             the args variable is ignored here 
            set my_req_msg_payload [TCP::payload]
            set my_req_msg_length [TCP::payload length]
             release the data already collected to be forwarded to destination.
            TCP::release
    } 
    
    when CLIENT_ACCEPTED {
        TCP::collect 2
        set header_collected "false"
        }
    }
    when CLIENT_DATA {
    
        if { not ($header_collected) } {
         Comment out one of the two binary scan lines below.
         use this form if lower order byte is last (big endian order)
        binary scan [TCP::payload] S message_length
         use this form if lower order byte is first (little endian order)
         binary scan [TCP::payload] s message_length
    
         as the message length collected does not include itself add 2 more onto the result
        
    
        if { $message_length > 0 } {
            set header_collected "true"
    
             check if we already have the required data in the buffer if so 
             capture the payload in vars and release it
            if {[TCP::payload length] == [expr { $message_length + 2 }] } {
                 set vars with payload info for HSL logging later
                call save_n_let_it_go
            } else {
                 collect data with the required length
                TCP::collect $message_length            
            }
           }
        } else {
           if we are here the actual message is collected.
          call save_n_let_it_go
        }
    }