iRule to create TPDU header

You are heading in the right direction. A few small things right off the bat:

1. Generally you don't need to set IP::client_addr, TCP::client_port and TCP::payload to a variable. For Tcl, it is faster not to do so (particularly for the payload);
2. You don't need to unset variables. Variables are scoped to the connection (unless they are declared global -- but don't do that -- or in the static:: namespace) so they will be freed when the connection closes.

With those quibbles out of the way: what is the server expecting? Does it want 32-bits (the IP) followed by 16 bits (the port) followed by the original data, or does it want the IP address as text, the literal colon, then the port, as text? Also, should this data be inserted at the start of the TCP stream (that's easy) or with each TCP segment (that's difficult, maybe impossible)?

Because the BIG-IP is a normally a full-proxy, for the load-balancing function, the client establishes a connection to the BIG-IP, then an LB decision is made, then the server connection is established. For each incoming segment from the client, data may be collected, modified, then forwarded toward the server. So the trick here is to grab the data from the first segment after the three way handshake and insert your header, followed by whatever data was in that segment. Thereafter, you may allow the flow to continue without alteration. The event where you may manipulate data from the client-side (before sending it to the server) is CLIENT_DATA, which is triggered when TCP::collect is invoked. In this case, it makes sense to do so in the CLIENT_ACCEPTED event, just as you had.

So, this should achieve what you're after:

when CLIENT_ACCEPTED {
    TCP::collect
}

when CLIENT_DATA {
    TCP::payload replace 0 0 [binary format "a*SIIIIIIIc4S" "IP36" 0x01 0x0 0x0 0x0 0x0 0x0 0x0000ffff 0x0 [split [IP::client_addr] .] [TCP::client_port]]
    TCP::release
}

The 0x01 is the "operation code". I used the value of 1 because I don't know what you wish that to be. I also assume that the IPv4 address is high-order padded (it's 64 bits, so I assume the first 32-bits are zero followed by the 32-bit IPv4 address). Finally, I assume all values should be encoded in network byte-order.

The binary format method takes a template that describes how you want to pack the variables that follow the template string. These have meaning as follows:

• a* - take the first parameter (the literal "IP36") and convert all of the letters in 8-bit ASCII encoded values;
• S - take the second parameter (since it is the second item in the template) -- which is the operational code -- and convert it to a 16-bit unsigned integer using big-endian byte order (which is the same as network byte order);
• IIIIII - each of these is a 32-bit unsigned integer in big-endian byte order. In your specification, there is a byte stream 0x00000000 00000000 00000000 00000000 00000000 0000ffff (24 bytes total, or 6 x 4 bytes);
• I - again, a 32-bit unsigned integer in big-endian byte order, this is the leading 4 bytes of the address field (which, by your layout, is 8 bytes, so I assume the first 4 bytes are zeroes);
• c4 - the c is a one byte unsigned integer. c4 means "assume a list of four elements, and convert each element into an 8-bit integer". The split does exactly this: it splits the textual representation of the client IP (as dotted-quad) into a list of four elements, where each element is one byte of the IP address is descending (and big-endian) order;
• S - again, an unsigned 16-bit integer, it is passed the value of the client source port.

This might be of some value:

• iRules 101 - 17 – Mapping Protocol Fields with the Binary Scan Command

Well, assuming that it is ISO 8073, there is an RFC (rfc905 from 1984!) and it relates to the original OSI specification for connection-oriented services (making it a cousin of TCP):

• ISO Transport Protocol Specification ISO DP 8073

Beyond that, the RFC is long and not terribly well organized (it is lifted directly from the ISO specification, apparently), so for better-or-worse, you'll have to try and sort through it. You might consider engaging F5 Professional Services, a for-pay consulting arm of F5. They should be able to take the time to work through protocol details with you and craft an appropriate iRule based on the specification.