Three things your proxy can’t do unless it’s a full-proxy
Proxies are one of the more interesting (in my no-doubt biased opinion) “devices” in the network. They’re the basis for caching, load balancing, app security, and even app acceleration services. They’re also a bridge between dev and ops and the network, being commonplace to all three groups and environments in most data center architectures. But not all proxies are built on the same architectural principles, which means not all proxies are created equal. A large number of proxies are half-proxies while others are full-proxies, and the differences between them are what mean the difference between what you can and cannot do with them. In fact, there are three very important things you can do with a full-proxy that you can’t do with a regular old proxy. Before we jump into those three things, let’s review the differences between them, shall we? Half-Proxy Half-proxy is a description of the way in which a proxy, reverse or forward, handles connections. Basically it’s describing the notion that the proxy only mediates connections on the client side. So it only proxies half the communication between the client and the app. The most important thing to recognize about a half-proxy is that it has only one network stack that it shares across both client and server. Full-Proxy By contrast, a full-proxy maintains two distinct network stacks – one on the client side, one of the app side – and fully proxies both sides, hence the name. While a full-proxy can be configured to act like a half-proxy, its value is in its typical configuration, which is to maintain discrete connections to both the client and the server. It is this dual-stack approach that enables a full-proxy to provide capabilities that a half-proxy with its single network stack simply cannot. The Three Things A full-proxy completely understands the protocols for which it proxies and is itself both an endpoint and an originator for those protocols and connections. This also means the full-proxy can have its own TCP connection behavior for each network stack such as buffering, retransmits, and TCP options. With a full-proxy each connection is unique; each can have its own TCP connection behavior. This means that a client connecting to the full-proxy device would likely have different connection behavior than the full-proxy might use for communicating with servers. Full-proxies can look at incoming requests and outbound responses and can manipulate both if the solution allows it. #1 Optimize client side and server side Because it can maintain separate network stacks and characteristics, a full-proxy can optimize each side for its unique needs. The TCP options needed to optimize for performance on the client side’s lower-speed, higher-latency network connection – particularly when mobile devices are being served – are almost certainly very different than those needed to optimize for performance on the server side’s high-speed, low latency data center network connection. A full-proxy can optimize both at the same time and thus provide the best performance possible in all situations. A half-proxy, with its single network stack, is forced to optimize for the average of its connections, which certainly means one side or the other is left with less than optimal performance. #2 Act as a protocol gateway Protocol gateways are an important tool in the architect’s toolbox particularly when transitioning from one version of an application protocol to another, like HTTP/1 to HTTP/2 or SPDY. Because a full proxy maintains those two unique connections, it can accept HTTP/2 on the client side, for example, but speak HTTP/1 to the server (app). That’s because a full-proxy terminates the client connection (the proxy is the server) and initiatives a different connection to the server (the proxy is the client). The protocol used on the client side doesn’t restrict the choice of protocols on the server side. Realistically, any protocol transition that makes sense (and even those that don’t) can be managed with a full-proxy. A programmable full-proxy ensures that even if its an uncommon (and thus not universally supported) that you can code up a gateway yourself without expending effort on reinventing the proxy-wheel. #3 Terminate SSL/TLS Technically this is a specialized case of a protocol gateway but the ascendancy of HTTP/S (and the urgency with which we are encouraged to deploy SSL Everywhere and Encrypt All The Things) makes me treat this as its own case. Basically terminating SSL/TLS is a critical capability in modern and emerging architectures because of the need to inspect and direct HTTP-based traffic (like REST API calls) based on information within the HTTP protocol that would otherwise be invisible thanks to encryption. The ability to terminate SSL/TLS means the proxy becomes the secure endpoint to which clients connect (and ultimately trust). Termination means the proxy is responsible for decrypting requests and encrypting responses and is thus able to “see” into the messages and use the data therein to make routing and load balancing decisions. So the next time you’re looking at a proxy, don’t forget to find out whether it’s a full proxy or not. Because without a full-proxy, you’re limiting your ability to really take advantage of its capabilities and reaping the benefits it can offer modern and emerging application architectures.2.8KViews1like3CommentsLDAP Proxy
Problem this snippet solves: Summary: An LDAP proxy used send read/write requests to different pools. For anyone that is interested, I recently was posed with the problem of proxying ldap requests. The requirement was to send read and write requests to different pools. As any familiar with ldap knows, you need to send a bind request to authenticate. The following will transparently resend the bind requests to the newly selected server prior to sending the new read/write request. Other functionality could be added to this, such as more verification of the ldap fields to ensure a valid request is being made. The expr commands could be optimized with proper bracing. See the TCL expr wiki page for details Code : class ldap_writes { 6 8 10 12 } # v11 compatabile version when RULE_INIT { # Read Pool set static::readPool sun_ldap_read # Write Pool set static::writePool sun_ldap_write # Turn on debugging set static::ldap_debug 0 # A lookup table for debugging array set static::msg_types { 0 "bind request" 1 "bind response" 2 "unbind request" 3 "search request" 4 "search response" 6 "modify request" 7 "modify response" 8 "add request" 9 "add response" 10 "delete request" 11 "delete response" 12 "modifydn request" 13 "modifydn response" 14 "compare request" 15 "compare response" 16 "abandon request" 23 "extended request" 24 "extended response" } } when CLIENT_ACCEPTED { set rebind 0 set binding "" set replayop "" set writing 0 TCP::collect } when CLIENT_DATA { # Grab the current payload collected set payload [TCP::payload] # Pull the first 2 bytes. binary scan $payload H2c ber_t ber_len # The first byte is the tag signifying an LDAP message, # Always is hex 30, if that is not so reject if { $ber_t ne "30" } { reject return } # The second byte is one of two values: # a) The length of the packet minus the above # defining byte and the length byte # OR # b) an octet describing how many subsequent bytes # hold the packet length # In either case the message type (what we are after) # follows the message id field which too can be a variable # number of bytes. set len_bytes 0 if { [expr {[expr {$ber_len + 0x100} % 0x100]} & 128] > 0 } { set len_bytes [expr {[expr ($ber_len + 0x100) % 0x100]} & 127] } # How many bytes is the message id binary scan $payload x[expr {3 + $len_bytes}]c msgid_bytes # The message type is then 4 bytes + number length bytes + number of # message id bytes offset. binary scan $payload x[expr {4 + $len_bytes + $msgid_bytes}]c msgtype # msgtype - BER encoded value, bits 1-5 are the actual # type, 6 is the data type, 7-8 are the data class # Here we only care about the lower 5 bits set msgtype [expr {$msgtype & 31}] if {$static::ldap_debug and [catch { log local0. "message type is: $static::msg_types($msgtype) $msgtype" } ] } { log local0. "Bad message type: $msgtype" reject } # Each connection should start with a bind request # We'll save this packet for later rebinding when we # flip between servers if { $msgtype == 0 } { if {$static::ldap_debug} {log local0. "Bind Request with: ldap_read"} set writing 0 set rebind 0 set binding $payload LB::detach pool $static::readPool # If we come across a write request and are currently not # sending data to the write pool, detach, and set the rebind # flag so we can send the bind packet before we actually send # our write request } elseif {[class match -- $msgtype equals ldap_writes] and $writing != 1} { if {$static::ldap_debug} {log local0. "Rebinding with: ldap_write"} set rebind 1 set writing 1 set replayop $payload TCP::payload replace 0 [TCP::payload length] $binding LB::detach pool $static::writePool # If we come across a read request while we are bound to a write server # we need to detach and rebind with a read server from the read pool } elseif {![class match -- $msgtype equals ldap_writes] and $writing == 1} { if {$static::ldap_debug} {log local0. "Rebinding with: ldap_read"} set rebind 1 set writing 0 set replayop $payload TCP::payload replace 0 [TCP::payload length] $binding LB::detach pool $static::readPool } TCP::release TCP::collect } when SERVER_CONNECTED { # A change in the type of request has been detected # requiring a rebind, we've sent the bind now we need to # wait for the response before we send the actual request if { $rebind == 1 } { TCP::collect } } when SERVER_DATA { if { $rebind == 1 } { set rebind 0 # See above for details on this block. Stupid iRules, no proc grrrr set payload [TCP::payload] # Pull the first 2 bytes. binary scan $payload H2c ber_t ber_len set len_bytes 0 if { [expr {[expr {($ber_len + 0x100) % 0x100}]} & 128] > 0 } { set len_bytes [expr {[expr ($ber_len + 0x100) % 0x100]} & 127] } binary scan $payload x[expr {3 + $len_bytes}]c msgid_bytes binary scan $payload x[expr {4 + $len_bytes + $msgid_bytes}]c msgtype set msgtype [expr {$msgtype & 31}] # If the msgtype we have here is for a bind response just discard # it as we don't need to send it to the client if {$msgtype == 1 } { TCP::payload replace 0 [TCP::payload length] "" } # Now send the actual read or write op to the server # It should now have processed the bind TCP::respond $replayop } TCP::release } # v10 CMP compatible version when RULE_INIT { # Read Pool set static::readPool sun_ldap_read # Write Pool set static::writePool sun_ldap_write # Turn on debugging set static::ldap_debug 0 # A lookup table for debugging array set static::msg_types { 0 "bind request" 1 "bind response" 2 "unbind request" 3 "search request" 4 "search response" 6 "modify request" 7 "modify response" 8 "add request" 9 "add response" 10 "delete request" 11 "delete response" 12 "modifydn request" 13 "modifydn response" 14 "compare request" 15 "compare response" 16 "abandon request" 23 "extended request" 24 "extended response" } } when CLIENT_ACCEPTED { set rebind 0 set binding "" set replayop "" set writing 0 TCP::collect } when CLIENT_DATA { # Grab the current payload collected set payload [TCP::payload] # Pull the first 2 bytes. binary scan $payload H2c ber_t ber_len # The first byte is the tag signifying an LDAP message, # Always is hex 30, if that is not so reject if { $ber_t ne "30" } { reject return } # The second byte is one of two values: # a) The length of the packet minus the above # defining byte and the length byte # OR # b) an octet describing how many subsequent bytes # hold the packet length # In either case the message type (what we are after) # follows the message id field which too can be a variable # number of bytes. set len_bytes 0 if { [expr [expr ($ber_len + 0x100) % 0x100] & 128] > 0 } { set len_bytes [expr [expr ($ber_len + 0x100) % 0x100] & 127] } # How many bytes is the message id binary scan $payload x[expr 3 + $len_bytes]c msgid_bytes # The message type is then 4 bytes + number length bytes + number of # message id bytes offset. binary scan $payload x[expr 4 + $len_bytes + $msgid_bytes]c msgtype # msgtype - BER encoded value, bits 1-5 are the actual # type, 6 is the data type, 7-8 are the data class # Here we only care about the lower 5 bits set msgtype [expr $msgtype & 31] if {$static::ldap_debug and [catch { log local0. "message type is: $static::msg_types($msgtype) $msgtype" } ] } { log local0. "Bad message type: $msgtype" reject } # Each connection should start with a bind request # We'll save this packet for later rebinding when we # flip between servers if { $msgtype == 0 } { if {$static::ldap_debug} {log local0. "Bind Request with: ldap_read"} set writing 0 set rebind 0 set binding $payload LB::detach pool $static::readPool # If we come across a write request and are currently not # sending data to the write pool, detach, and set the rebind # flag so we can send the bind packet before we actually send # our write request } elseif {[matchclass $msgtype equals $::ldap_writes] and $writing != 1} { if {$static::ldap_debug} {log local0. "Rebinding with: ldap_write"} set rebind 1 set writing 1 set replayop $payload TCP::payload replace 0 [TCP::payload length] $binding LB::detach pool $static::writePool # If we come across a read request while we are bound to a write server # we need to detach and rebind with a read server from the read pool } elseif {![matchclass $msgtype equals $::ldap_writes] and $writing == 1} { if {$static::ldap_debug} {log local0. "Rebinding with: ldap_read"} set rebind 1 set writing 0 set replayop $payload TCP::payload replace 0 [TCP::payload length] $binding LB::detach pool $static::readPool } TCP::release TCP::collect } when SERVER_CONNECTED { # A change in the type of request has been detected # requiring a rebind, we've sent the bind now we need to # wait for the response before we send the actual request if { $rebind == 1 } { TCP::collect } } when SERVER_DATA { if { $rebind == 1 } { set rebind 0 # See above for details on this block. Stupid iRules, no proc grrrr set payload [TCP::payload] # Pull the first 2 bytes. binary scan $payload H2c ber_t ber_len set len_bytes 0 if { [expr [expr ($ber_len + 0x100) % 0x100] & 128] > 0 } { set len_bytes [expr [expr ($ber_len + 0x100) % 0x100] & 127] } binary scan $payload x[expr 3 + $len_bytes]c msgid_bytes binary scan $payload x[expr 4 + $len_bytes + $msgid_bytes]c msgtype set msgtype [expr $msgtype & 31] # If the msgtype we have here is for a bind response just discard # it as we don't need to send it to the client if {$msgtype == 1 } { TCP::payload replace 0 [TCP::payload length] "" } # Now send the actual read or write op to the server # It should now have processed the bind TCP::respond $replayop } TCP::release } # v9 compatible version when RULE_INIT { # Read Pool set ::readPool sun_ldap_read # Write Pool set ::writePool sun_ldap_write # Turn on debugging set ::debug 0 # A lookup table for debugging array set ::msg_types { 0 "bind request" 1 "bind response" 2 "unbind request" 3 "search request" 4 "search response" 6 "modify request" 7 "modify response" 8 "add request" 9 "add response" 10 "delete request" 11 "delete response" 12 "modifydn request" 13 "modifydn response" 14 "compare request" 15 "compare response" 16 "abandon request" 23 "extended request" 24 "extended response" } } when CLIENT_ACCEPTED { set rebind 0 set binding "" set replayop "" set writing 0 TCP::collect } when CLIENT_DATA { # Grab the current payload collected set payload [TCP::payload] # Pull the first 2 bytes. binary scan $payload H2c ber_t ber_len # The first byte is the tag signifying an LDAP message, # Always is hex 30, if that is not so reject if { $ber_t ne "30" } { reject return } # The second byte is one of two values: # a) The length of the packet minus the above # defining byte and the length byte # OR # b) an octet describing how many subsequent bytes # hold the packet length # In either case the message type (what we are after) # follows the message id field which too can be a variable # number of bytes. set len_bytes 0 if { [expr [expr ($ber_len + 0x100) % 0x100] & 128] > 0 } { set len_bytes [expr [expr ($ber_len + 0x100) % 0x100] & 127] } # How many bytes is the message id binary scan $payload x[expr 3 + $len_bytes]c msgid_bytes # The message type is then 4 bytes + number length bytes + number of # message id bytes offset. binary scan $payload x[expr 4 + $len_bytes + $msgid_bytes]c msgtype # msgtype - BER encoded value, bits 1-5 are the actual # type, 6 is the data type, 7-8 are the data class # Here we only care about the lower 5 bits set msgtype [expr $msgtype & 31] if {$::debug and [catch { log local0. "message type is: $::msg_types($msgtype) $msgtype" } ] } { log local0. "Bad message type: $msgtype" reject } # Each connection should start with a bind request # We'll save this packet for later rebinding when we # flip between servers if { $msgtype == 0 } { if {$::debug} {log local0. "Bind Request with: ldap_read"} set writing 0 set rebind 0 set binding $payload LB::detach pool $::readPool # If we come across a write request and are currently not # sending data to the write pool, detach, and set the rebind # flag so we can send the bind packet before we actually send # our write request } elseif {[matchclass $msgtype equals $::ldap_writes] and $writing != 1} { if {$::debug} {log local0. "Rebinding with: ldap_write"} set rebind 1 set writing 1 set replayop $payload TCP::payload replace 0 [TCP::payload length] $binding LB::detach pool $::writePool # If we come across a read request while we are bound to a write server # we need to detach and rebind with a read server from the read pool } elseif {![matchclass $msgtype equals $::ldap_writes] and $writing == 1} { if {$::debug} {log local0. "Rebinding with: ldap_read"} set rebind 1 set writing 0 set replayop $payload TCP::payload replace 0 [TCP::payload length] $binding LB::detach pool $::readPool } TCP::release TCP::collect } when SERVER_CONNECTED { # A change in the type of request has been detected # requiring a rebind, we've sent the bind now we need to # wait for the response before we send the actual request if { $rebind == 1 } { TCP::collect } } when SERVER_DATA { if { $rebind == 1 } { set rebind 0 # See above for details on this block. Stupid iRules, no proc grrrr set payload [TCP::payload] # Pull the first 2 bytes. binary scan $payload H2c ber_t ber_len set len_bytes 0 if { [expr [expr ($ber_len + 0x100) % 0x100] & 128] > 0 } { set len_bytes [expr [expr ($ber_len + 0x100) % 0x100] & 127] } binary scan $payload x[expr 3 + $len_bytes]c msgid_bytes binary scan $payload x[expr 4 + $len_bytes + $msgid_bytes]c msgtype set msgtype [expr $msgtype & 31] # If the msgtype we have here is for a bind response just discard # it as we don't need to send it to the client if {$msgtype == 1 } { TCP::payload replace 0 [TCP::payload length] "" } # Now send the actual read or write op to the server # It should now have processed the bind TCP::respond $replayop } TCP::release }1.6KViews1like7CommentsFTp proxy via iRule
Hello, I'm trying to do an anonymous ftp connection through an F5 irule. My logs of the irule look like this: Rule /Common/iRule_ftp_proxy <CLIENT_ACCEPTED>: client FTP accepted Rule /Common/iRule_ftp_proxy <CLIENT_DATA>: client payload - USER anonymous@193.190.198.27 Rule /Common/iRule_ftp_proxy <CLIENT_DATA>: sitename:193.190.198.27 - cmd:USER - uid:anonymous Rule /Common/iRule_ftp_proxy <CLIENT_DATA>: address 193.190.198.27 port 21 Rule /Common/iRule_ftp_proxy <SERVER_CONNECTED>: connected to server Rule /Common/iRule_ftp_proxy <SERVER_DATA>: server payload 220-Welcome to the Belnet public FTP server ftp.belnet.be !This server is located in Brussels, Belgium and operated by Belnet, the BelgianEducation and Research Network. If you have any problem, question or mirrorrequest, please send them to ftpmaint@belnet.be.This archive is available through the following means:RSYNC rsync://rsync.belnet.be (IPv4)HTTP http://ftp.belnet.be (IPv4 + IPv6)FTP ftp://ftp.belnet.be (IPv4 + IPv6) Rule /Common/iRule_ftp_proxy <SERVER_DATA>: server found 220 ok Rule /Common/iRule_ftp_proxy <SERVER_DATA>: server payloadNote: opening too many parallel connections to this host is considered an abuse.All access is logged.Currently used storage capacity : 34T / 100T on /ftp220 193.190.198.27 FTP server ready Rule /Common/iRule_ftp_proxy <SERVER_DATA>: server payload 331 Anonymous login ok, send your complete email address as your password When I check with a wireshark capture on my client; the '331 Anonymous login ok, send your complete email address as your password' never reaches the client. Without the proxy, it does reach the client and filezilla knows how to deal with it. This is my server_data in the irule; when SERVER_DATA { if { $static::debug } { log local0. "server payload [TCP::payload]" } if { [TCP::payload] starts_with "220" }{ if { $static::debug } { log local0. "server found 220 ok" } TCP::respond "USER $uid\r\n" TCP::payload replace 0 [TCP::payload length] "" } TCP::release TCP::collect } I also tried modifying the rule like this: when SERVER_DATA { if { $static::debug } { log local0. "server payload [TCP::payload]" } if { [TCP::payload] starts_with "220" }{ if { $static::debug } { log local0. "server found 220 ok" } TCP::respond "USER $uid\r\n" TCP::payload replace 0 [TCP::payload length] "" } if { [TCP::payload] starts_with "331" }{ TCP::respond "PASS $uid@example.com\r\n" TCP::payload replace 0 [TCP::payload length] "" } TCP::release TCP::collect } This get's me one step further, but I'm still not able to connect. When I check with a wireshark capture on my client; the '331 Anonymous login ok, send your complete email address as your password' never reaches the client. Without the proxy, it does reach the client and filezilla knows how to deal with it. Rule /Common/iRule_ftp_proxy <SERVER_DATA>: server payload 230 Anonymous access granted, restrictions apply Why is the '331 server_data' not being forwarded to my client so it can respond to it, or is this the task of the proxy? And why is the '230 server data' not reaching the client? Cheers558Views1like0Comments