Forum Discussion

Shu_97237's avatar
Shu_97237
Icon for Nimbostratus rankNimbostratus
Jul 12, 2012

Dispatch messages from a single connection to pool members

Hi,

 

 

I'm trying to write irules to do message-based load balancing. I am not using SIP/radius/diameter, so cannot use an existing profile. The request messages are all sent over a single TCP connection. Each message has a 8-byte header, of which the last 4 bytes store the length of the message body. I want the load balancer to forward messages to pool members in a round-robin manner. Currently, the problem is after "TCP::notify request" is executed, the USER_REQUEST won't fire. The irules are pasted below:

 

 

 

when CLIENT_ACCEPTED {

 

log local0.debug "Get a connection"

 

TCP::collect

 

}

 

 

 

when CLIENT_DATA {

 

log local0.debug "in CLIENT_DATA"

 

binary scan [TCP::payload] II head rlen

 

if {($head & 0x3) == 1} {

 

if {[TCP::payload length] < $rlen} {

 

TCP::collect $rlen

 

log local0.debug "a message is received"

 

TCP::release $rlen

 

TCP::notify request

 

log local0.debug "requested"

 

}

 

}

 

}

 

 

 

when USER_REQUEST {

 

Dispatch to the pool

 

log local0.debug "in USER_REQUEST"

 

pool my_pool

 

}

 

 

 

 

Thank you for your help!

 

Shu

 

 

 

application delivery
dev
devops
iRules

7 Replies

Sort By
  • nitass's avatar
    nitass
    Icon for Employee rankEmployee
    Jul 14, 2012
    this is my testing. client sends 2 http requests in 1 tcp connection. each request is load balanced to each pool member.

    in irule, i determine end of each request by looking for \r\n\r\n. 

    [root@ve10:Active] config  b virtual bar list
virtual bar {
   snat automap
   pool foo
   destination 172.28.19.79:80
   ip protocol 6
   rules myrule
   profiles {
      mblb {}
      tcp {}
   }
}
[root@ve10:Active] config  b pool foo list
pool foo {
   members {
      200.200.200.101:80 {}
      200.200.200.102:80 {}
   }
}
[root@ve10:Active] config  b rule myrule list
rule myrule {
   when CLIENT_ACCEPTED {
   log local0. "client: [IP::client_addr]:[TCP::client_port]"
   log local0. "virtual: [IP::local_addr]:[TCP::local_port]"
   TCP::collect
}
when CLIENT_DATA {
   log local0. "request: [TCP::payload]"
   if {[TCP::payload] ends_with "\r\n\r\n"} {
      TCP::release
      TCP::notify request
   }
   TCP::collect
}
when SERVER_CONNECTED {
   log local0. "node: [IP::remote_addr]:[TCP::remote_port]"
   TCP::collect
}
when SERVER_DATA {
   log local0. "response: [TCP::payload]"
   TCP::release
   TCP::notify response
   TCP::collect
}
when SERVER_CLOSED {
   log local0. ""
}
when CLIENT_CLOSED {
   log local0. ""
}
}

/var/log/ltm
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : client: 172.28.19.251:37382
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : virtual: 172.28.19.79:80
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : request: GET / HTTP/1.0  User-Agent: ApacheBench/2.0.40-dev  Connection: Keep-Alive  Host: 172.28.19.79  Accept: */*
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : node: 200.200.200.102:80
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : response: HTTP/1.1 200 OK  Date: Sat, 14 Jul 2012 08:35:19 GMT  Server: Apache/2.2.3 (CentOS)  Last-Modified: Tue, 08 Nov 2011 12:26:29 GMT  ETag: "4183f1-30-47e02740"  Accept-Ranges: bytes  Content-Length: 48  Keep-Alive: timeout=15, max=100  Connection: Keep-Alive  Content-Type: text/html; charset=UTF-8      This is 102 host.  
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : request: GET / HTTP/1.0  User-Agent: ApacheBench/2.0.40-dev  Connection: Keep-Alive  Host: 172.28.19.79  Accept: */*
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule :
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : node: 200.200.200.101:80
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : response: HTTP/1.1 200 OK  Date: Sat, 14 Jul 2012 08:47:25 GMT  Server: Apache/2.2.3 (CentOS)  Last-Modified: Fri, 11 Nov 2011 14:48:14 GMT  ETag: "4183e4-3e-9c564780"  Accept-Ranges: bytes  Content-Length: 62  Keep-Alive: timeout=15, max=100  Connection: Keep-Alive  Content-Type: text/html; charset=UTF-8       This is 101 host.  
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule :
Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule :
  • Shu_97237's avatar
    Shu_97237
    Icon for Nimbostratus rankNimbostratus
    Jul 16, 2012
    Thank you nitass!

     

     

    I tried to follow your example, but have not succeeded yet. Using "\r\n\r\n" to find the end of message may not work in my application, which does not use HTTP. Is it standard for HTTP messages? In your example, you don't handle USER_REQUEST event. Is there a default action for USER_REQUEST event?

     

     

     

    My understanding is whether I use length field found in message header or use "\r\n\r\n" to find the end of a message, as long as I call TCP::notify request, USER_REQUEST should be triggered. In my case, it has never been triggered. What prevents it from happening?

     

     

     

    Below is the latest configuration I used:

     

     

     

    datastor {

     

    low water mark 80

     

    high water mark 92

     

    }

     

    deduplication {}

     

    shell write partition Common

     

    configsync {

     

    password crypt "****"

     

    }

     

    route default inet {

     

    vlan external

     

    }

     

    monitor MYMBLB {

     

    defaults from inband

     

    }

     

    node 10.124.66.35 {}

     

    node 10.124.66.48 {}

     

    pool MYMBLB_pool {

     

    monitor all MYMBLB

     

    members {

     

    10.124.66.35:cslistener {}

     

    10.124.66.48:cslistener {}

     

    }

     

    }

     

    snat snat_one_ip_test {

     

    translation 10.124.6.35

     

    origins {

     

    10.124.66.35

     

    10.124.66.48

     

    }

     

    vlans external enable

     

    }

     

    rule MYRULES {

     

    when CLIENT_ACCEPTED {

     

    log local0.debug "Get a connection"

     

    set client_closed 0

     

    TCP::collect 1000

     

    }

     

     

     

    when CLIENT_CLOSED {

     

    set client_closed 1

     

    }

     

     

     

     

     

    when SERVER_CONNECTED {

     

    log local0.debug "Server_connected"

     

    after 1000 -periodic if {$client_closed} {TCP::close}

     

    TCP::collect

     

    }

     

     

     

    when CLIENT_DATA {

     

    log local0.debug "in CLIENT_DATA"

     

     

    binary scan [TCP::payload] II head rlen

     

    if {($head & 0x3) == 1} {

     

    if {[TCP::payload length] < $rlen} {

     

    TCP::collect $rlen

     

    log local0.debug "a message is received"

     

    TCP::release $rlen

     

    TCP::notify request

     

    log local0.debug "requested"

     

    }

     

    }

     

    TCP::collect

     

    }

     

     

     

    }

     

    virtual MYMBLBLB {

     

    pool MYMBLB_pool

     

    destination 10.124.6.35:cslistener

     

    ip protocol tcp

     

    rules MYRULES

     

    profiles {

     

    mblb {}

     

    tcp {}

     

    }

     

    }

     

     

  • Shu_97237's avatar
    Shu_97237
    Icon for Nimbostratus rankNimbostratus
    Jul 16, 2012
    I read the wiki of TCP::notify again trying to understand the meaning of "the USER_REQUEST event will not be raised until the server-side TCP connection is complete". Maybe the problem is virtual server cannot establish a connection with the pool member. How can I test if the virtual server can talk to pool members correctly?
  • nitass's avatar
    nitass
    Icon for Employee rankEmployee
    Jul 21, 2012
    Using "\r\n\r\n" to find the end of message may not work in my application, which does not use HTTP. Is it standard for HTTP messages?i used http for testing since i do not have your application. in my case, \r\n\r\n is used to find an end of each message. in your case, you have to parse header to get payload length.

     

     

    In your example, you don't handle USER_REQUEST event. Is there a default action for USER_REQUEST event?you do not need to have USER_REQUEST event. TCP::notify is used to tell bigip it is an end of each message.

     

     

    additionally, TCP::notify is not working in 11.1.0. it is tracked as ID 383853. if you are using 11.1.0, please open a support case for verification.

     

     

    ID 383853 - Need a synchronous event to signal end of message from TCP rule event
  • nitass's avatar
    nitass
    Icon for Employee rankEmployee
    Jul 22, 2012
    not sure if code is correct. anyway, would you mind trying it? i assume only one message per packet.

     

     

    [root@ve10:Active] config  b rule myrule list
rule myrule {
   when CLIENT_ACCEPTED {
   TCP::collect
}
when CLIENT_DATA {
   binary scan [TCP::payload] @4I len
   if {[TCP::payload length] < [expr {8 + $len}]} {
      TCP::collect [expr {8 + $len - [TCP::payload length]}]
      return
   }
   TCP::release [expr {8 + $len}]
   TCP::notify request
   TCP::collect
}
}
  • Shu_97237's avatar
    Shu_97237
    Icon for Nimbostratus rankNimbostratus
    Jul 26, 2012
    Thank you very much nitass!

     

     

    The irules you wrote works. My F5 still doesn't work well, it should be caused by network configuration issue but not irule. The irules to do simple MBLB turned out pretty straight forward.

     

  • Shu_97237's avatar
    Shu_97237
    Icon for Nimbostratus rankNimbostratus
    Jul 26, 2012
    Posted By nitass on 07/22/2012 05:46 AM

     

    not sure if code is correct. anyway, would you mind trying it? i assume only one message per packet.

     

    nitass, I don't understand why you need to assume only one message per packet? Our message is bigger than a TCP packet, e.g. 2830 bytes. Is there anything to pay attention?