Forum Discussion

Philippe_CLOUP's avatar
Sep 17, 2007

looking inside persistence table, through different Virtual Servers

I have created an irule to store, using universal persistence, the 2 things:

 

- the Client-IP adress connected to the VS.

 

- the Mac adress of the "lasthop" that sent the request to the VS.

 

 

Why ?

 

Because i need to send all the trafic generated from the server inside the pool, to the client IP, THROUGH THE SAME EQUIPMENT THAT SENT the incoming connection.

 

 

I created a second iRule, to catch the persistence created on the incoming VS, each time the server is generating an outgoing connection (created a wildcard VS, in perf L4, with "any protocols" - CLIENT_ACCEPTED event, persist across virtuals selected).

 

 

The problem is that i cannot find the entry in the persistence table (persist lookup uie ....). If i try to add it in the table via this irule, the only entry i find is the incoming one, with the "aging time" resetted.

 

 

DO YOU THINK THE PERSISTENCE TABLE CANNOT BE SHARED THROUGH THE OTHER VS, IF THEY ARE ON DIFFERENT VLANS WITH DIFFERENT NODES AND POOLS ?

 

 

Thanks.

8 Replies

  • Do you have a universal persistence profile with "Match Across Virtual Servers" (and possibly "Match across services" if the pool on the incoming VIP is defined on a specific port) enabled, to both virtual servers?

     

     

    Are you using 'persist lookup uie $IP all' to get the existing record?

     

     

    Can you post your rule, vips and profiles with logging?

     

     

    Aaron
  • Here is a part of the bigip_base.conf from my test lab:

     

     

    -----------------------------------

     

     

    vlan PROD {

     

    tag 4094

     

    interfaces 1.1

     

    }

     

    vlan VLAN-RTR-Cisco {

     

    tag 4093

     

    interfaces 1.3

     

    }

     

    stp instance 0 {

     

    vlans

     

    PROD

     

    VLAN-RTR-Cisco

     

    interfaces

     

    1.1

     

    external path cost 20K

     

    internal path cost 20K

     

    1.3

     

    external path cost 20K

     

    internal path cost 20K

     

    }

     

    self allow {

     

    default

     

    tcp ssh

     

    tcp domain

     

    tcp snmp

     

    tcp https

     

    tcp 4353

     

    udp domain

     

    udp snmp

     

    udp efs

     

    udp 1026

     

    udp 4353

     

    proto ospf

     

    }

     

    self 192.168.100.141 {

     

    netmask 255.255.255.0

     

    vlan VLAN-RTR-Cisco

     

    allow default

     

    }

     

    self 192.168.155.141 {

     

    netmask 255.255.255.0

     

    vlan PROD

     

    allow default

     

     

     

     

     

    and the bigip.conf :

     

    -----------------------------

     

     

    profile persist pers_univ {

     

    defaults from universal

     

    mode universal

     

    timeout indefinite

     

    across services enable

     

    across virtuals enable

     

    across pools disable

     

    rule Persist-univ

     

    }

     

     

    pool RTR_OUT {

     

    monitor all gateway_icmp

     

    members 192.168.100.200:http

     

    }

     

    pool rtr_f5 {

     

    monitor all gateway_icmp

     

    members 192.168.155.254:any

     

    }

     

    rule Persist-univ {

     

    when CLIENT_ACCEPTED {

     

    set adr ""

     

    log local0. "LINK vlan_id : [LINK::vlan_id]"

     

    if {[LINK::vlan_id] == "4094"}

     

    {

     

    set adr [IP::client_addr]

     

    log local0. "vlan 4094 from client to servers - $adr"

     

    }

     

    else

     

    {

     

    set adr [IP::local_addr]

     

    log local0. "vlan 4093 from server to client - $adr"

     

    }

     

    set mac1 "00:15:c5:16:94:47"

     

    This mac adress is hardcoded, for tests only

     

    set pipo [concat [string range $adr 0 end]$mac1]

     

    set toto [persist lookup uie $pipo all]

     

    if { $toto == "" }

     

    {

     

    log local0. "UIE PERS - New persistence entry created"

     

    persist uie $pipo

     

    }

     

    else

     

    {

     

    log local0. "UIE PERS - Persistence existing in the table"

     

    }

     

    }

     

    }

     

    rule iRule_Cli2Srv {

     

    when CLIENT_ACCEPTED {

     

    log local0. "client accepted"

     

    adr is storing the IP of the client

     

    set adr [IP::client_addr]

     

    The mac is hardcoded for tests lab only

     

    set mac1 "00:15:c5:16:94:47"

     

    the uie stores the client IP and Mac of the last hop gateway

     

    set test [concat [string range $adr 0 end]$mac1]

     

    log local0. "test variable content : $test"

     

    set toto [persist lookup uie $pipo]

     

    if { $toto ==""}

     

    {

     

    log local0. "No persistence found in the uie table"

     

    persist uie $test

     

    log local0. "uie value stored in the table = $test"

     

    }

     

    else

     

    {

     

    log local0. "Persistence existing in the table : $test"

     

    }

     

     

    }

     

    }

     

    rule iRule_Srv2Cli {

     

    when CLIENT_ACCEPTED {

     

    log local0. "client accepted"

     

    set adr1 [IP::local_addr]

     

    set mac1 "00:15:c5:16:94:47"

     

    set test [concat [string range $adr1 0 end]$mac1]

     

    log local0. "Content of the variable test = $test"

     

    set toto [persist lookup uie $pipo]

     

    if { $toto == "" }

     

    {

     

    log local0. "No persistence Found"

     

    persist uie $test

     

    log local0. "persistence stored in the uie table = $test"

     

    }

     

    else

     

    {

     

    log local0. "Persistence found in the uie table = $test "

     

    }

     

    }

     

    }

     

    virtual address any {

     

    mask none

     

    }

     

    virtual VIP_OUT {

     

    pool rtr_f5

     

    destination any:any

     

    mask none

     

    vlans VLAN-RTR-Cisco enable

     

    rules iRule_Srv2Cli

     

    persist pers_univ

     

    }

     

    virtual VS_IN {

     

    snat automap

     

    pool RTR_OUT

     

    destination 192.168.155.108:http

     

    ip protocol tcp

     

    vlans PROD enable

     

    rules iRule_Cli2Srv

     

    persist pers_univ

     

    }

     

     

     

     

     

    AND HERE ARE THE LOGS OF MY TESTS:

     

    ----------------------------------------------------------

     

    First connection to the VIP of the servers

     

     

    Sep 17 18:44:26 tmm tmm[1666]: Rule: Persist-univ vlan 4094 from client to servers - 192.168.155.57

     

    Sep 17 18:44:26 tmm tmm[1666]: Rule: iRule_Cli2Srv test variable content : 192.168.155.5700:15:c5:16:94:47

     

    Sep 17 18:44:26 tmm tmm[1666]: Rule: iRule_Cli2Srv uie value stored in the table = 192.168.155.5700:15:c5:16:94:47

     

     

     

    Second connection to the VIP of the servers

     

     

    Sep 17 18:44:32 tmm tmm[1666]: Rule: Persist-univ vlan 4094 from client to servers - 192.168.155.57

     

    Sep 17 18:44:32 tmm tmm[1666]: Rule: iRule_Cli2Srv test variable content : 192.168.155.5700:15:c5:16:94:47

     

    Sep 17 18:44:32 tmm tmm[1666]: Rule: iRule_Cli2Srv Persistence existing in the table : 192.168.155.5700:15:c5:16:94:47

     

     

     

    First connection From the Server to the client (.57)

     

     

    Sep 17 18:44:57 tmm tmm[1666]: Rule: Persist-univ LINK vlan_id : 4093

     

    Sep 17 18:44:57 tmm tmm[1666]: Rule: Persist-univ vlan 4093 from server to client - 192.168.155.57

     

    Sep 17 18:44:57 tmm tmm[1666]: Rule: Persist-univ UIE PERS - New persistence entry created

     

    Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli client accepted

     

    Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli Content of the variable test = 192.168.155.5700:15:c5:16:94:47

     

    Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli No persistence Found

     

    Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli persistence stored in the uie table = 192.168.155.5700:15:c5:16:94:47

     

     

     

    What is the content of the persistence table ???

     

    2 entries with the same uie persistence value.

     

     

     

    [root@bip:Active] config b persist all show all

     

    PERSISTENT CONNECTIONS

     

    | Mode universal Value 192.168.155.5700:15:c5:16:94:47

     

    | virtual any:any node 192.168.155.254:any age 30sec

     

    | Mode universal Value 192.168.155.5700:15:c5:16:94:47

     

    | virtual 192.168.155.108:http node 192.168.100.200:http age 55sec

     

     

     

  • Deb_Allen_18's avatar
    Deb_Allen_18
    Historic F5 Account
    The config shown is the inside pair of a firewall sandwich.

    The requirement here is to force server-initiated outbound connections to traverse the same firewall as inbound connections initiated by the same client.

    I think the following approach would work:

    On the external pair, persist inbound traffic by source IP.

    On the internal pair, first create this class cross-referencing server MAC to IP address. Class type is "string". Insert the appropriate MAC address for each internal firewall IP (must match MAC address format as returned by LINK::lasthop):
    class fw_MAC2IP {
      00:00:00:00:00:00 192.168.100.200
      00:00:00:00:00:00 192.168.100.201
    }
    The class will be used for direct outbound node selection by IP based on inbound lasthop MAC.

    On the internal pair, persist inbound & outbound using the following iRule applied to both VS:

    --- For inbound connections, it enforces simple persistence and saves lasthop mac by src IP for reciprocal traffic.

    --- For outbound connections, it looks up dest IP in session table to get fw MAC, then looks up fw MAC in class to get fw IP.
    rule bidirectional_firewall_persistence {
    when RULE_INIT {
      set ::timeout 86400
    }
    when CLIENT_ACCEPTED {
      set server ""
      set origin_vlan [LINK::vlan_id]
      log local0. "vlan ID: $origin_vlan  client: [IP::remote_addr]"
      
      if {$origin_vlan == "4094"}{
         for client originated connections, use server pool & simple persistence 
        set session_key [IP::remote_addr]
        log local0. "client connecting from vlan $origin_vlan - session_key = >$session_key<"
         select inbound pool by name & apply simple persistence
        pool RTR-OUT
        persist source_addr 86400
         create/update session table entry for reciprocal traffic
        session add uie {$session_key any virtual} [LINK::lasthop] $::timeout
        log local0. "RTR-OUT pool selected, source_addr persistence & session table entry added for $session_key"
      } else {
         for server originated conns, look up nexthop MAC for that dest IP in session table
        set session_key [IP::local_addr]
        log local0. "server connecting from vlan $origin_vlan - session_key = >$session_key<"
        set fwMAC [session lookup uie {$session_key any virtual}]
        log local0. "fwMAC found: >$fwMAC< - session_key = >$session_key<"
        if {$fwMAC != ""}{
           If session table entry already exists, refresh to update timeout
          session add uie {$session_key any virtual} $fwMAC $::timeout
          log local0. "session table entry refreshed - session_key = >$session_key<"
           then look up fw IP address in class by MAC
          set fwIP [findclass $fwMAC $::fw_MAC2IP " "]
          log local0. "fw IP >$fwIP< found for MAC $fwMAC - session_key = >$session_key<"
          if {$fwIP != ""}{
             if fw IP address is found in the class for this MAC, directly select fw node
            log local0. "directly selecting fw node $fwIP - session_key = >$session_key<"
            node $fwIP 0
          } else {
             if no fw IP address was found in the class for this MAC, log an error
             This should never happen if class list contains the correct data in the correct format
            log local0. "No IP address found for MAC $fwMAC  Check class list for proper entries."
            log local0. "LB'ing outbound conn to rtr_f5 pool - session_key = >$session_key<"
             select outbound pool by name
            pool rtr_f5
          }
        } else {
           if no session table entry found, load balance the connection
          log local0. "No session table entry for $session_key. LB'ing outbound conn to rtr_f5 pool."
           select outbound pool by name
          pool rtr_f5
        }
      }
    }
    when LB_FAILED {
       detach and choose a new server if the selected server fails to respond.
      LB::detach
      LB::reselect
    }
    }

    HTH

    /deb
  • The result of this iRule is the following:

     

     

    [root@Internal:Active] config b persist all show all

     

    PERSISTENT CONNECTIONS

     

    | Mode source addr Value 10.3.3.252

     

    | virtual 10.3.3.50:80 node 10.5.5.100:80 age 144sec

     

    | Mode universal Value $session_key

     

    | virtual 10.3.3.50:80 node any:any age 102sec

     

     

     

    I think there is a mistake in the line :

     

     

    session add uie {$session_key any virtual} ....

     

     

    the $session_key is not interpreted as a variable.

     

     

  • Deb_Allen_18's avatar
    Deb_Allen_18
    Historic F5 Account
    hmm, I have used that syntax before & not had a issue w/variable expansion... What version are you on?

    You can try changing the 2 lines with that command to explicitly expand the variable:

    session add uie {[list $session_key] any virtual} ....

    If that doesn't work, seeing the logs along with the persist table would be helpful in debugging the situation.

    HTH

    /deb

  • leozou_80567's avatar
    leozou_80567
    Historic F5 Account
    Hi deb,

     

    I test this irule. I have some problem. I can find the lasthop MAC, But I can't find out the lasthop's IP.

     

    When I use real lasthop MAC to replace $fwMAC,I can find out the lasthop's IP.

     

    -----------------------------------------------------------------------

     

    set fwIP [findclass 00:d0:c9:96:83:f7 $::fw_MAC2IP " "] is OK.

     

    set fwIP1 [findclass $fwMAC $::fw_MAC2IP " "] does't OK.

     

    ------------------------------------------------------------------------------

     

     

    So I think there is some problem in $fwMAC.

     

    I check below rule:

     

    ---------------------------------------------------------------------------

     

    if {$origin_vlan == "4093"}{

     

    for client originated connections, use server pool & simple persistence

     

    set session_key [IP::remote_addr]

     

    log local0. "client connecting from vlan $origin_vlan - session_key = >$session_key<"

     

    select inbound pool by name & apply simple persistence

     

    pool pool2-https

     

    persist source_addr 86400

     

    create/update session table entry for reciprocal traffic

     

    session add uie {$session_key any virtual} [LINK::lasthop] $::timeout

     

    log local0. "http_pool1 pool selected, source_addr persistence & session table entry added for $session_key"

     

    set fwMAC [session lookup uie {$session_key any virtual}]

     

    log local0. "session add uie is --->$fwMAC<---------"

     

    ------------------------------------------------------------------------------

     

    when testing, the display is:

     

     

    Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : vlan ID: 4093 client: 10.9.1.11

     

    Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : client connecting from vlan 4093 - session_key = >10.9.1.11<

     

    Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : http_pool1 pool selected, source_addr persistence & session table entry added for 10.9.1.11

     

    Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : session add uie is --->00:d0:c9:96:83:f7

     

    Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : vlan ID: 4093 client: 10.9.1.11

     

    Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : client connecting from vlan 4093 - session_key = >10.9.1.11<

     

    Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : http_pool1 pool selected, source_addr persistence & session table entry added for 10.9.1.11

     

    Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : session add uie is --->00:d0:c9:96:83:f7

     

     

     

    The "session add uie is --->00:d0:c9:96:83:f7" does not include" <-----"

     

    So I think there is some problem about the $fwMAC.

     

     

    Would you pls help check what has happen? For I am urgent to use this iRule.

     

    Thanks.

     

     

     

     

  • Deb_Allen_18's avatar
    Deb_Allen_18
    Historic F5 Account
    hmm....

    The variable itself seems to be expanding just fine, but I have no idea why the trailing characters are not logged.

    It might be worth looking at the output of "b list " to see if those characters are in the config. If the rule looks correct in the config (well really either way), I'd recommend contacting F5 Support to look into the reason for the missing characters in the log command output.

    You could try using the MAC value in a slightly different way, I guess. Maybe just checking the last 2 characters in the MAC?
     set fwIP [findclass [getfield $fwMAC ":" 6] $::fw_MAC2IP " "] 
    against a modified class:
    class fw_MAC2IP {
      10.9.1.11 f7
      10.9.1.12 f8
    }

    HTH

    /deb

  • Here's an updated but untested version of the iRule for 10.1.0 or higher:

    rule bidirectional_firewall_persistence {
    
    when RULE_INIT {
    
     Log debug to /var/log/ltm? (2=all including debug, 1=only log missing mac2ip datagroup entries, 0=none)
    set static::fw_debug 1
    
     Time in seconds to store mappings for
    set static::timeout 86400
    
     Pool name for outbound routers
    set static::outbound_pool "outbound_pool"
    
     Pool name for inbound routers
    set static::inbound_pool "inbound_pool"
    
     Name of string datagroup which maps mac addresses to IP addresses for the routers
    set static::mac2ip_class "mac2ip_class"
    
     VLAN ID of upstream clients
    set static::client_vlanid 4094
    }
    
    when CLIENT_ACCEPTED {
    
    set server ""
    set origin_vlan [LINK::vlan_id]
    if {$static::fw_debug}{log local0. "vlan ID: $origin_vlan client: [IP::client_addr]"}
    
     Check if this a client originated connection by VLAN ID
    if {$origin_vlan == $static::client_vlanid}{
    
     for client originated connections, use server pool & simple persistence 
    set session_key [IP::client_addr]
    if {$static::fw_debug}{log local0. "client connecting from vlan $origin_vlan - session_key = \"$session_key\""}
    
     select inbound pool by name & apply simple persistence
    pool $static::ingbound_pool
    persist source_addr 86400
    
     create/update session table entry for reciprocal traffic
    session add uie [list $session_key any virtual] [LINK::lasthop] $static::timeout
    if {$static::fw_debug}{log local0. "$static::router_out_pool pool selected, source_addr persistence & session table entry added for $session_key"}
    
    } else {
    
     for server originated conns, look up nexthop MAC for that dest IP in session table
    set session_key [IP::local_addr]
    if {$static::fw_debug}{log local0. "server connecting from vlan $origin_vlan - session_key = \"$session_key\""}
    
    set fwMAC [session lookup uie [list $session_key any virtual]]
    if {$static::fw_debug}{log local0. "fwMAC found: \"$fwMAC\" - session_key = \"$session_key\""}
    if {$fwMAC ne ""}{
    
     If session table entry already exists, refresh to update timeout
    session add uie [list $session_key any virtual] $fwMAC $static::timeout
    if {$static::fw_debug}{log local0. "session table entry refreshed - session_key = \"$session_key\""}
    
     then look up firewall MAC address in the datagroup to get the firewall IP address
    set fwIP [class match -value $fwMAC equals $static::mac2ip_class]
    if {$static::fw_debug}{log local0. "fw IP \"$fwIP\" found for MAC $fwMAC - session_key = \"$session_key\""}
    if {$fwIP ne ""}{
    
     if fw IP address is found in the class for this MAC, directly select fw node
    if {$static::fw_debug}{log local0. "directly selecting fw node $fwIP - session_key = \"$session_key\""}
    node $fwIP 0
    
    } else {
    
     if no fw IP address was found in the class for this MAC, log an error
     This should never happen if class list contains the correct data in the correct format
    if {$static::fw_debug}{log local0. "No IP address found for MAC $fwMAC Check class list for proper entries."}
    if {$static::fw_debug}{log local0. "LB'ing outbound conn to $static::outbound_pool pool - session_key = \"$session_key\""}
     select outbound pool by name
    pool $static::outbound_pool
    }
    } else {
     if no session table entry found, load balance the connection
    if {$static::fw_debug}{log local0. "No session table entry for $session_key. LB'ing outbound conn to $static::outbound_pool pool."}
     select outbound pool by name
    pool $static::outbound_pool
    }
    }
    }
    
    when LB_FAILED {
     detach and choose a new server if the selected server fails to respond.
    LB::detach
    LB::reselect
    }
    }
    

    Aaron