Forum Discussion

Nick_T1's avatar
Nick_T1
Icon for Nimbostratus rankNimbostratus
Aug 09, 2014

Modifying external datagroup within an irule

Hi all, I'm attempting to log certain requests to an external datagroup in order to provide the logged data to management. The situation is we are testing out a new design of our site. I've got working irules to provide certain users a cookie and then changing their pool based on the cookies value. I would like to then log to a file those users who have opted out. The goal is to do something like this tmsh command but inside the irule:

 

tmsh modify ltm data-group uid_optout_list records add { $fool_UID }

Where the $fool_UID is found by examining the incoming cookie and parsing out their UID. This happens earlier in the code and is working as expected, e.g. log local0. "NAT - UID $fool_UID" performs as expected.

 

Relevant subset of pool mapping code

 

 the cookie will either be "true%3D[an epoch time]" or "false%3D[an epoch time], so we split out true or false to test
          if {([lindex [split [HTTP::cookie value "NewArticleOptIn"] "%3D"] 0] equals "true") and not ($legacy_query_var equals "true")} {
            log local0. "NAT - hit article, optin true - $path"
            if { [class match $path starts_with tmf_poolmap_newsiteall]} {
              log local0. "NAT - class file hit - in poolmap"
              set tmfpool $tmfpool-django
            }
          } elseif {([lindex [split [HTTP::cookie value "NewArticleOptIn"] "%3D"] 0] equals "false") and not ($legacy_query_var equals "true")} {
             checking for people who have opted out
            get the Fool UID from Fool cookie. parse the UID out by using a temporary variable to get everything after the = and then splitting on the & to remove the end
            set fool_cookie [HTTP::cookie value "Fool"]
            set fool_UID1 [lindex [split $fool_cookie "="] 1]
            set fool_UID [lindex [split $fool_UID1 "&"] 0]
            log local0. "NAT - attempting to log false"
            unset fool_UID1 fool_UID fool_cookie
          }

Code setting the cookie for certain users based on point of entry into the site and if they already have it or not.

 

when HTTP_REQUEST {
    if { [class match $path starts_with tmf_poolmap_newsite] } {
        if the cookie doesn't exist, set it to true on response. we don't want to mess with an existing cookie
        if {not [HTTP::cookie exists "NewArticleOptIn"]} {
            get the Fool UID from Fool cookie. parse the UID out by using a temporary variable to get everything after the = and then splitting on the & to remove the end
            set fool_cookie [HTTP::cookie value "Fool"]
            set fool_UID1 [lindex [split $fool_cookie "="] 1]
            set fool_UID [lindex [split $fool_UID1 "&"] 0]
            unset fool_UID1
            if {[class match $fool_UID equals tmf_uid_list]} {
                set new_user "true"
            }
        }
    }
}

when HTTP_RESPONSE {
    if new_user isn't null, we need to send them the cookie
    if {[info exists new_user]} {
        set now [clock seconds]
        set cookie [format "%s=%s; domain=%s; expires=%s; path=/;" "NewArticleOptIn" "true%3D$now" ".fool.com" "[expr $now + 31536000]"]
        HTTP::header insert "Set-Cookie" $cookie
         remove the variables if they exist
        unset new_user
        unset fool_UID
        unset fool_cookie
    }
}

So where I currently am logging to /var/ltm/log, I'd like to drop it into an external datagroup instead so that I end up with one list of unique keys. I've spent a few hours looking for this and can't seem to arrive at a solution, so any guidance would be appreciated. We're currently on 10.2.4 and will be updating to 11.5.1 soon, so my suspicions would be that this needs to use tmsh instead of bigpipe, but if bigpipe gets me to a working solution today, that's fine with me. There is also likely a more elegant way to parse their UID out from the Fool cookie, but that way was quick and working. If you want to advise me on that, the cookie's structure is

 

Uid=&someotherdata

Thanks for your time!

 

P.S. in case you're wondering, I'm not calling my opt-out users fools! I work at The Motley Fool, so it's a term we use positively.

 

6 Replies

  • Just to clarify, the command you suggested "tmsh modify ltm data-group uid_optout_list records add { $fool_UID }" is for an internal datagroup, not an external datagroup. An external datagroup get loaded from a file, which is either on the F5 or external to the F5 (but reachable).

     

    There is no way to affect the control plane (datagroup content) from the data plane in v10.x without involving a third party. Your best bet would be to write a log entry, have that syslog'd to an external system like Logstash and have a trigger there which will run a script to ssh to the F5 and execute the required tmsh command.

     

    In v11.5.1 you can streamline this using iControl REST. Also, I would investigate whether iCall/iStats can help you as an alternative.

     

  • Here's an example of an REST call we use to instruct the F5 to pull a datagroup from our Adobe CQ CMS;-

     

    curl -k -u cd_user:cd_user -H "Content-Type: application/json" -X PUT -d '{"name":"dg_www.company.com.au_uri_longest","partition":"~CompanyDigital","source-path": "http://10.200.58.96:4502/etc/tools/f5mappings.txt.html?match=longest"}' https://10.200.58.180/mgmt/tm/sys/file/data-group/~CompanyDigital~dg_www.company.com.au_uri_longest

     

    I have another idea - you could use a Sideband connection to write a new datagroup entry to an external file (this would involve having an external system to host the file and accept writes via the sideband connection), and then use an iCall/iStat to trigger the external datagroup update from the external file. I'm pretty sure that would work.

     

  • Thanks for your quick response and multiple routes to take. I'll see how they want to proceed tomorrow and go from there.

     

  • @player: I'm sorry for not following up when I saw your question originally!

     

    I ended up creating a sub-table off of the session table that I stored the user's preferences in to allow for a "master" preference that would follow them across devices. I modified this irule (Session Table Exporting) and exported the user's information for our marketing team. We're also switching now to all internal datagroups to ease the manipulation within irules and for the use of iControl from an external application.