Forum Discussion

Tony2020's avatar
Tony2020
Icon for Nimbostratus rankNimbostratus
Mar 24, 2017

Error on LTM with getting IP from data group in an IRULE

Hi All,

 

Here is a few questions I have that someone may know of...

 

Please help with the TLC error below. Is there something wrong with the iRule where it's giving this error on the load balancer in the /var/log/ltm? Is the "set CHECK_IP" incorrect or something in the data group (DG-ALLOWED-IP-XFF) the irule doesn't like? It seems to be working from my testing, but the errors is concerning me. We need to resolve this before we go live soon.

 

Also can you guys help to format this into a much better code? Seems there are too many "if" statement trying to do the same thing...maybe group it into an easier format where if the IP address from the XFF is in the data group, than allow to the URIs also in the data group the users is trying to access..

 

Here are the objectives:
  1. when users goes to URIs defined in the "DG-ALLOWED-URI-LIST" data group, and their IP in the XFF header matches what's defined in the data group "DG-ALLOWED-IP-XFF", they will be allowed to access. If not, then their session will get rejected trying to go to those URI. All other access is allowed.

 

 

  1. The second block matches the --> [HTTP::uri] eq "/uri1/uri2/uri3/adminpage" --- so if the user is trying to get here via a link on the main page, or go there directly, their IP in the XFF header must match the data group "DG-ALLOWED-IP-XFF", and if so, they will get redirected to --> https://[HTTP::host]/secret/uri1/uri2/uri3/adminpage/

 

ERROR from /var/log/ltm: -- seems to be something with the $CHECK_IP event...

 

Mar 23 23:18:29 F5LTM01 err tmm2[19841]: 01220001:3: TCL error: /Common/iRULE-TEST-XFF - bad IP network address format (line 7)invalid IP match item %2527 for IP class /Common/DG-ALLOWED-IP-XFF (line 7) invoked from within "class match $CHECK_IP eq "DG-ALLOWED-IP-XFF""

 

 

 

Note: The IP in the data group was changed to 1.1.1.1/2.2.2.2 as an example to protect our real IP.

 

ltm data-group internal DG-ALLOWED-IP-XFF {
    records {

        1.1.1.1/32 { }

        2.2.2.2/32 { }

    }

    type ip

}


ltm data-group internal DG-ALLOWED-URI-LIST {
    records {

        /secure1 { }

        /secure2 { }

        /secure3 { }

    }

    type string

}



when HTTP_REQUEST {
if { [active_members POOL-WEBSERVER-443] < 1  } {
HTTP::redirect " http://site.maintenance-page.com"

} else {

set CHECK_IP [getfield [HTTP::header values X-Forwarded-For] " " 1]

  if { !([class match $CHECK_IP eq "DG-ALLOWED-IP-XFF"]) } {

      if { [class match [HTTP::uri] eq "DG-ALLOWED-URI-LIST"] } {
          reject 
       }

      if { ([class match $CHECK_IP eq "DG-ALLOWED-IP-XFF"]) } {

       if { [HTTP::uri] eq "*/uri1/uri2/uri3/adminpage*"} {

           HTTP::redirect "https://[HTTP::host]/secret/uri1/uri2/uri3/adminpage/" }

           log local0. "/adminpage redirect to /secret for internal users: \ [HTTP::uri]->[IP::client_addr]->[IP::local_addr]"

       }

    }

  }

}

Appreciate any help anyone can provide.

 

11 Replies

  • Guys/ F5,

     

    can anyone help with this? Thank you in advance! We have to get this working by next week and I am in a crunch here. I think i got most of it, but there has to be a better or more efficient way to put all of this together, and the errors I am getting in the LTM log files is a bit concerning..it works for the most part, but the errors should not be there.

     

    Thank you!

     

  • Don't use quotes for the datagroup/class:

    if { !([class match $CHECK_IP eq "DG-ALLOWED-IP-XFF"]) } {

    should be:

    if { !([class match $CHECK_IP eq DG-ALLOWED-IP-XFF]) } {

    If the above doesn't work, try using catch:

    if { !([catch {class match $CHECK_IP eq DG-ALLOWED-IP-XFF} ]) } {

    For troubleshooting purposes use "log local0." statement in order to capture the incoming request value.

  • Hello Tony,

    Thoughts on the XFF header:
    1. Contents of X-Forwarded-For are not guaranteed to be valid or accurate IP addresses.
    2. Multiple headers are possible in a single request.
    3. If a single XFF header contains multiple addresses, it will likely be comma-space delimited e.g.
      1.1.1.1, 2.2.2.2
      though not guaranteed.
    4. It is possible to have a combination of multiple XFF headers with some containing multiple addresses.
    5. The X-Forwarded-For header can be inserted by anyone with any address they choose. It is important to understand filtering by XFF does not provide security.

    If possible, it would be preferable if a custom header colud be added similar to what CDNs do, e.g.

    True-Client-IP
    , to reduce the chance of anomalous values from occuring.

    Parsing IPs from X-Forwarded-For

    When using the method:

    [getfield [HTTP::header values X-Forwarded-For] " " 1]

    which is similar to:

    [lindex [HTTP::header values X-Forwarded-For] 0]

    you may end up with something like:

    1.1.1.1,

    which is not a valid IP causing a TCL error. What I have done in the past

    I'll be honest, I'm not 100% clear on your 2nd objective, but here is a possible rule based on my interpretation. This may not be the most efficient rule, but it should help to reduce the number of TCL error you may encounter and maybe point you in the right direction.

    when HTTP_REQUEST {
        if {[class match [HTTP::uri] starts_with DG-ALLOWED-URI-LIST]} {
             this is a secure URI
    
             check the real source IP
            set MATCH [class match [IP::client_addr] equals DG-ALLOWED-IP-XFF]
    
             cycle the X-Forwarded-For headers
            foreach HDR [HTTP::header values X-Forwarded-For] {
                if {$MATCH} { break }
                 format the X-Forwarded-For header into a valid TCL list
                foreach IP [string trim [regsub -all {[,; ]+} $HDR " "]] {
                    if {[catch {set MATCH [class match $IP equals DG-ALLOWED-IP-XFF]}] != 0} {
                        log local0.err "[virtual name]: Client [IP::client_addr] \
                        had an invalid X-Forwarded-For address - \x22${IP}\x22"
                        set MATCH 0
                    }
                    if {$MATCH} { break }
                }
            }
            if {!$MATCH} { HTTP::respond 403 Connection close }
            unset MATCH
        }
    }
    
    Notes
    1. This rule checks your URI once. The example/objective you provided seems to be performing more than one match so it may need to be modified.
    2. The outside/first
      foreach
      loop cycles each XFF header individually. Usually there is only 1, but it is possible for multiple headers to exist.
    3. The inside/second
      foreach
      loop uses
      regsub
      +
      string trim
      to strip any seperator characters resulting in a valid TCL list.
    4. When matching the address with
      class match
      , the
      catch
      command is used to prevent a TCL error and a log message created for later review.
    5. Instead of
      reject
      , the rule is responding with a 403 Forbidden. The
      reject
      command works too, but my preference is a 403 response which is effectively the HTTP version of reject.
    6. The logic in this rule can also match the real source IP and won't cycle X-Forwarded-For values if it matches. This may or may not be what you need. If not, the rule would need a
      set MATCH 0
      in its place.
    • Tony2020's avatar
      Tony2020
      Icon for Nimbostratus rankNimbostratus

      Thank you Jeremy!

       

      If I want to integrate this in, how would you add it to this code? So if your XFF IP is in the data group, and you are trying to access this URI "/uri1/uri2/uri3/adminpage", than redirect to "/secret/uri1/uri2/uri3/adminpage/"...

       

      Thank you for your recommendation.

       

       if { ([class match $CHECK_IP eq "DG-ALLOWED-IP-XFF"]) } {
      
             if { [HTTP::uri] eq "*/uri1/uri2/uri3/adminpage*"} {
      
                 HTTP::redirect "https://[HTTP::host]/secret/uri1/uri2/uri3/adminpage/" }
    • Jeremy_Church_3's avatar
      Jeremy_Church_3
      Icon for Cirrus rankCirrus

      Tony,

      There are a few options. This rule is not the most efficient way to accomplish your request, but it should be simple to understand, maintain and modify if needed.

      I'll break the logic up into 3 parts:
      1. Determine if the request is for a protected URL.
      2. Check if XFF IP is allowed.
      3. Allow, Redirect or reject depending on results.
      when HTTP_REQUEST {
           1. Check for secure URL
           variable to track if the site is secure
          set SEC 0
          if {[class match [HTTP::uri] starts_with DG-ALLOWED-URI-LIST]} {
              set SEC 1
          } elseif { [string match -nocase {*/uri1/uri2/uri3/adminpage*} [HTTP::uri]] } {
              set SEC 1
              set REDIR "/secret[HTTP::uri]"
          }
      
           2. Check if IP/XFF is allowed
          if { $SEC } {
               this is a secure URL
               check the real source IP
              set MATCH [class match [IP::client_addr] equals DG-ALLOWED-IP-XFF]
      
               cycle the X-Forwarded-For headers
              foreach HDR [HTTP::header values X-Forwarded-For] {
                  if {$MATCH} { break }
                   format the X-Forwarded-For header into a valid TCL list
                  foreach IP [string trim [regsub -all {[,; ]+} $HDR " "]] {
                      if {[catch {set MATCH [class match $IP equals DG-ALLOWED-IP-XFF]}] != 0} {
                          log local0.err "[virtual name]: Client [IP::client_addr] \
                          had an invalid X-Forwarded-For address - \x22${IP}\x22"
                          set MATCH 0
                      }
                      if {$MATCH} { break }
                  }
              }
               3. Allow, redirect or reject
               check for match and perform redirect if needed
              if {$MATCH} {
                  if {[info exists REDIR]} {
                      HTTP::respond 302 Location $REDIR
                      unset REDIR
                  }
              } else {
                  HTTP::respond 403 Connection close
              }
              unset MATCH
          }
          unset SEC
      }
      
      Notes
      1. To match the URI you want to redirect, I used
        string match -nocase
        , but the same can be accomplished with

        if { [string tolower [HTTP::uri]] contains "/uri1/uri2/uri3/adminpage" }

        Both have the same result.
      2. This rule only redirects the secure URL if the client has an approved IP. Another option is to redirect anything that matches
        /uri1/uri2/uri3/adminpage
        regardless of source IP or what's in the XFF header. From the user's perspective, the result would be the same and the rule would be a little shorter and probably a little faster.
      3. Instead of using a whole URL in the redirect, the rule performs a relative redirect pre-pending
        /secure
        onto the original URI. When the browser redirects, it will use the same scheme and domain.
      4. If a redirect is needed, it's critical that the URI you redirect to matches something in the DG-ALLOWED-URI-LIST data-group. If not, you will create a redirect loop.
      5. Instead of using
        HTTP::redirect
        , this rule uses
        HTTP::respond 302 Location ...
        I prefer this method because it clearly shows it's a 302 redirect and makes it simple to change to a 301 if desired. It also allows adding additional headers to the response.

        HTTP::redirect /secure[HTTP::uri]

        is the same as

        HTTP::respond 302 Location /secure[HTTP::uri]

      If this is a rule you plan to use in a production environment, it is important you test it and understand it.

      Let me know how it works.

    • Tony2020's avatar
      Tony2020
      Icon for Nimbostratus rankNimbostratus

      Thanks Jeremy! I appreciate your help on this.

      Here is a version I did, which is much more simpler than your version. Would this work?

      first section monitors the pool for availability. if all nodes are down, it will redirect to a maintenance page.

      second section checks the XFF IP, and if it is in the data group called "DG-ALLOWED-IP-XFF" and they are trying to access the URI in another group called "DG-URI-LIST", it will be allowed, and if your IP is not in the list, you get rejected.

      Third section, most important -- if your IP in the XFF header matches what is in the data group above, and you are trying to go to /uri1/uri2/uri3/adminpage* -- than you will get redirected to another URI https://[HTTP::host]/secret/uri1/uri2/uri3/adminpage/"

      that only users with the IP in the XFF allowed list should get access and redirected to. I used the "switch -glob" statement to match this..

      I have not tested your version yet. Will it do the same as what I have just posted?

      ltm data-group internal DG-ALLOWED-IP-XFF {
      
          records {
      
              1.1.1.1/32 { }
      
              2.2.2.2/32 { }
      
          }
      
          type ip
      
      }
      
      
      ltm data-group internal DG-URI-LIST {
      
          records {
      
              /URI1 { }
      
              /URI2 { }
      
          }
      
          type string
      
      }
      
      
      
      
      
      when HTTP_REQUEST {
      
      if { [active_members POOL-WEBSERVER-443] < 1 } {
      
      HTTP::redirect " http://site-maintenance.com"
      
      
      } else {
      
      
      set CHECK_IP [getfield [HTTP::header values X-Forwarded-For] " " 1]
      
        if { !([class match $CHECK_IP eq DG-ALLOWED-IP-XFF]) } {
      
            if { [class match [HTTP::uri] eq DG-URI-LIST] } {
                   reject 
                     }
                   }
      
              log local0. "Client IP Discard: \ [HTTP::host][HTTP::uri]->[IP::client_addr]->[IP::local_addr]:[TCP::local_port]"
      
      
             switch -glob [HTTP::uri] {
      
               "*/uri1/uri2/uri3/adminpage*" {
      
             if { ([class match $CHECK_IP eq DG-ALLOWED-IP-XFF]) } {
      
                HTTP::redirect "https://[HTTP::host]/secret/uri1/uri2/uri3/adminpage/" }
      
                 log local0. "redirect to admin area for internal users: \ [HTTP::uri]->[IP::client_addr]->[IP::local_addr]"
      
             }
      
          }
      
        }
      
      }