Forum Discussion
Hello Tony,
Thoughts on the XFF header:- Contents of X-Forwarded-For are not guaranteed to be valid or accurate IP addresses.
- Multiple headers are possible in a single request.
- If a single XFF header contains multiple addresses, it will likely be comma-space delimited e.g.
though not guaranteed.1.1.1.1, 2.2.2.2
- It is possible to have a combination of multiple XFF headers with some containing multiple addresses.
- The X-Forwarded-For header can be inserted by anyone with any address they choose. It is important to understand filtering by XFF does not provide security.
If possible, it would be preferable if a custom header colud be added similar to what CDNs do, e.g.
True-Client-IP
, to reduce the chance of anomalous values from occuring.
Parsing IPs from X-Forwarded-For
When using the method:
[getfield [HTTP::header values X-Forwarded-For] " " 1]
which is similar to:
[lindex [HTTP::header values X-Forwarded-For] 0]
you may end up with something like:
1.1.1.1,
which is not a valid IP causing a TCL error.
What I have done in the past
I'll be honest, I'm not 100% clear on your 2nd objective, but here is a possible rule based on my interpretation. This may not be the most efficient rule, but it should help to reduce the number of TCL error you may encounter and maybe point you in the right direction.
when HTTP_REQUEST {
if {[class match [HTTP::uri] starts_with DG-ALLOWED-URI-LIST]} {
this is a secure URI
check the real source IP
set MATCH [class match [IP::client_addr] equals DG-ALLOWED-IP-XFF]
cycle the X-Forwarded-For headers
foreach HDR [HTTP::header values X-Forwarded-For] {
if {$MATCH} { break }
format the X-Forwarded-For header into a valid TCL list
foreach IP [string trim [regsub -all {[,; ]+} $HDR " "]] {
if {[catch {set MATCH [class match $IP equals DG-ALLOWED-IP-XFF]}] != 0} {
log local0.err "[virtual name]: Client [IP::client_addr] \
had an invalid X-Forwarded-For address - \x22${IP}\x22"
set MATCH 0
}
if {$MATCH} { break }
}
}
if {!$MATCH} { HTTP::respond 403 Connection close }
unset MATCH
}
}
Notes
- This rule checks your URI once. The example/objective you provided seems to be performing more than one match so it may need to be modified.
- The outside/first
loop cycles each XFF header individually. Usually there is only 1, but it is possible for multiple headers to exist.foreach
- The inside/second
loop usesforeach
+regsub
to strip any seperator characters resulting in a valid TCL list.string trim
- When matching the address with
, theclass match
command is used to prevent a TCL error and a log message created for later review.catch
- Instead of
, the rule is responding with a 403 Forbidden. Thereject
command works too, but my preference is a 403 response which is effectively the HTTP version of reject.reject
- The logic in this rule can also match the real source IP and won't cycle X-Forwarded-For values if it matches. This may or may not be what you need. If not, the rule would need a
in its place.set MATCH 0
- Tony2020Mar 25, 2017Nimbostratus
Thank you Jeremy!
If I want to integrate this in, how would you add it to this code? So if your XFF IP is in the data group, and you are trying to access this URI "/uri1/uri2/uri3/adminpage", than redirect to "/secret/uri1/uri2/uri3/adminpage/"...
Thank you for your recommendation.
if { ([class match $CHECK_IP eq "DG-ALLOWED-IP-XFF"]) } { if { [HTTP::uri] eq "*/uri1/uri2/uri3/adminpage*"} { HTTP::redirect "https://[HTTP::host]/secret/uri1/uri2/uri3/adminpage/" }
- Jeremy_Church_3Mar 25, 2017Cirrus
Tony,
There are a few options. This rule is not the most efficient way to accomplish your request, but it should be simple to understand, maintain and modify if needed.
I'll break the logic up into 3 parts:- Determine if the request is for a protected URL.
- Check if XFF IP is allowed.
- Allow, Redirect or reject depending on results.
Noteswhen HTTP_REQUEST { 1. Check for secure URL variable to track if the site is secure set SEC 0 if {[class match [HTTP::uri] starts_with DG-ALLOWED-URI-LIST]} { set SEC 1 } elseif { [string match -nocase {*/uri1/uri2/uri3/adminpage*} [HTTP::uri]] } { set SEC 1 set REDIR "/secret[HTTP::uri]" } 2. Check if IP/XFF is allowed if { $SEC } { this is a secure URL check the real source IP set MATCH [class match [IP::client_addr] equals DG-ALLOWED-IP-XFF] cycle the X-Forwarded-For headers foreach HDR [HTTP::header values X-Forwarded-For] { if {$MATCH} { break } format the X-Forwarded-For header into a valid TCL list foreach IP [string trim [regsub -all {[,; ]+} $HDR " "]] { if {[catch {set MATCH [class match $IP equals DG-ALLOWED-IP-XFF]}] != 0} { log local0.err "[virtual name]: Client [IP::client_addr] \ had an invalid X-Forwarded-For address - \x22${IP}\x22" set MATCH 0 } if {$MATCH} { break } } } 3. Allow, redirect or reject check for match and perform redirect if needed if {$MATCH} { if {[info exists REDIR]} { HTTP::respond 302 Location $REDIR unset REDIR } } else { HTTP::respond 403 Connection close } unset MATCH } unset SEC }
- To match the URI you want to redirect, I used
, but the same can be accomplished withstring match -nocase
Both have the same result.if { [string tolower [HTTP::uri]] contains "/uri1/uri2/uri3/adminpage" }
- This rule only redirects the secure URL if the client has an approved IP. Another option is to redirect anything that matches
regardless of source IP or what's in the XFF header. From the user's perspective, the result would be the same and the rule would be a little shorter and probably a little faster./uri1/uri2/uri3/adminpage
- Instead of using a whole URL in the redirect, the rule performs a relative redirect pre-pending
onto the original URI. When the browser redirects, it will use the same scheme and domain./secure
- If a redirect is needed, it's critical that the URI you redirect to matches something in the DG-ALLOWED-URI-LIST data-group. If not, you will create a redirect loop.
- Instead of using
, this rule usesHTTP::redirect
I prefer this method because it clearly shows it's a 302 redirect and makes it simple to change to a 301 if desired. It also allows adding additional headers to the response.HTTP::respond 302 Location ...
is the same asHTTP::redirect /secure[HTTP::uri]
HTTP::respond 302 Location /secure[HTTP::uri]
If this is a rule you plan to use in a production environment, it is important you test it and understand it.
Let me know how it works.
- Tony2020Mar 25, 2017Nimbostratus
Thanks Jeremy! I appreciate your help on this.
Here is a version I did, which is much more simpler than your version. Would this work?
first section monitors the pool for availability. if all nodes are down, it will redirect to a maintenance page.
second section checks the XFF IP, and if it is in the data group called "DG-ALLOWED-IP-XFF" and they are trying to access the URI in another group called "DG-URI-LIST", it will be allowed, and if your IP is not in the list, you get rejected.
Third section, most important -- if your IP in the XFF header matches what is in the data group above, and you are trying to go to /uri1/uri2/uri3/adminpage* -- than you will get redirected to another URI https://[HTTP::host]/secret/uri1/uri2/uri3/adminpage/"
that only users with the IP in the XFF allowed list should get access and redirected to. I used the "switch -glob" statement to match this..I have not tested your version yet. Will it do the same as what I have just posted?
ltm data-group internal DG-ALLOWED-IP-XFF { records { 1.1.1.1/32 { } 2.2.2.2/32 { } } type ip } ltm data-group internal DG-URI-LIST { records { /URI1 { } /URI2 { } } type string } when HTTP_REQUEST { if { [active_members POOL-WEBSERVER-443] < 1 } { HTTP::redirect " http://site-maintenance.com" } else { set CHECK_IP [getfield [HTTP::header values X-Forwarded-For] " " 1] if { !([class match $CHECK_IP eq DG-ALLOWED-IP-XFF]) } { if { [class match [HTTP::uri] eq DG-URI-LIST] } { reject } } log local0. "Client IP Discard: \ [HTTP::host][HTTP::uri]->[IP::client_addr]->[IP::local_addr]:[TCP::local_port]" switch -glob [HTTP::uri] { "*/uri1/uri2/uri3/adminpage*" { if { ([class match $CHECK_IP eq DG-ALLOWED-IP-XFF]) } { HTTP::redirect "https://[HTTP::host]/secret/uri1/uri2/uri3/adminpage/" } log local0. "redirect to admin area for internal users: \ [HTTP::uri]->[IP::client_addr]->[IP::local_addr]" } } } }
- Tony2020Mar 28, 2017Nimbostratus
Jeremy, can you comment to the question above. I have tested your version and section one does not have the DG to validate if the users XFF real client ip is in the DG-ALLOWED-IP-LIST. The remote users needs to be in this IP DG to be able to access the URIs in the DG-ALLOWED-URI-LIST and also be allowed/redirected to the "secret" URI. If they are not in the list, they just access the normal admin URI (/uri1/uri2/uri3/adminpage) but do not get redirected to the "secret" URI/section if they are not in this list.
Thank you! Tony
- Jeremy_Church_3Mar 29, 2017Cirrus
Tony,
If you are asking about the iRule you wrote above, based on my understanding of what you are attempting to accomplish, your statement is correct.
If they are not in the list, they just access the normal admin URI (/uri1/uri2/uri3/adminpage) but do not get redirected to the "secret" URI/section if they are not in this list.
In my example rule, you are correct, section 1 does not have the check. As I mentioned, to accomplish the behavior you described, I wrote the rule in 3 logical parts and all 3 are needed unless it is re-written.
If you do not get redirected with the rule I wrote, then something doesn't match somewhere or something else is wrong. I have no visibility into your environment or the actual URLs. I am under the assumption you provided example URLs so I wrote something based on your description and examples provided. It is not possible for me to analyze why a request isn't matching or if some other configuration is possibly interfering with the iRule behavior.
I know that may not be much help, but I'm not sure what more I can do.
- Tony2020Mar 30, 2017Nimbostratus
Jeremy, thank you for your response and help.
Here is a breakdown of what I was trying to do.
- when a users connect and their IP address matches the data group "DG-ALLOWED-IP-XFF" from XFF header, and they try to access the URIs in the "DG-URI-LIST", they should be allowed to access without issues. If users IP in the XFF does not match, they should get dropped. No need to redirect.
set CHECK_IP [getfield [HTTP::header values X-Forwarded-For] " " 1] if { !([class match $CHECK_IP eq DG-ALLOWED-IP-XFF]) } { if { [class match [HTTP::uri] eq DG-URI-LIST] } { reject } }
- If they try to click on a link on the main page that has the URL of /uri1/uri2/uri3/adminpage* and their XFF IP is in the allowed group, then irule should redirect them and allow access to "https://[HTTP::host]/secret/uri1/uri2/uri3/adminpage/" -- where "/secret" link is the only difference compared to what they originally accessed. If their IP is not in the data group "DG-ALLOWED-IP-XFF", then it should not redirect and allow them access to that link. The only thing we are doing here is checking the client's IP, if match, redirect to the new link, otherwise just let them in access the /uri1/uri2/uri3/adminpage*
switch -glob [HTTP::uri] { "*/uri1/uri2/uri3/adminpage*" { if { ([class match $CHECK_IP eq DG-ALLOWED-IP-XFF]) } { HTTP::redirect "https://[HTTP::host]/secret/uri1/uri2/uri3/adminpage/" }
I think the code I put will work, will need to test. But I wasn't sure if there is a more elegant and easier way to write this other than using the "switch -glob"....
Thanks again,
- Jeremy_Church_3Apr 05, 2017Cirrus
Hello Tony,
Sorry it took so long to get back.
I did mis-understand what you were trying to accomplish. I made a slight modification that should satify your requirements.
In the check, instead of a simple
, I added anif / else
to bypass the 403 response if the URL is the adminpage.elseif
when HTTP_REQUEST { 1. Check for secure URL variable to track if the site is secure set SEC 0 if {[class match [HTTP::uri] starts_with DG-ALLOWED-URI-LIST]} { set SEC 1 } elseif { [string match -nocase {*/uri1/uri2/uri3/adminpage*} [HTTP::uri]] } { set SEC 1 set REDIR "/secret[HTTP::uri]" } 2. Check if IP/XFF is allowed if { $SEC } { this is a secure URL check the real source IP set MATCH [class match [IP::client_addr] equals DG-ALLOWED-IP-XFF] cycle the X-Forwarded-For headers foreach HDR [HTTP::header values X-Forwarded-For] { if {$MATCH} { break } format the X-Forwarded-For header into a valid TCL list foreach IP [string trim [regsub -all {[,; ]+} $HDR " "]] { if {[catch {set MATCH [class match $IP equals DG-ALLOWED-IP-XFF]}] != 0} { log local0.err "[virtual name]: Client [IP::client_addr] \ had an invalid X-Forwarded-For address - \x22${IP}\x22" set MATCH 0 } if {$MATCH} { break } } } 3. Allow, redirect or reject check for match and perform redirect if needed if {$MATCH} { if {[info exists REDIR]} { HTTP::respond 302 Location $REDIR unset REDIR } } elseif {[info exists REDIR]} { unset REDIR } else { HTTP::respond 403 Connection close } unset MATCH } unset SEC }
Is that closer to what you need?
Let me know how it works.
- Tony2020Apr 05, 2017Nimbostratus
no problem on delay. thank you Jeremey. appreciate all you help! nicely and efficiently done coding.