Topics


Blogs


Forums


Samples


Media


Labs


Resources

 




DevCentral > Weblogs > Lori MacVittie - Two Different Socks
 Using "X-Forwarded-For" in Apache or PHP
posted on Monday, June 02, 2008 4:20 AM

An issue that often comes up for users of any full proxy-based product is that the original client IP address is often lost to the application or web server. This is because in a full proxy system there are two connections; one between the client and the proxy, and a second one between the proxy and the web server. Essentially, the web server sees the connection as coming from the proxy, not the client. 

Needless to say, this can cause problems if you want to know the IP address of the real client for logging, for troubleshooting, for tracking down bad guys, or performing IP address specific tasks such as geocoding. Maybe you're just like me and you're nosy, or you're like Don and you want the webalizer graphs to be a bit more interesting (just one host does not a cool traffic graph make, after all!).

That's where the "X-Forwarded-For" HTTP header comes into play. Essentially the proxy can, if configured to do so, insert the original client IP address into a custom HTTP header so it can be retrieved by the server for processing.

If you've got a BIG-IP you can simply enable the ability to insert the "X-Forwarded-For" header in the http profile. Check out the screen shot below to see just how easy it is. Yeah, it's that easy.

If for some reason you can't enable this feature in the HTTP profile, you can write an iRule to do the same thing.

when HTTP_REQUEST { HTTP::header insert "X-Forwarded-For" [IP::client_addr] }

Yeah, that's pretty easy, too. So now that you're passing the value along, what do you do with it?

Modifying Apache's Log Format

Well, Joe has a post describing how to obtain this value in IIS. But that doesn't really help if you're not running IIS and like me have chosen to run a little web server you may have heard of called Apache.

Configuring Apache to use the X-Forwarded-For instead of (or in conjunction with) the normal HTTP client header is pretty simple. ApacheWeek has a great article on how to incorporate custom fields into a log file, but here's the down and dirty. Open your configuration file (usually in /etc/httpd/conf/) and find the section describing the log formats. Then add the following to the log format you want to modify, or create a new one that includes this to extract the X-Forwarded-For value:

    %{X-Forwarded-For}i

That's it. If you don't care about the proxy IP address, you can simply replace the traditional %h in the common log format with the new value, or you can add it as an additional header. Restart Apache and you're ready to go.

Getting the X-Forwarded-For from PHP

If you're like me, you might have written an application or site in PHP and for some reason you want the real client IP address, not the proxy IP address. Even though my BIG-IP has the X-Forwarded-For functionality enabled in the http profile, I still need to access that value from my code so I can store it in the database.

$headers = apache_request_headers(); $real_client_ip = $headers["X-Forwarded-For"];

That's it, now I have the real IP address of the client, and not just the proxy's address.

Happy Coding & Configuring!

Imbibing: Coffee



 
      

Feedback


6/4/2008 2:53 AM
Gravatar My fast solution for logging in the same apache-log even if the header X-Forwarder-For not exist.

LogFormat "%{X-Forwarder-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" f5_forwarder
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

SetEnvIf X-Forwarder-For "^.*\..*\..*\..*" is-forwarder

CustomLog logs/access_log combined env=!is-forwarder
CustomLog logs/access_log f5_forwarder env=is-forwarder
Robert

6/10/2008 7:19 AM
Gravatar How safe is it to rely on the IP address captured from X_FORWARDED_FOR HTTP Header for IP-based access restriction?
Venkatesh Kanchan

6/10/2008 7:24 AM
Gravatar The IP address in the HTTP header will only be as reliable as the source. If the BIG-IP (or proxy) is talking directly to the client, the IP address captured is reliable. If you aren't sure about the source of the request - i.e. the client might be accessing the service through a different proxy first, then you can't be certain that the address forwarded is the correct one.

Basically, if the communication is directly between the client and the BIG-IP (assuming your firewall isn't reverse NAT'ing incoming connections) then it's safe to rely on it.

Lori
Lori MacVittie

6/18/2008 7:13 AM
Gravatar I have the same issue. But my webserver only allow https. How can I enable this setting in F5? I cannot apply http profile in this case.
Edwin

6/18/2008 11:11 AM
Gravatar Hi Edwin,

I asked the DevCentral team about this one and the answer is that if you're terminating SSL on the BIG-IP you can use the HTTP profile. But if you aren't terminating it, i.e. you're just letting it pass through, then you won't be able to insert the XFF.

Lori
Lori MacVittie

7/22/2008 6:49 PM
Gravatar Venkatesh Kanchan,

If you're going to use the client IP address LTM receives on the TCP connection as a means to authenticate clients, you should remove all existing XFF headers and then insert a new one using an iRule. This should be considered very secure, as you're not accepting any client side input and instead depending on the actual client IP LTM sees.

when HTTP_REQUEST {
# Remove all existing XFF headers
while {[HTTP::header exists "X-Forwarded-For"}{
HTTP::header remove "X-Forwarded-For"
}
# Insert a new XFF header with the client IP
HTTP::header insert "X-Forwarded-For" [IP::client_addr]
}

Aaron
Aaron

8/22/2008 7:24 AM
Gravatar X-Forwarded-For
Pingback/TrackBack

12/27/2008 7:58 PM
Gravatar The examples given here deal with one tier of f5 - but what if there are two ? I work in a two tiers architecture with one f5 in front of the web tier and another in front of the app tier. Using X-Forwarded-For I see the original ip address of the client in the web tier, but in the app tier I only see the ip address of the web server that made the call to the app tier servers.

How do I resolve this ?

-Kevin.
Kevin

12/29/2008 7:22 AM
Gravatar In the app tier, there must be a way in the application to link the requests from the web tier to requests from the client. If such a link does not exist, the web tier code must be rewritten in order to create such a link.

If such a link exists (but the IP address must be inferred indirectly based upon knowledge from the incoming client request), the F5 can embed information in the web tier request to the app tier, but only if the web tier request passes through the same F5 that handled the original client request. Only a single device which has knowledge from both requests can possibly link them together.

I cannot come up with a specific solution without a lot more details on the application and how it works.
pchang

1/26/2009 3:27 AM
Gravatar Om het origineel ip in de logs van het host die geproxied wordt in de access logs uit te schrijven moet je dit IP uit de headers halen. LogFormat "% i %l %u %t \"%r\" %>s %b \"% i\" \"% i\"" combinedproxy CustomLog /var/log/apache2/access....
Confluence: ICT

1/28/2009 8:35 AM
Gravatar I was trying to write a module for apache in C language using apache development headers....
and i was retirving the IP address from the structure request_rec *r

r->connection->remote_ip

Any idea how to use this (or something else) to get real IP address from X-Forwarded-For header.

pratyush

6/3/2009 10:22 AM
Gravatar Thanks to everybody for their knowledgeable comments. Most of the stuff here was really helpful in getting my F5 to pass the client's IP. One problem I ran into is that im getting two IP addresses in my apache logs..The client's and my proxy's ip address. IS any one running into the same problem...How can I get apache to only log the client's ip address?
Avendanl

6/3/2009 1:25 PM
Gravatar You should only be seeing one IP in the logs. If you're seeing two it's likely because you have a header being inserted with the original address that apache is pulling out and logging, but you're still logging the actual source from the proxy as well. Have you checked your conf settings?

From what I've seen, BIG-IP doesn't pass two addresses. ;)

#Colin
Colin

6/11/2009 9:48 AM
Gravatar Lori's comment is incorrect after the first sentence; the question about using it for security can only be answered "Don't, it is completely unreliable."

Any client can forge this header and send it with any request. It is not only proxies that can insert it. Only trust it if you have created it yourself via your own proxy/loadbalancer from the real client IP.
Allen

5/17/2009 9:02 AM
Gravatar Tomcat : Adresse IP de l’internaute, load balancer, reverse proxy et header Http X-Forwarded-For | Blog Xebia France
Pingback/TrackBack

7/27/2009 5:39 AM
Gravatar There is also the PHP variable: $_SERVER['HTTP_X_FORWARDED_FOR']
Alex

7/1/2009 7:42 PM
Gravatar Is there a way to restrict access to dirs/locations in pure apache config based on X-Forwarded-For ? ie, with out shtml, php,etc...
Pete

7/2/2009 3:31 AM
Gravatar @Pete

Hmmm...I'm not an expert in the Apache mod_X configurations, but I would think that between mod_security and mod_rewrite that you could restrict access based on an just about any HTTP header in the configuration without requiring any code (SHTML, PHP, etc...).

I know you can certainly do that with an iRule, it's pretty simple stuff to base access on an HTTP header value so I'm guessing the same is true of mod_rewrite/mod_security.

Lori
Lori MacVittie

7/9/2009 9:46 AM
Gravatar Is there anything extra that needs to be done on Apache in order for the header to show up in the log? Actually I can't see any header in the Apache log when a request is forwarded from Big IP, but they are present when a request is submitted to Apache directly.
jwang

7/9/2009 9:51 AM
Gravatar @jwang

Other than the mentioned change to the httpd.conf - and a restart - there shouldn't be anything you need to do for Apache.

Lori MacVittie

8/30/2009 5:41 AM
Gravatar I'm using .htaccess files for restrict access to my files, from source IP or networks.
Now I receive all requests with one IP(F5 or proxy) and .htaccess files don't works.
What I do for correct working .htaccess files?
Thanks a lot.
vova

9/13/2009 3:49 AM
Gravatar I wasn't able to get it to work, so I first doubted 'pound'. I installed ngrep and found out with the command 'ngrep -q '^GET .* HTTP/1.[01]' that pound was really adding a header.
I then tried to add this logging to my lighttpd-server on my NAS. It was working there....

Finally I found out it was merely a typo in Robert's example...
He's talking about X-ForwardeR-For instead of X-Forwarded-For.

I always use copy-paste... I wish he did as well. Well, thanks anyhow as it was a good barebones implementation....



LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" proxy
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

SetEnvIf X-Forwarded-For "^.*\..*\..*\..*" is-forwarder

CustomLog logs/access_log combined env=!is-forwarder
CustomLog logs/access_log proxy env=is-forwarder
Frater

11/24/2009 3:58 AM
Gravatar WILS: Client IP or Not Client IP, SNAT is the Question
Lori MacVittie

1/13/2010 9:47 PM
Gravatar http://en.wikipedia.org/wiki/X-Forwarded-For<br /><br />devcentral.f5.com/.../3323.aspx<br /><br />blog.techstacks.com/.../...orwarded-for.html<br /><br />http://code.google.com/p/xebia-france/wiki/Rem...
JIRA: Beanbasket
 Leave Feedback
Title  
Name  
Email
Url
Comments   
Please add 4 and 7 and type the answer here: