Quantcast



Docs

Blogs

Forums

Samples

Media

Labs

Resources
 




DevCentral > Weblogs > Joe Pruitt - A Software Architect's take on Network Security
 IIS X-Forward-For ISAPI Filter
posted on Friday, September 23, 2005 2:05 PM

A recent customer issue came up where they were load balancing servers but we unable to get the true client address logged in their IIS logs. They had their servers fronted by a BIG-IP and when clients would make requests the address passed to the server was the internal address of the BIG-IP and not that of the client.

This is a common issue with proxies and fortunately there is a standard for forwarding client information. It is the HTTP X-Forwarded-For header which is handled by most proxies. So, I set out to find an existing ISAPI filter to replace the c-ip (client ip) log value in IIS with the contents of the X-Forwarded-For header (if it exists). I was amazed to find that I couldn't find a single instance of any open source (or even commercial) filter that would do this.

So, I dug out Visual Studio and whipped up a filter that does just that. It's very basic and contains no user configuration so all you need to do is plug it into your Web Applications list of ISAPI Filters within the IIS Administration and you're set to go.

We've released the source under the iControl End User License Agreement (available in any iControl SDK download). You can download it in the CodeShare section of DevCentral. If you find a way to optimize this filter, please let me know and I'll update the sources here.

After 24-hours of posting, a customer already returned some performance testing on the filter indicating that it only effected the traffic by less than 1 percent. I'm sure there are ways to optimize the memory allocation in the filter to speed this up a bit more, but I'll leave that for the community to work on.

Oh, and it should be noted that the X-Forwarded-For header isn't supported the same way across all proxy products so you'll want to make sure you test this out before using it. It is expecting the header to only contain an IP Address as it does a straight substitution on the value in the c-ip section of the log entry.

Enjoy!

-Joe

[Listening to: Letterbomb - Green Day - American Idiot (03:48)]

Categories:  

Email This
  del.icio.us
      

Feedback

10/19/2005 7:27 AM
Gravatar Hi Joe

Just out of interest I too have been looking for some thing like this for the passed x years with no avail, I am not a c++ code writer but I could not find-see any code which checked if the address fell in to a private range i.e.

10.0.0.0 through 10.255.255.255
169.254.0.0 through 169.254.255.255 (APIPA)
172.16.0.0 through 172.31.255.255
192.168.0.0 through 192.168.255.255

And if it did leave the original ip as is. Which happens if some body has a network with a internal proxy. Does you DLL cover this or are there any plans to do this or could you produce one it would be greatly appricated

John
Johndoe1234@usa.com

p.s. yes this my real name

John
10/19/2005 9:03 AM
Gravatar John,

Currently the code does a straight replacement IF the X-Forwarded-For header exists.

I guess I'm not clear on why internal networks are different than external ones. In your case doesn't the internal proxy add the X-Forwarded-For header with the true client address? If so, then this filter will replace the address.

The real problem is that the proxy is hiding the original source address and throwing in it's own. So there is no way for the filter to know what the true address is unless the proxy is passing it through directly. In which case you don't need the X-Forwarded-For header. Am I missing something?

I'd be glad to make modifications to the filter if it makes sense to a wide audience. The C++ code is very simple...

-Joe
Joe Pruitt
10/20/2005 9:14 AM
Gravatar I get Johns point and this is why.

comp behind proxy server
=>
proxy server
=>
Internet via isp without proxy
=>
IIS server
=>
x-forwarded.dll
=>
log

Which is no good as you cannot discover anything, location, dns etc as nothing can be determined from the thus resolved IP because by now it is in the log and you cant get any information from it as it is in one of the reserved internal ranges.

This situation obtains where for instance a company is operating an internal network meaning that all external access will be proxied but the isp is not proxying. Therefore if you substitute every x-forward without excluding internal IP ranges any log of such an access will be useless.

As the dll stands it seems that it assumes that any proxy will be an ISP proxy and that therefore the substitution of the x-forwarded will be correct. However this is not guaranteed to be the case.

The dll does exactly the right thing in the majority of cases. i.e. where the access is proxied by the ISP which is frankly most of the time. However if the ISP does not proxy but the machine accessing you is on an internal network then you will lose the real IP not gain it.

Its the best thing I have seen so far that addresses this problem, in fact its the only thing I have seen so far that addresses this problem. I have always been amazed that there wasnt something like this as a server utility anyway as most big ISPs use proxies to cut down bandwidth use.

So well done for coming up with something to deal with this. Which is more than I have!
Dave
10/20/2005 9:36 AM
Gravatar Good points - I'm sold. I'll incorporate the check for internal networks and bypass replacement of the X-Forwarded-For header if the source address is from the ranges specified above.

Since I can't really test this on my setup, if anyone is willing to test out my code changes, I'll be able to post them for everyone to share.

Oh, and I've made a few more optimizations in the code for the cases when X-Forwarded-For headers aren't passed in.

-Joe
Joe Pruitt
10/21/2005 1:28 AM
Gravatar Be happy to test for you. Let me know what you need back etc.
John
10/21/2005 10:13 AM
Gravatar Thanks.

The only thing I really need from you is your contact information? You can send it to me privately by clicking on the "Contact Me" link on the top left of my blog. Then I can get you the updated filter.

-Joe
Joe Pruitt
11/11/2005 9:18 AM
Gravatar This is a very cool thing. Feel free to send a copy off to me, I've love to test too.

- Lynn
lcrumbling -a-t- kioware.com


Lynn Crumbling
3/29/2006 12:55 AM
Gravatar Hello,

I have some troubles activating this ISAPI filter on IIS6.

I did the following:
- Copy the Release DLL to d:\apps\iis\f5xforwardedfor.dll
- Go to IIS6 manager, then Web Service Extensions. Created a new application, name it 'F5 X-Forwarded-For', added the DLL, set status to 'Allow', and save.
- Next, I went to the website, and added the DLL as ISAPI filter, with name 'F5 X-Forwarded-For'.
- Next, stop/start IIS. Status of ISAPI filter is 'Unknown'. Reboot server, still the status is 'Unknown'.

I'm probably missing something terribly obvious, but I can't figure out -what- I'm missing.

Joost
j.b.m.deheer -at- kpn.com
Joost de Heer
3/29/2006 11:17 AM
Gravatar If the status is Unknown, that means that you haven't sent any traffic through it. Try accessing one of the pages on that website and the status will either be an up arrow (good) or a red arrow (bad) but shouldn't be Unknown.

Also, I don't believe you need to create a new application. All I did was to add it to the filters list within an existing website.

-Joe
Joe Pruitt
3/29/2006 11:48 PM
Gravatar Thanks, that was indeed the problem.
Joost de Heer
5/1/2006 8:15 AM
Gravatar Hi,

It seems that this plugin does not work on SSL connections. On the same machine it does work with HTTP. Is the ISAPI not processing SSL requests?
Bas
5/10/2006 8:41 AM
Gravatar Does it actually add an extra header in IIS' log file called X-Forwarded with "real" client's IP or will it simply translate "proxied" IP and list it under C-IP header?
Thanks for a great little utility, much appreciated!
Dimitri
5/10/2006 9:04 AM
Gravatar Glad you like it! As far as I know there is no way to add headers to the IIS log facility. So, this filter will replace the C-IP value with the contents of the X-Forwarded-For header (if it exists).
Joe Pruitt
5/23/2006 3:19 PM
Gravatar Not sure what's different with a SSL configured server. As far as I'm aware, the SSL decryption is performed before the content gets to the ISAPI filters. I see no reason why one would work and the other wouldn't.
Joe Pruitt
6/29/2006 7:11 AM
Gravatar I too hhave not been able to get SSL sites to work with X-Forwarded-For. I do not see a way to configure a profile for https that would allow you to insert the X-Forwarded header information.
In some preliminary testing it does not work, however http does on the same server.
Carl Z
8/2/2006 6:16 PM
Gravatar SSL encrypts the whole HTTP request, including headers.

Unless the Big IP itself is doing the encryption (and it can, with the proper add-on module), it will not be able to add a header to the encrypted request. Moving SSL to the Big IP should solve the problem.

I currently do just that to save on certificate licensing costs and it works wonderfully, X-Forwarded-For headers and all.
Nick
8/23/2006 7:12 AM
Gravatar Just as Nick says;

When the SSL certificate is on the webserver(s) and not on the loadbalancer the loadbalancer passes the packet through and can't modify the http packet (because it's encrypted)

Don't get confused by the REMOTE_ADDR and REMOTE_HOST because these are not sent via the HTTP request. These variables are set by the IIS service.

Don't get confused either as you might think "The traffic contains a source and destination adress right?" Yes, they do, but it's on another level, not the HTTP packet layer.

So why this plugin? Well that's because on HTTP traffic -which is also when BIG IP does the encryption and decryption- from the load balancer to the webserver the client IP is set by adding a HTTP_X_FORWARDED_FOR. This variable is not logged by IIS, thus the F5xforwarded for DLL will copy this value to (i think) the REMOTE_ADDR, which is logged to by IIS to the logfiles.
Bas
8/30/2006 7:04 AM
Gravatar Hi,

I tried to use this dll for my IIS servers running behind BigIPs (using oneconnect, with X-forwarded-for insertion). Initially, I didn't had bigip ip addresses in the logs but an issue with C-IPs because of oneconnect.

But now I got strange entries in my logs: I have on a single line, a C-IP containing multiple IPs, separated by a ",+" sign.

Any idea ?

Thanks,

Vincent
Vincent
9/1/2006 8:19 AM
Gravatar Hi again,

Apparently, these entries are linked to proxies using the same X-Forwarded-For header.

So I manage to insert another header (custom one) through the bigip and another ISAPI filter that use the custom header instead.

This solved my problem.

Thanks anyhow for the code !

Regards,

Vincent
Vincent
11/7/2006 3:23 PM
Gravatar How can I adopt this for ISA 2004. I need to log actual client IP addresses and not the Big IP self address.
Sott
9/9/2007 5:07 AM
Gravatar Hi:

Thanks , it helps a lot when using BigIP.

But I have a headache about another issue, the "IP Address and Domain Name Restrictions" of IIS can not recognizind the real client ip ,even after using this ISAPI.

Can any ISAPI solve this problem?

thanks.

Luc.
Luc
9/11/2007 12:24 PM
Gravatar Will this filter modify the packet before it is processed by the destination? The situation is that a client outside of the firewall makes a request for a certificate to an internal web server. The caching appliance currently sends the its IP address to the internal web server instead of the workstation's IP address (outside of the firewall )that originally requested the certificate. The interal web server then tries to send the certificate to the IP address of the caching device because that is who requested it.
If I understand this coorectly, I could put this DLL file on my internal web server and then the web server will then send the certificate to the workstation that actually requested it.

Correct?
Chuck T.
9/11/2007 12:46 PM
Gravatar It's nothing that complicated. All it does is read the X-Forwarded-For HTTP header sent to the webserver from the firewall (or other intermediary) and replaces the c-ip field in the IIS logs with that value for that request. Otherwise, there is no other way with IIS to show the true identity of a client if a firewall changes the source ip address.

-Joe
Joe Pruitt
9/11/2007 2:08 PM
Gravatar In other words, the filter only affects the IIS log file and does nothing to pass the information to another application. Correct.
Chuck T.
9/19/2007 11:59 AM
Gravatar It seams that this ISAPI is breaking kernel cache in IIS. As soon as I applied the ISAPI to the webserver, I no longer get any URI's cached.

Does anyone have this issue or know how to fix?
Zach
9/20/2007 3:32 PM
Gravatar I am using IIS6 and it works fine. but the IP format is not 100% correct. The IP has displayed as IP,+IP? How to fix it?
...80 - 71.213.54.89,+71.213.54.89 Mozilla...
It really should be
...80 - 71.213.54.89 Mozilla...

How to get ride of ,+ and 2nd set of IP?

thanks,
Xin Wang
10/2/2007 7:22 AM
Gravatar This is really cool!
Here we have ISA and squid before a request gets to our web. Both case, X-Forwarded-For takes the form of IP, IP, ...
So I modified Joe,s original source a bit to take only the first IP for the replacement.
Thanks,
kazu
kazu
10/2/2007 8:11 AM
Gravatar Kazu, can you post your code changes or send them to me and I'll get them integrated into the download?

Thanks!

-Joe
Joe Pruitt
10/5/2007 5:06 AM
Gravatar No problem about that but as you yourself said "X-Forwarded-For header isn't supported the same way across all proxy products", this will not be for everybody.

I see here two saying that it is like "IP,+IP.." in their env while it is "IP IP..." with squid and ISA.

BTW, I am a newbie here, so do not know how to post codes...
kazu
10/18/2007 9:07 AM
Gravatar The dll fails to load when I add it in ISAPI filters. Please help
Anish
10/23/2007 11:37 AM
Gravatar the IP,+IP format is on both 32 and 64 bit (IIS6). I also discovered that in some case 2nd IP is not as same as 1st IP.
Xin Wang
10/29/2007 10:08 AM
Gravatar There is another similar approach and code available from David Wang who is one of the microsoft IIS team I beleive, who also posted his solution within a few days of this one in Sept 2005 !
http://blogs.msdn.com/david.wang/archive/2005/09/28/HOWTO-ISAPI-Filter-which-Logs-original-Client-IP-for-Load-Balanced-IIS-Servers.aspx

THat is configurable with an ini file to give the header requested. I have been using it for a year or so without problems.
Mike Williams
10/29/2007 11:33 AM
Gravatar Thanks for the link Mike. I believe the code in that blog post is very similar to my filter. I've also got support for an ini-file override of the header name.

I think I've got David's post by 5 days though B-).

-Joe
Joe Pruitt
10/31/2007 8:26 AM
Gravatar I was able to get the ISAPI filter working for the IIS Logs. However I need the ability to grab the X-Forwarded-For information for use in an application using IIS6. We have seen no X-Forwarded-For header getting added when doing a var dump.

Any help would be appreicated.

Alan
Alan Karelitz
10/31/2007 8:36 AM
Gravatar The var dump Alan is talking about is our custom Coldfusion-based http response header dump.

What we thought would be good to see (and I know your invention isn't meant to address this), is the X-Forwarded-For incoming request header information being rewritten onto the IIS header so ColdFusion or ASP could see it and utilize the value.

Thanks!
Adam Mihlfried
10/31/2007 10:32 AM
Gravatar This filter does not do anything with the headers. It looks for a header (X-Forwarded-For by default, but overridable by a ini file). If that header is found, it replaces the contents of the IIS C-IP value with the contents of that header.

It does not add or remove headers so any HTTP header that is passed into it, will be passed on to the web application. There is no rewriting needed as this is just a filter. It just sniffs whats coming in and passes it through to the backend.

So, if a X-Forwarded-For header is passed it to the filter, it will be passed out as well. Are you seeing different behvavior? The source is available for download as well if you want to look for yourself at what the filter does.

-Joe
Joe Pruitt
11/9/2007 2:22 PM
Gravatar This may solve some of your problems:
http://www.winfrasoft.com/X-Forwarded-For.htm
Steven Hope
12/8/2007 12:08 PM
Gravatar I recently started using HAProxy and I need an ISAPI filter for IIS. Your plugin is exactly what I need but for some reason this happens when I try to load it. Does it really matter what directory its in while testing?


Event Type: Error
Event Source: W3SVC-WP
Event Category: None
Event ID: 2214
Date: 12/8/2007
Time: 3:05:07 PM
User: N/A
Computer: CA-WEB2
Description:
The HTTP Filter DLL C:\Documents and Settings\Administrator\Desktop\F5XForwardedFor\F5XForwardedFor2005\Release\F5XForwardedFor.dll failed to load. The data is the error.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 05 00 00 00 ....
Jason Dobyns
12/14/2007 3:25 PM
Gravatar I want to use this dll in 64 Bit iis web server, there is a dll for this propouse???
NelToRemex
12/14/2007 3:43 PM
Gravatar The download zip file contains 32bit and 64bit versions.

-Joe
Joe Pruitt
1/4/2008 8:13 AM
Gravatar Hello,
Great piece of work - thanks a lot. From a little bit of research it seems that the X-forwarded-for header will contain a list similar to

X-Forwarded-For: client1, proxy1, proxy2

if the request has been forwarded via proxies. Will this filter just return the client and ignore the proxies?

Thanks again
Stuart Wilkinson
1/4/2008 1:59 PM
Gravatar The filter will take the entire string and replace it in the logs. The source is available if you want to tweak the implementation.

Glad you like it!

-Joe
Joe Pruitt
3/12/2008 10:02 PM
Gravatar Simple and Helpfull. All that I needed.

Thank You!
Antonio Meireles
4/16/2008 11:39 AM
Gravatar Greetings-
Any known issues for this filter related to IIS 6? I can't get the filter to load in IIS 6 worker process isolation mode. IIS 5 compatibility mode seems to work fine. Thanks!!
Jason
4/17/2008 3:59 PM
Gravatar No issues that I know about. I would just make sure that the security settings on the dll are set so that IIS is able to load it properly.

-Joe
Joe Pruitt
4/29/2008 3:17 AM
Gravatar It seems that in the IIS log the last proxy's I.P. address has been replaced with the x-forwarded-for headers, the I.P. addresses of all previous proxys.

It's still important for me to know the I.P. address of the last proxy before the web server. Presumably I should compile my own version of the ISAPI filter to preserve this information?
Kenny
5/2/2008 8:21 AM
Gravatar So what do people do when moving to IIS7 when no ISAPI filter is allowed?
Ed
5/7/2008 7:20 PM
Gravatar Has anyone come up with a solution for "IP Address and Domain Name Restrictions" settings in IIS behind an f5?

What i mean is if you want to restrict access to your web server from specific addresses, generally you can add these is IIS and control and manage who will be able to access your application.
David
6/20/2008 11:36 AM
Gravatar We have the apache load balancer between two IIS 6.0 servers. I installed this ISAPI filter in order to get real client IP by load balancer passing X-FORWARDED-FOR. The only problem is that sometimes IIS logs the real client IP, but sometimes still logs the load balancer IP. For example, you access a xx.ASP page and this .ASP page invokes a yy.JS page too. Inside IIS log, the load balancer IP is logged to access xx.ASP, but the real client IP is logged to access yy.JS. The logging format is IIS W3C Extended Log File Format.

Does anyone have any idea why this is happening? Thanks in advance.
HH
6/24/2008 10:36 AM
Gravatar Hi Joe, thank you for another great product in this ISAPI filter.

We're having issues with BEA Support in getting them to understand how we can get the same functionality working with BEA WebLogic 8.x. For some reason, apparently BEA has not thought this through.

Are there other users besides us who use WL behind an LTM?
Ty Young
6/26/2008 5:31 AM
Gravatar Is this ISAPI filter purely for changing the logging information or would it work for IP authentication in IIS as well?
I have some virtual directories on my site that are locked down by client IP ranges - this obviously does not work when all you get is the F5 ip address. The only way I have been advised around this at present involves switching off Address Translation on the VS and changing the default gateway of the nodes to the F5 (not sure if this will work if F5 and nodes are on the same subnet)
JD
6/26/2008 10:01 AM
Gravatar Right now, all it does is look for an incoming X-Forwarded-For HTTP header. If it's found, the value in it is used for the c-ip value in the IIS access logs. I'm sure you could write a filter that does what you propose, or take the source from the XFF filter and tailor it to your needs, but this one doesn't do what you are asking.

-Joe
Joe Pruitt
8/15/2008 2:22 AM
Gravatar Does the 64bit version work?
Jon Dyke
8/25/2008 10:08 AM
Gravatar As far as I can tell it does, but I have no way to test it.

-Joe
Joe Pruitt
8/26/2008 3:23 AM
Gravatar Cool. Works Perfectly.


I am running squid/Reverse proxy.

And also, I am using squid to provide SSL service for my site.

I was using SSLwrap, but the performance is slow, as the connections are closed, and it doesn't allow for parallel requests.

So I tried squid, much faster, but still I am not able to track the IP addresses.

But this filter makes things 100%.

thanks.

Now I know hows on my ssl server.

It would be cool, if the filter could modify:
REMOTE_ADDR=192.168.0.1
REMOTE_HOST=192.168.0.1

so, these fields would show the real ip address of the client, thus making perl scripts, and other applications not know they are behind a proxy.

anyways. great job.
Sheldon Irving
9/2/2008 8:15 AM
Gravatar IIS 7 and ISAPI DLLs.

IIS 7 still supports ISAPI DLLS. The KB article is located here:

http://technet.microsoft.com/en-us/library/cc754174.aspx
Joni
9/2/2008 2:37 PM
Gravatar Joni, thanks for the link! I'm sure others out there will appreciate it.

-Joe
Joe Pruitt
9/11/2008 3:26 PM
Gravatar Joe, Thank You! This is exactly what I needed.

I am curious, is there any advantage/disadvantage to using this in an iRule vs. an HTTP Profile (SOL4816).

Thanks,
Jason
Jason
9/11/2008 3:30 PM
Gravatar Great, Glad it helped you out. The HTTP profile will be the best performing option but there are cases where you may want to do this conditionally. If so, then you're best bet is in an iRule.

-Joe
Joe Pruitt
9/26/2008 8:44 AM
Gravatar This is a nice piece of work to fix the IIS log files. However, we need a way to trap the true client IP within our Web application. To do that, we use the Request.ServerVariables object within .NET to trap the xForward info sent from the proxie (BigIP LTM).

Here is an example:

Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs)
Response.Write(Request.ServerVariables("HTTP_X_FORWARDED_FOR"))
End Sub

I imagine other Web development platforms have access to the server variables as well.

Hope this helps.
Eddie
9/26/2008 9:01 AM
Gravatar Thanks for the tip Eddie. That header should be available to any web apps that have access to the HTTP request headers. IIS likes to prefix "HTTP_" ahead of all the HTTP header names to distinguish them from the ones that it inserts. For Apache, you'll like just access the "X-Forwarded-For" header directly.

-Joe
Joe Pruitt
 Leave Feedback
Title  
Name  
Email
Url
Comments   
Please add 3 and 3 and type the answer here: