Forum Discussion

erin_landis_211's avatar
erin_landis_211
Icon for Nimbostratus rankNimbostratus
Nov 04, 2015

Apache2.2 logging "Real" Client IP's - Mod_CloudFlare

Howdy,

 

Goal: To present the "real" client IP to the PHP handler and apache logging thread in apache2.2. I've used Mod_Cloudflare for this in the past and it worked great. We've got php code that relies on the source address being accurate and we need to log the "real" client IP for auditing purposes.

 

Environment: I've got a CentOS 6 server running Apache2.2 and PHP 5.3, with mod_Cloudflare installed. The server is a member of a pool on a F5 Big-IP 2000 series H.A. Pair. We are doing SSL termination at the F5, where the F5 decrypts, then re-encrypts traffic on the backend. The F5 is configured to inject the X-Forwarded-For header with the real client IP. SNAT is enabled on the vserver.

 

Client (10.1.20.55) -> F5 -> Server1 (10.1.10.11)

 

The F5 vserver address is 10.1.10.10 The F5 local self ips are 10.1.10.1 and 10.1.10.2 The F5 floating self IP is 10.1.10.3

 

Problem: Apache is not logging the correct IP when clients connect through the F5 vserver. PHP is also not picking up the correct IP. However, when I curl the backend node directly, mod_cloudflare works fine.

 

Relevant Config: The apache config is pretty vanilla, with minimal changes made.

 

The F5 configuration is correct, according to F5 support. There is an http profile on the vserver with the X-Forwarded-For header enabled.

 

The mod_cloudflare config is below:

 

  Configure Mod_CloudFlare

 LoadModule cloudflare_module /usr/lib64/httpd/modules/mod_cloudflare.so

 
  Trusted Proxies (LB's)
  HQ and DR
    CloudFlareRemoteIPHeader X-Forwarded-For
    CloudFlareRemoteIPTrustedProxy 10.0.0.0/8 
    CloudFlareRemoteIPTrustedProxy 10.1.10.3 10.1.10.1 10.1.10.2
     Uncomment DenyAllButCloudFlare to return a 403 status to all requests
     that do not originate from an IP defined in CloudFlareRemoteIPTrustedProxy
    DenyAllButCloudFlare
 

Troubleshooting: I can issue this curl request directly from the load balancer and apache will log the correct IP:

 

 curl --header "X-Forwarded-For: 1.2.3.4" -k https://10.1.10.11/test.txt

I understand that this comes from the local self IP, but the mod_cloudflare config is set to allow all of the relevant IPs.

 

When I browse to the Vserver, apache logs the wrong client IP (the floating IP on the F5).

 

When 10.0.0.0/8 is allowed in mod_cloudflare, I can test from other systems (with curl) on the 10.x.x.x network and apache behaves properly. As far as I can tell, the issue is with the F5 and mod_cloudflare interacting.

 

I've run a tcpdump on the backend server (with encryption off) and can confirm that the F5 is setting the X-Forwarded-For header properly.

 

I also tried moving the vserver IP to a different network, so that the Vserver and Floating IPs and backend node weren't all on the same network, no dice.

 

Plea For Help: Has anybody ran into this before? Does anybody have Apache2.2 and mod_cloudflare working behind an F5? Anything I should be trying that I haven't? I would also be interesting in knowing how someone is logging the correct IP in apache2.2 sans mod_cloudflare, if that is possible. Bear in mind that php needs to know the correct source IP as well.

 

Thanks!

 

3 Replies

  • Ok, we were able to resolve this issue with the following:

     

    1) Logging real client IP's in apache logs:

     

    Adding "%{X-Forwarded-For}i" to the LogFormat config in httpd.conf worked:

     

    LogFormat "%{X-Forwarded-For}i - %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%{X-Forwarded-For}i - %h %l %u %t \"%r\" %>s %b" common
    

    2) Passing the real client IP to php

     

    I talked this over with a developer on staff and we were able to use php's auto_prepend_file to make this happen:

     

    .htaccess file:

     

    php_value auto_prepend_file proxy.php
    

    proxy.php contents:

     

    Note: We switched out $trusted_proxy with the floating IP address of the load balancer.

     

    
    

    This works as expected.

     

    Note that this is very much a workaround (think bandaids/ducktape) until the code can be updated or a more robust solution can be found. I, for one, am hoping for a mod_f5 🙂

     

    This link provided the inspiration for that code: https://devcentral.f5.com/questions/apache22-logging-quotreal-quot-client-ips-mod_cloudflare