Learn F5 Technologies, Get Answers & Share Community Solutions Join DevCentral


Questions and Answers

Loading... Loading...

Please help with a design issue that I am having.

At one of my client here we have a requirement to setup F5 LTM for load balancing a couple of .Net Remoting Application Servers. These are no different than any other web/app servers, except the traffic is all binary over http from a FAT Client.

One of the requirements from the business was to not use SNAT as this application is global, accessed from at least 6 countries and they want to be able to see traffic as is and trace end points if they need to. (when I say AS-IS, I mean without changing source ip address)

I proposed the setup as below:

Servers will have 2 NIC cards, one connected to F5 Internal VLAN and the other connected to Core Network for sys admin/monitoring/backup etc. This way application traffic is segmented. I set the Default Gateway for the F5 NIC to F5's floating IP Address and also made a NAT entry on F5 for the server to be able to talk to Database, Documentum, FileServers, Messaging etc. This application is heavy on Database and is liked to many other systems.

The issue that I am having is:
If any of the back end system, Database or Documentum is on the same VLAN as CORE NIC, its not able to route because of the same network and it tries to reach them directly from the F5 NIC without going to Default Gateway and this fails as the destination server cannot return traffic to the private F5 internal IP Address.

For now I fixed the issue by placing the servers in totally separate VLANS from all of the other systems. Now I am challenged with another issue, that is, this application makes web service calls to bunch of websites that are setup on the same server, which are also failing for the same reason as above.

Big Questions for me now is:
1) Is the design that I proposed to use 2 NIC's and both having Default Gateways, is it good/bad or ugly? From the best of my knowledge on a windows server, one can have any number of Default Gateways and the route is decided by metric and Bindings.

2) Is there any other design that any one can recommend?

3) Is anyone successful with not using SNAT and still able to use dual nic traffic segmentation?

Any help is highly appreciated and I am more than happy to give more details as required.

Thanks--Chenna

7 Answer(s):

Hi Chenna,

I would say that design is pretty ugly, unfortunately. From what I've seen, Windows boxes don't usually handle multiple default gateways well and there's no way to avoid the asymmetric routing.

I would recommend one of two things:

1) Keep the servers isolated in a VLAN behind LTM, and set up a wildcard virtual server on the LTM (0.0.0.0:0) and set it to IP forwarding, all protocols. Then have your core routers route to that isolated VLAN through the front end of the LTM (floating address). Then all sysadmin/monitoring/backup traffic will simply route through the LTM to the servers. Likewise the servers will be able to initiate outbound connections and be routed out, so you can get rid of the NAT. The only concern here is the level of that routed admin/etc traffic (especially backups). Depending on the LTM platform you have and how many servers we're talking about, it usually isn't a big problem unless you have a large number.

2) Use SNAT and use X-Forwarded-For headers in the http profile to insert the real client IP in the http request. We have an ISAPI filter for IIS here on DevCentral (Click here - link to Joe's blog...I can't find it in the CodeShare anymore ) which replaces the client IP recorded in IIS logs with the X-Forwarded-For IP. If you aren't using IIS there's probably some other mechanism to use to log the XFF header.

Hope that helps!
Denny
Denny: Appreciate your response.

For your option 1, we will not be able to segment application and sys admin traffic and also I have to route all traffic through LTM. The network team here wouldn't like that at all. Also making a route on the CORE Routers to the Floating IP will void the private vlan thing completely, as it won't be private anymore. In short, you are recommending to use a single NIC configuration instead of Dual.

For Option 2, I did explore the option of using ISAPI filters, but that only helps for IIS Logging. Any sniffing or QOSing on Layer 3 would still not be possible as the client IP Address always remains the SNATed one.

If there is no other way, then I guess I have to live with what I have but I just can't convince myself that there are no other options of achieving this. I am sure a number of other companies want to preserve source IP for various reasons.

I have a similar situation for our data center in Swiss, there I am using the nPath configuration with just one NIC and One VLAN and it works great.
Just to clarify on option 1...depending on what you mean by "segment", the sysadmin traffic will go through the wildcard vip while application traffic will traverse the normal vip, so you *can* see separate statistics and use tcpdump filters to determine what traffic is what, at least from the LTM's perspective.

Dual NIC setups without SNAT can work, there just can't be any clients to the application vip on the networks that the 2nd NIC can route to. I have customers that do backups and admin over a 2nd NIC to keep that traffic off LTM, but those functions are all that happens on those VLANS.

So tell the network team I'm sorry they don't like the way TCP/IP routing works hehe

Denny
Denny: Quick clarification on the SNAT solution:

The nature of the applications here is that, 90% of the apps are 2-tiered, meaning web servers directly talk to Database and other back end stuff without an intermediate app layer. If we use SNAT, I am wondering how the servers will be able to talk to these backend systems as the return traffic will not have a route to the servers, since they would return to the servers private IP.
Denny: Quick clarification on the SNAT solution:

The nature of the applications here is that, 90% of the apps are 2-tiered, meaning web servers directly talk to Database and other back end stuff without an intermediate app layer. If we use SNAT, I am wondering how the servers will be able to talk to these backend systems as the return traffic will not have a route to the servers, since they would return to the servers private IP.


The SNAT translation will translate the source to an address held by LTM. LTM will forward the traffic to the DB with an LTM source address, and track the connection in the SNAT table. When the response comes back from the db, it will be bound for the LTM address. When the response gets to LTM, the SNAT is reversed, and the response is forwarded to the original requester:
 
clientIP
| ^
V |
VS-IP
-------------------
| (LTM) |
-------------------
SNAT-IP
| ^
V |
serverIP


The client never sees the server's real IP, and the server never sees the client's real IP. As long as the destination device has a route back to the LTM address, traffic will flow as expected.

hth
/deb
Thanks for all your help. I'll keep exploring my options and keep you posted.
To update this thread, for the most part the design of using 2 NICs with 2 Default Gateways is working fine. Like Denny said, we cannot have clients in the same VLAN as the CORE NIC. We created a new VLAN to keep the CORE NICs and that took care of the routing issues. From Windows point of view, I bound only the CORE NIC for any Microsoft Specific Netbios, Domain and File and Print Services. Also from within IIS, we bound all of the websites to the IP Addresses on the F5 NIC, so the application traffic always originates and terminates at F5 NIC.

In spite of all of this, I still have one issue where some of the websites make web service calls to themselves and these web services URLs are using the alias that points to the VIP. Its a scenario, where a pool member is trying to access its own VIP from the same box. For now, I am working around this problem but making a hosts file entry on the pool members to point to the local F5 NIC's IP instead of going to the DNS and resolving to VIP. This seems to be working fine, except its an additional step and dependency.

Welcome comments and suggestions.

Your answer:

You must be logged in to reply. You can login here.