Hi, sorry to bother you with what might be a very basic question to some of you... The thing is that even being familiar with F5 i am not a network guy (more sysadmin and webadmin) and therefore i am a bit lost... Here is my need (F5 BigIP v11.5) : We have 2 smtp relay servers (running linux). Let's consider their respectiv IP as 10.21.1.67 and 10.21.1.68. These 2 linux servers are part of a smtp pool (members : 10.21.1.67:25 and 10.21.1.68:25) used as the default pool for a F5 public Virtual Server (144.144.144.147:25). I wish these 2 linux servers would be able to relay incoming and outgoing emails to mail server which is on internet (cloud based). The BigIP have several VLANs, including one to the VLAN where those 2 servers are (10.21.1.x) and one to the public VLAN (144.144.144.x which is indeed a DMZ) . Means the BigIp also have a SelfIP on these VLANs (let's consider these self IPs are respectively 10.21.1.16 and 144.144.144.140). The default gateway for both linux servers is 10.21.1.16 (self Ip of the F5): root@myhost:~ ip route show default via 10.21.1.16 dev eth0 And i am lost bout what i should/could do : 1-add a static route ? 2-add an "IP forwarding" virtual server ? 3-need both ? 4-anything else ? I tried solution 1 like this (unsuccessfull) : Name : route_to_internet Destination : 0.0.0.0 Netmask : 0.0.0.0 Resource : Gateway Address 144.144.144.129 (the default gateway of another VLAN connected to the F5. This VLAN is indeed a DMZ) I was expecting that thanks to this route i would be able for instance to ping from my linux servers but i am not. Then i tried 2 as such : Name : IP_Forwarding_VirtualServer Partition / Path : Common Description : Type : Forwarding IP Source Address : 10.21.1.67/32 (the IP of one of my linux server, to test) Destination Address/Mask : 172.217.16.164 ("current" IP of www.google.com... i wish i could target something like 0.0.0.0/0.0.0.0 but not accepted) Service Port : all ports (for the time being, for my "ping test"). Now i am able to ping (without the need of the static route) but since i am not able to eter a destination like 0.0.0.0/0 it is not really helpful Well i am sure that reading this post you will understand that ... i am lost 😞 and need some help Thanks a lot to whoever would try

0.0.0.0/0 VS (PerfL4-type) actually is the best solution to establish internet connectivity for the back-end servers via F5 in your case. Static routes only define the next hop to reach a destination, but they do not effectively cause a routing to take place. They are useful as supplementary exceptions to default route. I'll post the config I use in 99% of cases. It consists of a custom FastL4 profile to make F5 a non-intrusive middleman (behave like a router)A pool which has either one or more default gateways to reach Internet,And a very broad 0.0.0.0/0 Virtual Server which listens on internal (server-side) VLANs that should have Internet connectivity via F5. This VS will only get a connection if there's no closer match available. conf: ltm profile fastl4 pr_fastl4_stateless { app-service none defaults-from fastL4 loose-close enabled loose-initialization enabled reset-on-timeout disabled } ltm pool pool_default_gw { members { 172.16.0.1:any { address 172.16.0.1 session monitor-enabled } } monitor gateway_icmp } ltm virtual vs_0.0.0.0_any { destination 0.0.0.0:any mask any pool pool_default_gw profiles { pr_fastl4_stateless { } } source 0.0.0.0/0 translate-port disabled vlans { vlan_server_side_2010 } vlans-enabled vs-index 4 }

Thanks a lot Hannes, i'll try to understand and then implement your solution. So far i am really stucked with some very basic question about Virtual Server of type "Forwarding (IP)" : if the destination of my VS is the IP of (any port) the ping to works. But if the destination is 0.0.0.0/0 (any port) ping does not work...and i really don't understand why. "0.0.0.0/0" should englobe any IPv4, am i wrong ? Thanks so much if you can again help.

It would be possible due to a second Virtual Server, a better candidate to handle your connection to ''. To throw out some stupid speculation, maybe you have another 0.0.0.0/0 VS for a specific protocol which has a different server-side configuration, or maybe you have a 87.252.198.0/24 VS (matches public address pool). See order of precedence for Virtual Server matching: https://support.f5.com/csp/article/K14800

static route versus IP forwarding VS

17 Replies

Hannes_Rapp_162
Nacreous
May 29, 2017
0.0.0.0/0 VS (PerfL4-type) actually is the best solution to establish internet connectivity for the back-end servers via F5 in your case. Static routes only define the next hop to reach a destination, but they do not effectively cause a routing to take place. They are useful as supplementary exceptions to default route.

I'll post the config I use in 99% of cases.

It consists of a custom FastL4 profile to make F5 a non-intrusive middleman (behave like a router)
A pool which has either one or more default gateways to reach Internet,
And a very broad 0.0.0.0/0 Virtual Server which listens on internal (server-side) VLANs that should have Internet connectivity via F5. This VS will only get a connection if there's no closer match available.

conf:

ltm profile fastl4 pr_fastl4_stateless { app-service none defaults-from fastL4 loose-close enabled loose-initialization enabled reset-on-timeout disabled } ltm pool pool_default_gw { members { 172.16.0.1:any { address 172.16.0.1 session monitor-enabled } } monitor gateway_icmp } ltm virtual vs_0.0.0.0_any { destination 0.0.0.0:any mask any pool pool_default_gw profiles { pr_fastl4_stateless { } } source 0.0.0.0/0 translate-port disabled vlans { vlan_server_side_2010 } vlans-enabled vs-index 4 }
- Stéphane_PICARD
  Nimbostratus
  May 29, 2017
  Thanks a lot Hannes, i'll try to understand and then implement your solution. So far i am really stucked with some very basic question about Virtual Server of type "Forwarding (IP)" : if the destination of my VS is the IP of (any port) the ping to works. But if the destination is 0.0.0.0/0 (any port) ping does not work...and i really don't understand why. "0.0.0.0/0" should englobe any IPv4, am i wrong ?
  
  Thanks so much if you can again help.
- Hannes_Rapp_162
  Nacreous
  May 29, 2017
  It would be possible due to a second Virtual Server, a better candidate to handle your connection to ''. To throw out some stupid speculation, maybe you have another 0.0.0.0/0 VS for a specific protocol which has a different server-side configuration, or maybe you have a 87.252.198.0/24 VS (matches public address pool).
  
  See order of precedence for Virtual Server matching: https://support.f5.com/csp/article/K14800
Hannes_Rapp
Nimbostratus
May 29, 2017
0.0.0.0/0 VS (PerfL4-type) actually is the best solution to establish internet connectivity for the back-end servers via F5 in your case. Static routes only define the next hop to reach a destination, but they do not effectively cause a routing to take place. They are useful as supplementary exceptions to default route.

I'll post the config I use in 99% of cases.

It consists of a custom FastL4 profile to make F5 a non-intrusive middleman (behave like a router)
A pool which has either one or more default gateways to reach Internet,
And a very broad 0.0.0.0/0 Virtual Server which listens on internal (server-side) VLANs that should have Internet connectivity via F5. This VS will only get a connection if there's no closer match available.

conf:

ltm profile fastl4 pr_fastl4_stateless { app-service none defaults-from fastL4 loose-close enabled loose-initialization enabled reset-on-timeout disabled } ltm pool pool_default_gw { members { 172.16.0.1:any { address 172.16.0.1 session monitor-enabled } } monitor gateway_icmp } ltm virtual vs_0.0.0.0_any { destination 0.0.0.0:any mask any pool pool_default_gw profiles { pr_fastl4_stateless { } } source 0.0.0.0/0 translate-port disabled vlans { vlan_server_side_2010 } vlans-enabled vs-index 4 }
- Stéphane_PICARD
  Nimbostratus
  May 29, 2017
  Thanks a lot Hannes, i'll try to understand and then implement your solution. So far i am really stucked with some very basic question about Virtual Server of type "Forwarding (IP)" : if the destination of my VS is the IP of (any port) the ping to works. But if the destination is 0.0.0.0/0 (any port) ping does not work...and i really don't understand why. "0.0.0.0/0" should englobe any IPv4, am i wrong ?
  
  Thanks so much if you can again help.
- Hannes_Rapp
  Nimbostratus
  May 29, 2017
  It would be possible due to a second Virtual Server, a better candidate to handle your connection to ''. To throw out some stupid speculation, maybe you have another 0.0.0.0/0 VS for a specific protocol which has a different server-side configuration, or maybe you have a 87.252.198.0/24 VS (matches public address pool).
  
  See order of precedence for Virtual Server matching: https://support.f5.com/csp/article/K14800
dragonflymr
Cirrostratus
May 29, 2017
Hi,

First of all check stats of your Forwarding (IP) server when doing ping - if traffic is intercepted by this VS you will see packets reported, if you can't see packets then some other object is catching your traffic.

Of course it would be better to do tcpdump on interface facing your Linux servers. Something like that:

tcpdump -nni [your interface or vlan name facing Linux servers]:nnn -s0 'host [ip of Linux from which ping is started] and icmp'

In trace you should see BIG-IP listener like that lis=/Common/your_listener_name

If you want your wildcard VS (either Performance L4 or Forwarding IP) to only catch SMTP traffic then you can set Service Port to 25 (or port actually used by your servers to connect).

Considering VS config posted by Hannes I would say most important omission is that along with translate-port disabled you should have translate-address disabled.

Second would be that SNAT is not enabled. You need at least SNAT Automap to have replies from destination servers reach BIG-IP - of course assuming that you will not have route to 10.21.1.0/24 network configured on devices between BIG-IP and Internet to point to BIG-IP interface in 144.144.144.0/24 network.

I would say that setting fastL4 profile to loose-close and loose-initialization is not necessary and a bit dangerous.

We are talking here about connections initiated by servers to Internet, so they always should start with proper 3 Way Handshake.

As all traffic will be flowing via VS (not like with Direct Server Return setup) so BIG-IP will see as well all TCP close attempts (FIN packets).

Enabling loose-initialization allows bad formed connections or malicious packets to reach Internet - just my opinion.

And if loose options are not used, then it's better to enable reset-on-timeout.

Sure if you don't care what is leaving your network you can keep suggested config.

Piotr

Stéphane_PICARD

Nimbostratus

May 30, 2017

Thanks to all of you trying to help. I am reading carefully your posts and try to extract what i can understand and keeps on testing...Very very sorry to keep on being a bit lost and therefore bothering you...

-disabled all my virtual servers BUT the IP forwarding one (it is a test F5 device) to be sure some traffic cannot be catched by another VS

-reset the statistics on this IP forwarding VS

-tcpdump on the F5 in front of my linux servers

Case 1 : the IP forwarding VS is targetting 0.0.0.0/0

ltm virtual IP_Forwarding_any {
    destination 0.0.0.0:any
    ip-forward
    mask any
    profiles {
        fastL4 { }
    }
    source 10.21.1.67/32
    source-address-translation {
        type automap
    }
    translate-address disabled
    translate-port disabled
    vs-index 28
}

Ping does not work :

root@chgva-srv-smt02:~ ping www.google.fr
PING www.google.fr (216.58.198.35) 56(84) bytes of data.
From 10.21.1.18 icmp_seq=1 Destination Net Unreachable
From 10.21.1.18 icmp_seq=2 Destination Net Unreachable

N.B. : 10.21.1.18 is the floating self IP of my F5 device.

On tcpdump on my F5 (listening on all interfaces) i can see the ICMP requests to mil04s04 (which is probably google), but part of them only (seems to be 1 out of 2, why ????) going through the IP forwarding VS :

[admin@f503:Active] ~  tcpdump -i 0.0 -vv host 10.21.1.67|grep ICMP
tcpdump: listening on 0.0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:48:01.882920 IP (tos 0x0, ttl 64, id 53093, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f3.1e100.net: ICMP echo request, id 51313, seq 2234, length 64 in slot1/tmm0 lis=/Common/IP_Forwarding_any
11:48:01.882941 IP (tos 0x0, ttl 255, id 12459, offset 0, flags [DF], proto ICMP (1), length 56)
    10.21.1.18 > 10.21.1.67: ICMP net mil04s04-in-f3.1e100.net unreachable, length 36
        IP (tos 0x0, ttl 63, id 53093, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f3.1e100.net: ICMP echo request, id 51313, seq 2234, length 64 out slot1/tmm0 lis=
11:48:02.882863 IP (tos 0x0, ttl 64, id 53243, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f3.1e100.net: ICMP echo request, id 51313, seq 2235, length 64 in slot1/tmm0 lis=/Common/IP_Forwarding_any
11:48:02.882877 IP (tos 0x0, ttl 255, id 12465, offset 0, flags [DF], proto ICMP (1), length 56)
    10.21.1.18 > 10.21.1.67: ICMP net mil04s04-in-f3.1e100.net unreachable, length 36
        IP (tos 0x0, ttl 63, id 53243, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f3.1e100.net: ICMP echo request, id 51313, seq 2235, length 64 out slot1/tmm0 lis=
11:48:03.882808 IP (tos 0x0, ttl 64, id 53354, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f3.1e100.net: ICMP echo request, id 51313, seq 2236, length 64 in slot1/tmm0 lis=/Common/IP_Forwarding_any
11:48:03.882824 IP (tos 0x0, ttl 255, id 12471, offset 0, flags [DF], proto ICMP (1), length 56)

The VS statistics show that traffic IN but no OUT

Seems that the ICMP requests reach the F5 but then the F5 have no route to internet and herefore does not know what to do (hence the traffic out of VS = 0 ?)

In such case i would need a static route ? I created one as such :

Name    Internet
Partition / Path    Common
Description 
Destination 0.0.0.0
Netmask 0.0.0.0
Resource    Gateway Address   144.144.144.129

Now ping does not give any message anymore (at least i don't have the "Destination Net unreachable" anymore). tcpdump show that only the ICMP requests arrive (now displaying the /Common/IP_Forwarding_any VS name for each and every ICMP request, which was not the case before) :

[admin@f5g03:Active] ~  tcpdump -i 0.0 -vv host 10.21.1.67|grep ICMP
tcpdump: listening on 0.0, link-type EN10MB (Ethernet), capture size 65535 bytes
12:03:09.801031 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 40)
    10.21.1.18 > 10.21.1.67: ICMP echo request, id 29803, seq 61240, length 20 out slot1/tmm0 lis=
12:03:09.801351 IP (tos 0x0, ttl 64, id 5304, offset 0, flags [none], proto ICMP (1), length 40)
    10.21.1.67 > 10.21.1.18: ICMP echo reply, id 29803, seq 61240, length 20 in slot1/tmm0 lis=
12:03:09.869465 IP (tos 0x0, ttl 64, id 35972, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f35.1e100.net: ICMP echo request, id 51313, seq 3142, length 64 in slot1/tmm0 lis=/Common/IP_Forwarding_any
12:03:10.869492 IP (tos 0x0, ttl 64, id 36196, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f35.1e100.net: ICMP echo request, id 51313, seq 3143, length 64 in slot1/tmm0 lis=/Common/IP_Forwarding_any
12:03:11.869374 IP (tos 0x0, ttl 64, id 36369, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f35.1e100.net: ICMP echo request, id 51313, seq 3144, length 64 in slot1/tmm0 lis=/Common/IP_Forwarding_any
12:03:12.869384 IP (tos 0x0, ttl 64, id 36374, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f35.1e100.net: ICMP echo request, id 51313, seq 3145, length 64 in slot1/tmm0 lis=/Common/IP_Forwarding_any
12:03:13.869423 IP (tos 0x0, ttl 64, id 36616, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f35.1e100.net: ICMP echo request, id 51313, seq 3146, length 64 in slot1/tmm0 lis=/Common/IP_Forwarding_any

Statistics of the VS again shows traffic IN but not OUT. Here i am sure it goes through my IP forwarding VS but i don't receive the answer means i need to further investigate on the FW/gateway side i guess. Now let's have a look at case 2 ....

Case 2 : the IP forwarding VS have the destination IP :

ltm virtual IP_Forwarding_any {
    destination 216.58.198.35:any
    ip-forward
    mask 255.255.255.255
    profiles {
        fastL4 { }
    }
    source 10.21.1.67/32
    source-address-translation {
        type automap
    }
    translate-address disabled
    translate-port disabled
    vs-index 28
}

Ping works fine :

root@chgva-srv-smt02:~ ping www.google.fr
PING www.google.fr (216.58.198.35) 56(84) bytes of data.
64 bytes from mil04s04-in-f35.1e100.net (216.58.198.35): icmp_seq=1 ttl=255 time=0.172 ms
64 bytes from mil04s04-in-f35.1e100.net (216.58.198.35): icmp_seq=2 ttl=255 time=0.356 ms

tcpdump looks fine :

[admin@f5g03:Active] ~  tcpdump -i DMZ-intern -vv host 10.21.1.67|grep ICMP
tcpdump: listening on DMZ-intern, link-type EN10MB (Ethernet), capture size 65535 bytes
11:10:58.907658 IP (tos 0x0, ttl 64, id 36299, offset 0, flags [DF], proto ICMP (1), length 84)
    10.21.1.67 > mil04s04-in-f3.1e100.net: ICMP echo request, id 51313, seq 11, length 64 in slot1/tmm0 lis=
11:10:58.907691 IP (tos 0x0, ttl 255, id 1298, offset 0, flags [DF], proto ICMP (1), length 84)
    mil04s04-in-f3.1e100.net > 10.21.1.67: ICMP echo reply, id 51313, seq 11, length 64 out slot1/tmm0 lis=

Very surprisingly to me :

1) the VS statistics remains stucked to 0 2) ping works fine (without any route)

Would means the traffic goes some other way without my IP forward VS but what is more than strange is that the ping starts working fine as soon as i set the proper google destination IP is this same IP forward VS ???????

More than thanks if one of you keeps on trying to help me

dragonflymr
Cirrostratus
May 30, 2017
Hi,

When you receive Net Unreachable then of course cause is lack of routing entry matchin received packet. To use Forwarding (IP) you have to have either static or dynamic routes.

In the simplest setup at lest Default Route configured via Routes.

Now situation when you did that is some kind of error, trace is showing ICMP requests send from BIG-IP self IP to your Linux host:

10.21.1.18 > 10.21.1.67: ICMP echo request

That for sure will not work, opposite direction should work - as your first tcpdump where:

10.21.1.67 > mil04s04-in-f3.1e100.net: ICMP echo request

I would suggest using

tcpdump -nni [internal vlan name]:n -e 'icmp and host 10.21.1.67'

and

tcpdump -nni [external vlan name]:n -e 'icmp and host [self ip used as src packet - set by SNAT automap'

Just open two terminal to BIG-IP and run both at the same time, then send ping from your Linux host

Piotr

BTW: What do you mean by "disabled all my virtual servers"? Just setting Disabled in VS config? If so you are not in fact disabling precedence rules used by BIG-IP.

By default setting is that if there is VS that best matches traffic and this VS is disabled, less matching VS will not be used.

Something like that:

VS1 (disabled) 10.10.10.1:443
VS2 (enabled) 0.0.0.0:443

VS2 will not be used even if traffic is directed to port 443.

If you wan't to really disable VS use this method:

Choose Enabled On but leave Selected list empty

or

Choose Disabled On and in Selected place VLAN from which you are sending pings

or

Create tunnel object of type tcp-forward, let's say blackhole. Then assign it to Selected panel for Enabled On
dragonflymr
Cirrostratus
May 30, 2017
Sorry, I was looking at first pings in the trace, latter on there are pings from Linux to google via (probably) BIG-IP self IP.

Assuming that your VS has All Protocols set and BIG-IP receiving ping then some strange situation there is.

Try to run:

tcpdump -nni 0.0:np -s0 -e host 10.21.1.67

If possible open another terminal windows to big IP and launch this commands:

watch -n 0.1 tmsh show sys connection cs-client-addr 10.21.1.67

watch -n 0.1 tmsh show sys connection ss-client-addr [ip used by automap]

Piotr
dragonflymr
Cirrostratus
May 31, 2017
Some more ideas for test:

Can you ping google from F5? If so which VLAN is used for outgoing requests, and did those request go back via the same VLAN.

Can you check routing using ip route get [google IP - 216.58.198.35]

What about other type of connections from Linux - can you reach any site in the Internet - for example using curl -k https://www.google.fr -v

If not what is in the trace on F5, did F5 send RST packets to Linux? If so you can try:

tmsh reset-stats net rst-cause

then

watch -n 1 tmsh show net rst-cause

Repeat connection using curl and look what reasons for reset are listed by previous command.

To be honest it's very basic setup I did hundreds of times before and never had issue. Very mysterious.

If you can, as last resort reboot F5 - to be honest more than once I have situation when something did not work but should - magically after reboot it started - especially on VE.

Piotr

Forum Discussion

static route versus IP forwarding VS

17 Replies

Recent Discussions

google-visualization-issues

Advise on setting up IRULE

google autenticator broken barcode

Port Group on F5OS network setting

vulnerabilities (CVE-2020-36363), CVE-2016-0736,CVE-2019-6593-FOR VERSION BIGIP LTM-16.1.4

Related Content

Load Balancing versus Application Routing

iRules Recipe 4: Static Maintenance Page

Clustering versus load-balancing

Static Route

Microservices versus Microsegmentation