Learn F5 Technologies, Get Answers & Share Community Solutions Join DevCentral

Filter by:
  • Solution
  • Technology
Clear all filters
Answers

iRule for SMTP: Passing Client IP Addr to backend mail servers

When SNATs are used for a virtual server, the backend SMTP servers cannot get the client IP address. This irule is intended to replace the string after "EHLO" or "HELO" in mail client initiation with the client's real IP address. For us, this could enable us to track down an offending mail originating device.

when CLIENT_ACCEPTED {
    set c-addr [IP::client_addr]
    log local0. "Client addr: $c-addr"
}

when CLIENT_DATA {
    STREAM::expression {@^EHLO.*\r\n@@ @^HELO.*\r\n@@}
    STREAM::enable
    event STREAM_MATCHED enable
}

when STREAM_MATCHED {
    set mstring [STREAM::match]
    log local0. "STREAM_MATCHED: string: $mstring"
    if {$mstring starts_with "EHLO"} {
        set replacment "EHLO $c-addr\r\n"
        log local0. "STREAM_MATCHED: replacement string: $replacement"
        STREAM::replace "$mstring/$replacment"
    }
    if {$mstring starts_with "HELO"} {
        set replacment "HELO $c-addr\r\n"
        log local0. "STREAM_MATCHED: replacement string: $replacement"
        STREAM::replace "$mstring/$replacment"
    }
    event STREAM_MATCHED disable
}

when SERVER_DATA {
    STREAM::disable
}

This is just an idea at this moment, and I won't be able to test the code until I find a suitable test environment for it; but for now, any comment is welcome as to if this will work at all and if yes what can be improved. Thanks.

1
Rate this Question

Answers to this Question

placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

This is pretty nice.

CLIENT_DATA and SERVER_DATA are not going to trigger on their own. You have to initiate TCP::collect in the CLIENT_ACCEPTED or SERVER_CONNECTED. That said, you don't need these events.

Also, "event STREAM_MATCHED enable" is not needed. When there is a match, this event will fire.

Simplify the code under STREAM_MATCHED for easier troubleshooting:

when STREAM_MATCHED {
    set mstring [STREAM::match]
    log local0. "STREAM_MATCHED: string: $mstring"
    set replacment [string range $mstring 0 1]
    append replacment "LO $c-addr\r\n"
    log local0. "STREAM_MATCHED: replacement string: $replacement"
    STREAM::replace "$mstring/$replacment"
}

Don't forget the generic (empty) STREAM profile to be added to the virtual. Here is a good link for more info: https://devcentral.f5.com/articles/ltm-stream-profile-multiple-replacements-regular-expressions#.U3i7UygfSok

Finally, why not test this using a mocked up SMTP server. Setup up an echo server and use telnet for a client. See if the BigIP does the replacement.

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Thanks so much, John! Here's a new version of it based on your advice:

when CLIENT_ACCEPTED {
    set c-addr [IP::client_addr]
    log local0. "Client addr: $c-addr"
    STREAM::expression {@^EHLO.*\r\n@@ @^HELO.*\r\n@@}
    STREAM::enable
}

when STREAM_MATCHED {
    set mstring [STREAM::match]
    log local0. "STREAM_MATCHED: string: $mstring"
    set replacment [string range $mstring 0 1]
    append replacment "LO $c-addr\r\n"
    log local0. "STREAM_MATCHED: replacement string: $replacement"
    STREAM::replace "$mstring/$replacment"
    event STREAM_MATCHED disable
}

when SERVER_CONNECTED {
    STREAM::disable
}

I can't test this (it compiled alright) yet because my devices are all firewall'ed off at the moment.

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

The SERVER_CONNECTED event is not needed.

One more thing. In the STREAM::replace command, only the replacement is needed.

STREAM::replace $replacment

Now test it by using a linux server behind a BigIP virtual and netcat such as in these examples.

http://stackoverflow.com/questions/8375860/echo-server-with-bash

You should be able to use telnet and type:

HELO 1.2.3.4
or
EHLO 1.2.3.4

and it echos back "xxLO [your IP address]"

0
Comments on this Answer
Comment made 18-May-2014 by Jie 2751
Thanks John. The problem I have is the VIP is currently blocked.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Just a thought... But why don't you add a received: header? That way the sending IP is available for anything you want to put in the sequence (e.g. spamassassin etc).

H

0
Comments on this Answer
Comment made 18-May-2014 by Jie 2751
What I would like to have is to add an "X-Originating-IP" header, but that would involve modifying the "DATA" section in the protocol, which will make things more complicated, even if the downstream MTAs do actually honour it at all. The "HELLO" or "EHLO" field seems to be the right place to put client's real IP addr, to replace any hostname the client puts in.
0
Comment made 18-May-2014 by Jie 2751
Not in the DATA section, but in the header section before it.
0
Comment made 19-May-2014 by Hamish 3414
It's not necessarily any more complicated... You could just INSERT a new header before the first one (i.e. prepending a new one, just like an MTA does) that adds the correctly formatted received header. Exactly doing the exact same thing as a normal MTA. Altering headers, rather than the envelope. You also have the advantage that this info is available all the way through to delivery. Wheras the info you're changing in the envelope won't necessarily be... (Headers pass to the MDA and MUA. Envelope does not). H
0
Comment made 25-May-2014 by Jie 2751
One cannot add a non-designated mail header outside the DATA section, and I do not want to modify the DATA section if I can help it, as it is a risky business there - imagine the user also adds an "x-originating-ip:" header in there... And I don't intend this to be a mail server and the info I add to the envelope seems to survive throughout the chain in our case, which is all I want.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Just a short note for now:

I will need to modify this to accommodate the sending of multiple messages in the same TCP connection.

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

To accommodate multiple messages, check out this SMTP proxy iRule example. https://devcentral.f5.com/wiki/iRules.SMTPProxy.ashx

It uses [TCP::collect] and the CLIENT_DATA event as well as [TCP::payload replace] instead of [STREAM::replace]. It offers more flexibility and allows multiple messages search/replace operations.

0
Comments on this Answer
Comment made 18-May-2014 by Jie 2751
I started with that approach while trying to log client IP addr to a remote log server, in an earlier, separate thread. The problem was that there might be just too much data to collect for an irule to handle due to resource limits. nitass suggested to use a stream profile, which got me thinking about adding the address to the EHLO/HELO field directly.
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

i think stream profile may be fine with multiple messages. anyway, i think we are having an issue about newline (\r\n) in stream expression.

0
Comments on this Answer
Comment made 19-May-2014 by Jie 2751
How do you mean? Is there a current bug about it? What's not working?
0
Comment made 19-May-2014 by nitass 13357
it seems new line does not match the stream expression (i.e. it will match whole instead of the helo/ehlo line). have you tested it?
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Take out the \r\n from the expression. The (.*) should match that as well.

STREAM::expression {@^EHLO.*@@ @^HELO.*@@}

Here is an example:

(System32) 30 % regexp {EHLO.*} "EHLO 1.1.1.1\r\n" var1
1
(System32) 31 % puts $var1
EHLO 1.1.1.1

(System32) 32 % 
0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

I have finally had a chance to actually test an SMTP irule using the stream profile, and here's my latest version, which seems to work, at least under v10.2.4:

when RULE_INIT {
    set static::smtp_debug 1
}

when CLIENT_ACCEPTED {
    set caddr [IP::client_addr]
    if { ${static::smtp_debug} } { log local0. "Client addr: $caddr" }
    STREAM::expression {@[hH][eE][lL][oO] @@ @[eE][hH][lL][oO] @@ @354 End data with @354 End data with @ @250 2.0.0 Ok: queued as @250 2.0.0 Ok: queued as @}
    STREAM::enable
    set end_data_with_seen 0
}

when STREAM_MATCHED {
    set mstring [STREAM::match]
    if { ${static::smtp_debug} } { log local0. "STREAM_MATCHED: string: \"$mstring\"" }

    if { $mstring contains "354 End data with " } {
        if { ${static::smtp_debug} } { log local0. "STREAM_MATCHED: end_data_with_seen incremented." }
        incr end_data_with_seen
        return
    } elseif { $mstring starts_with "250 2.0.0 Ok: queued as " } {
        if { ${static::smtp_debug} } { log local0. "STREAM_MATCHED: seen queued_as: closing conn." }
        TCP::close
        return
    } else {
        if { $end_data_with_seen < 1 } {
            if { ${static::smtp_debug} } { log local0. "STREAM_MATCHED: init string seen: end_data_with_seen is 0." }
            set replacement [string range $mstring 0 1]
            append replacement "LO \[$caddr\]"

            if { ${static::smtp_debug} } { log local0. "STREAM_MATCHED: replacement string: \"$replacement\"" }
            STREAM::replace $replacement
        } else {
            if { ${static::smtp_debug} } { log local0. "STREAM_MATCHED: end_data_with_seen: not 0: stream disabled." }
            STREAM::disable
        }
    }
}

when SERVER_CONNECTED {
    STREAM::expression {@[hH][eE][lL][oO] @@ @[eE][hH][lL][oO] @@ @354 End data with @354 End data with @ @250 2.0.0 Ok: queued as @250 2.0.0 Ok: queued as @}
    STREAM::enable
}

Of course it is not perfect, and I don't think it can be perfect, as it seems we have run into the limitations of the BRE library the stream profile uses, or of my understanding of how to use the stream functionality.

As a result, I have to terminate the connection after a mail message is delivered to the backend SMTP server in this irule.

Some of the text strings in the regex expressions are specific to my mail server in order to minimize the chances of unexpected/incorrect/unwanted matching.

0
Comments on this Answer
Comment made 10-May-2018 by The-messenger 361

Jie, have you looked at this with new versions of big-ip? I'm very interested.

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

I have finally decided to change to another approach: to validate client's ID (hostname or IP addr) via DNS lookup and make a deny/allow decision based on the result of the lookup. The irule is here:

https://devcentral.f5.com/questions/an-irule-to-validate-client-id-via-dns-lookup-using-the-stream-profile

0