Forum Discussion

Jamie_Ostrowsk1's avatar
Jamie_Ostrowsk1
Icon for Nimbostratus rankNimbostratus
Apr 11, 2019
Solved

HTTP request data missing from my logging profile results

BIGIP LTM version 13.0.1

I'm trying to set up a logging profile for our F5's so they will send request logs to a remote syslog server.

Unfortunately even though I am able to see the packets being sent to the syslog server, they only contain the date and F5 host information, not the HTTP request data.

To set it up, I created the logging profile:

ltm profile request-log /Common/request-syslog {
    app-service none
    defaults-from /Common/request-log
    log-request-logging-errors no
    log-response-by-default yes
    log-response-logging-errors disabled
    proxy-close-on-error no
    proxy-respond-on-logging-error no
    proxy-response none
    request-log-error-pool none
    request-log-error-protocol mds-udp
    request-log-error-template none
    request-log-pool /Common/Syslog
    request-log-protocol mds-udp
    request-log-template "<160> $DATE_MON $DATE_D $TIME_HH24:$TIME_MM:$TIME_SS $BIGIP_HOSTNAME $DATE_YYYY $HTTP_METHOD $HTTP_PATH $HTTP_QUERY $HTTP_REQUEST"
    request-logging enabled
    response-log-error-pool none
    response-log-error-protocol mds-udp
    response-log-error-template none
    response-log-pool /Common/Syslog
    response-log-protocol mds-udp
    response-log-template "<160> $DATE_MON $DATE_D $TIME_HH24:$TIME_MM:$TIME_SS $BIGIP_HOSTNAME $DATE_YYYY          $HTTP_METHOD $HTTP_PATH $HTTP_QUERY $HTTP_REQUEST"
    response-logging disabled
}

When I issue a HTTP request to the virtual host I attached this logging profile to, <160> $DATE_MON $DATE_D $TIME_HH24:$TIME_MM:$TIME_SS $BIGIP_HOSTNAME $DATE_YYYY are all logged,

but $HTTP_METHOD $HTTP_PATH $HTTP_QUERY $HTTP_REQUEST are NOT included in the packet payload.

Here is a copy of the virtual host entry:

`

ltm virtual /Common/npddvaultdev1076_https {
    description "Dev Site"
    destination /Common/10.212.71.16:443
    ip-protocol tcp
    mask 255.255.255.255
    persist {
        /Common/source_addr {
            default yes
        }
    }
    pool /Common/npddvaultdev1076_http.pool
    profiles {
        /Common/request-syslog { }
        /Common/tcp { }
        /Common/wildcard.dev.foo.com {
            context clientside
        }
    }
    source 0.0.0.0/0
    source-address-translation {
        type automap
    }
    translate-address enabled
    translate-port enabled
}
  • After further testing, it turned out there was an invisible character pasted into the template that was causing the problem.

     

2 Replies

  • Hi Jamie,

     

    Your problem is due to HSL Protocol. You have a size limit you can't exced 1024 bytes using UDP protocol (I already had the same problem using ASM logs)

     

    You have to modify your "HSL Protocol" in your Request login profile: from UDP to TCP.

     

    Important: The increased size limits apply only to messages sent to remote syslog server using an iRule and egressing a TMM interface. Messages sent to a remote syslog server directly from syslog will continue to be limited to 1024 bytes.

     

    https://support.f5.com/csp/article/K8306?sr=36922430

     

    For Info:

     

    in the syslog protocol RFC Syslog have a 1KB message limit. This, and other deficiencies in the syslog protocol, is the reason why modern syslog daemons such as rsyslog support enhanced protocols with features such as TCP transport, encryption etc. There was also some effort within the IETF to standardize an improved syslog protocol, which resulted in RFC5424, RFC5425, and RFC 5426. Here, the minimum maximum message size is relatively small (depending on the transport layer), however implementations are allowed to support larger messages as well.

     

    https://tools.ietf.org/html/rfc3164

     

    Keep me in touch

     

    regards

     

  • After further testing, it turned out there was an invisible character pasted into the template that was causing the problem.