Forum Discussion

Greg_33558's avatar
Greg_33558
Icon for Nimbostratus rankNimbostratus
Feb 09, 2015

Where to find raw HTTP Request for ASM violations?

Where can I find the raw HTTP Request that is logged with ASM violations?

 

The version that is printed to the browser in Security->Event Logs->Application->Requests appears to be inserting a Replacement Character at the location of (presumably) a multibyte UTF-8 character. If I cut and paste the text from the browser, it contains a UTF-8 Left-to-Right Embedding character and the Replacement Character (with words in between). But I'm guessing that Replacement Character is not what the client actually sent, just what I'm ending up with after it's been processed by the web stack.

 

I have explored the underlying MySQL database a little and the violation itself is logged in PRX.REQUEST_LOG, but the HTTP Request that gets shown when I browse in the web UI is not in that table, nor do I see it in any other table in PRX. It's in the database somewhere, because I can string match it in ibdata1.

 

Does anyone know which table stores the HTTP Request raw data for ASM violations?

 

Is there any documentation for the schema of the databases in MySQL on the F5 (DCC, PLC, PRX, WIZ)?

 

ASM Advanced WAF
mysql
security

6 Replies

Sort By
  • Amit_Karnik's avatar
    Amit_Karnik
    Icon for Nimbostratus rankNimbostratus
    Feb 09, 2015

    For what it is worth....

     

    I routinely use the export to export the violation data into Excel. You can then add some Excel formulas to mine the data off the excel sheet into additional columns.

     

    It is not straightforward but most of the times it works for us with a little tweaking.

     

    Please post to this forum if you do find a good way to analyze the data.

     

    Best.

     

  • gowenfawr's avatar
    gowenfawr
    Icon for Nimbostratus rankNimbostratus
    Feb 11, 2015

    I have found the raw HTTP request in a file on the F5:

     

    /var/adm/request_log/0000000001

     

    This file appears to be a binary format which contains the raw requests and various other recognizable information about the request. For example, among a lot of binary character mush I can see 'no_ext', the URI, and the IP address of the virtual involved as well as the raw HTTP request. Clearly this file contains info about multiple requests, one after the other, with associated data.

     

    It was possible to pull out the raw request and find what I suspected was true - the request contained a 0xe2 byte, which is a valid UTF-8 multibyte prefix, but without valid multibyte data following it (in fact, the next characters are the '

     

    I would love to see tools or documentation for dissecting this file, but for my purposes it was sufficient to go in and search for the string I needed to extract and write out the lines of the HTTP request.

     

  • Martin_Vlasko's avatar
    Martin_Vlasko
    Icon for Altocumulus rankAltocumulus
    Oct 09, 2018

    Hi Greg,

     

    I found a solution for myself and I believe it could help you too. If you are a friend with Python, you can expand existing version of f5-common-python SDK to provide access to raw Requests data.

     

    I have posted my solution on project forum:

     

    https://github.com/F5Networks/f5-common-python/issues/1486

     

    The example I mentioned there is for Violations. But you can apply very similar code to fetch Requests as well.

     

    I have used this SDK to write a script to export ASM learning suggestions to an Excel spreadsheet with almost all information which are normally available via GUI.

     

  • Martin_Vlasko_1's avatar
    Martin_Vlasko_1
    Icon for Altostratus rankAltostratus
    Oct 09, 2018

    Hi Greg,

     

    I found a solution for myself and I believe it could help you too. If you are a friend with Python, you can expand existing version of f5-common-python SDK to provide access to raw Requests data.

     

    I have posted my solution on project forum:

     

    https://github.com/F5Networks/f5-common-python/issues/1486

     

    The example I mentioned there is for Violations. But you can apply very similar code to fetch Requests as well.

     

    I have used this SDK to write a script to export ASM learning suggestions to an Excel spreadsheet with almost all information which are normally available via GUI.