Forum Discussion

scott_sams_8256's avatar
scott_sams_8256
Icon for Nimbostratus rankNimbostratus
Feb 20, 2009

content tracking

we found an irule here by Kirk (thanks Kirk) that breaks content down by count. any genius out there that can take this and add total bytes for each category? our ecomm group wants to see the breakdown in the classification of traffic.

 

 

thanks!

 

 

 

 

Content_ Type_ Tracking

 

Enter a topic name to show or a new topic name to create; then press Enter

 

.

 

Contributed by: kirkbauer - kirk@f5.com

 

Description

 

 

Have you ever wondered what types of content are served by your web servers? Have you wondered how many requests could be offloaded to the BIG-IP if you added the RAM Cache module? Or maybe how much acceleration you may get from WebAccelerator ? Try out this iRule. Most of the time everything other than the HTML pages should be cacheable.

 

iRule Source

 

 

First, create statistics profile named "ContentType" with following entries:

 

HTML

 

Images

 

Scripts

 

Documents

 

Stylesheets

 

Other

 

Now associate this Statistics Profile to the virtual server. Then apply the following iRule.

 

To view the results, go to Statistics -> Profiles - Statistics

 

 

when HTTP_RESPONSE {

 

switch -glob [HTTP::header "Content-type"] {

 

"image/*" { STATS::incr "ContentType" "Images" }

 

"text/html" { STATS::incr "ContentType" "HTML" }

 

"text/css" { STATS::incr "ContentType" "Stylesheets" }

 

"*javascript" { STATS::incr "ContentType" "Scripts" }

 

"text/vbscript" { STATS::incr "ContentType" "Scripts" }

 

"application/pdf" { STATS::incr "ContentType" "Documents" }

 

"application/msword" { STATS::incr "ContentType" "Documents" }

 

"application/*powerpoint" { STATS::incr "ContentType" "Documents" }

 

"application/*excel" { STATS::incr "ContentType" "Documents" }

 

"default" { STATS::incr "ContentType" "Other" }

 

}

 

}

 

 

application delivery
dev
devops
iRules

16 Replies

Sort By
  • JRahm's avatar
    JRahm
    Icon for Admin rankAdmin
    Feb 20, 2009
    Ah, been a while for stats profile and I don't have the means to test right now, but how about under each switch match:

    STATS::incr "ContentType" "Images_bytes" [HTTP::payload length]

    or another profile:

    STATS::incr "ContentLength" "Images" [HTTP::payload length]

    So the rule would expand to: 

     
 when HTTP_RESPONSE { 
   switch -glob [HTTP::header "Content-type"] { 
     "image/*" { 
                      STATS::incr "ContentType" "Images" 
                      STATS::incr "ContentLength" "Images" [HTTP::payload length] 
                    }

    ...and so on. I assumed another profile for the content length, but you could keep in the same profile with different fields as long as you don't exceed the max. Untested, but it just might work! HTH

  • scott_sams_8256's avatar
    scott_sams_8256
    Icon for Nimbostratus rankNimbostratus
    Feb 23, 2009
    thanks. i loaded that but not knowing what it is exactly doing i cannot seem to correlate the values. guess i better read up on that.
  • scott_sams_8256's avatar
    scott_sams_8256
    Icon for Nimbostratus rankNimbostratus
    Feb 24, 2009
    have you ever seen this break a website? i have loaded it on two vips and both give broken images or no images and sometimes resets the browser connection. any ideas? did i mess something up here?

     

     

    when HTTP_RESPONSE {

     

    switch -glob [HTTP::header "Content-type"] {

     

    "image/*" { STATS::incr "ContentType" "Images"

     

    STATS::incr "ContentLength" "Images" [HTTP::payload length]

     

    }

     

    "text/html" {

     

    STATS::incr "ContentType" "HTML"

     

    STATS::incr "ContentLength" "HTML" [HTTP::payload length]

     

    }

     

    "text/css" { STATS::incr "ContentType" "Stylesheets" }

     

    "*javascript" { STATS::incr "ContentType" "Scripts" }

     

    "text/vbscript" { STATS::incr "ContentType" "Scripts" }

     

    "application/x-shockwave-flash" { STATS::incr "ContentType" "Flash" }

     

    "video/x-flv" { STATS::incr "ContentType" "FlashVideo" }

     

    "default" { STATS::incr "ContentType" "Other" }

     

    }

     

    }
  • JRahm's avatar
    JRahm
    Icon for Admin rankAdmin
    Feb 24, 2009
    Does the contentLength profile exist? Check /var/log/ltm to see if you are seeing any errors
  • scott_sams_8256's avatar
    scott_sams_8256
    Icon for Nimbostratus rankNimbostratus
    Feb 24, 2009
    thanks, i created that profile and just added images and html. in my log file i get this

     

     

    Feb 24 13:13:13 tmm tmm[955]: 01220001:3: TCL error: Rule http_stat_tracker - Error: ContentLength/HTML: bad field name or profile not attached (line 1) invoked from within "STATS::incr "ContentLength" "HTML" [HTTP::payload length] " ("text/html" arm line 2) invoked from within "switch -glob [HTTP::header "Content-type"] { "image/*" { STATS::incr "ContentType" "Images" STATS::incr "ContentLength" "Images" [HTTP::paylo..."

     

     

    not sure which profile i associate to vip. i used the origional, created the second stat profile. it works some but other times breaks my website.
  • JRahm's avatar
    JRahm
    Icon for Admin rankAdmin
    Feb 25, 2009
    Hmm, maybe it is necessary to only have a single stats profile per virtual, my bad. So in that case, I believe you will just need additional fields in the original profile, maybe call them HTML_len and Images_len: 

     
 when HTTP_RESPONSE { 
     switch -glob [HTTP::header "Content-type"] { 
       "image/*" { 
                        STATS::incr "ContentType" "Images"  
                        STATS::incr "ContentType" "Images_len" [HTTP::payload length] 
                      } 
   } 
 }

    32 fields should be enough to meet your needs.
  • scott_sams_8256's avatar
    scott_sams_8256
    Icon for Nimbostratus rankNimbostratus
    Feb 25, 2009
    yes that worked. i just did the image as a test and it worked. i need to figure out the length calculation. i have a page that had a 64k image as a background. i register a hit and it shows 1.4k in length.

     

     

    thanks!
  • scott_sams_8256's avatar
    scott_sams_8256
    Icon for Nimbostratus rankNimbostratus
    Feb 25, 2009
    so i got back to this again and decided to sniff my http connection and realized my mtu on this stream is set to 1460 bytes - 1.4k. it appears that this counts as it reads each frame. so no matter what i will always increment by 1.4k? i think i am closer but it is erratic on what it is reporting and it is far less than i would expect.
  • JRahm's avatar
    JRahm
    Icon for Admin rankAdmin
    Feb 26, 2009
    Hmm, that doesn't sound right, most likely becuase you're not doing an HTTP::collect prior (not sure you really want to collect all the HTTP payload just to count it, though). You could try using the content length header instead:

     

     

    STATS::incr "ContentType" "Images_len" [HTTP::header value "Content-Length"]
  • JRahm's avatar
    JRahm
    Icon for Admin rankAdmin
    Feb 26, 2009
    Forgot to mention that you'll want to make sure that header exists before calling it 

     
 if { [HTTP::header exists "Content-Length"] } { 
   STATS::incr "ContentType" "Images_len" [HTTP::header value "Content-Length"]  
 }

    Of course, this will lead to potentially incomplete byte counts.