Collecting website statistics

OK..so I took your sugestions with one addition (see below) and I am currently monitoring in production (after I tested in stage)...we are about 1/2 as busy as we were the other day, averaging around 600 HTTP requests per sec. alertd is running at 20% and syslog-ng at 0.9 which gives me an average of just under 50% total CPU (not TMM) utilization.

 

 

Do you think the CPU utilization gows liner with the connection count? so if my traffic is 50% of what we usually see, will the CPU increase by 50%?

 

 

Here is the current rule, I had to add the [clock format] in the log statement as we could not chomp the LTM date/time stamp effeciently

 

 

 

when HTTP_REQUEST {

 

set log_format "[IP::client_addr] - - \"[HTTP::method] [HTTP::uri] \

 

HTTP/[HTTP::version]\" \"[HTTP::header value Referer]\" \"[HTTP::header value User-Agent]\""

 

}

 

when HTTP_RESPONSE {

 

log local1.info "\[[clock format [clock seconds] -format {%d/%b/%Y:%T %z} ]\] $log_format [HTTP::status] [HTTP::header value Content-Length]"

 

}

 

 

 

 

We are going to let this run for a few hours and then take it off for the weekend; it would be great if anyone could comment on my CPU to connections question.

 

 

Thanks !

 

-L