Forum Discussion

Jace_45978's avatar
Jace_45978
Icon for Nimbostratus rankNimbostratus
Nov 02, 2009

v10.0.1 cpu0 goes off on its own

6900 LTM running v10.0.1 HF2 4 core unit

 

 

LTM shows that the CPUs are all balanced during the day normally around 40% (during peek time traffic) we are doing ssl termination and hardware compression.

 

 

There was a need to add the following iRule:

 

 

rule XXComRestrictVHostsProd {

 

when HTTP_REQUEST priority 50 {

 

if {[HTTP::header exists Host]} {

 

set hostHeader [HTTP::header Host]

 

if { $hostHeader equals {admin.xx.com} or $hostHeader equals {tool.xx.com} } {

 

log local0. "Potential header manipulation detected. Received request for $hostHeader from [IP::client_addr]"

 

HTTP::header replace Host www.xx.com

 

}

 

}

 

}

 

}

 

 

When this rule was applied to the VS it was originally applied with the log local0 un-commented and the cpu0 would run a 80% during mid traffic time while the other 3 cores would be 10%...during peek time the cpu0 would max at 100% and cause the site to be extremely slow. Commenting the log local0 causes the cpu0 and the other cpus to line up and have a nice balance around 40% during peek time.

 

 

We do have a ticket open but was curious if anyone has any insight to why cpu0 would max out while using the above iRule and the log function. Even with the log function enabled the /var/log/ltm was not getting much if any data from this irule unless we generated the known condition.

 

normal operation of this LTM is over 100k connections, 250ssl_tps, 600 new connects/sec,

 

thanks

 

application delivery
dev
devops
iRules

4 Replies

Sort By
  • The_Bhattman's avatar
    The_Bhattman
    Icon for Nimbostratus rankNimbostratus
    Nov 02, 2009
    Run the following command

     

     

    top -p `pidof tmm` -p `pidof bigd`

     

     

    To see what process is running high

     

     

    CB
  • hooleylist's avatar
    hooleylist
    Icon for Cirrostratus rankCirrostratus
    Nov 02, 2009
    80% CPU0 and 10% on the other cores sounds like the issue might be that the VIP was demoted from CMP (using all available cores - Click here). Was there a global variable in the iRule--or was it literally what you posted? I don't think anything in the rule you posted would prevent CMP from working. Unless maybe alertd and/or syslog-ng are only running on CPU0 and that's what was using the CPU cycles on CPU0 while logging was enabled.

    You could enable timing on the iRule (Click here) and compare the results with and without the log statement.

    Also, you could use a switch statement for more efficient evaluation of the host header value: 

     
 when HTTP_REQUEST priority 50 { 
    if {[HTTP::header exists Host]} { 
       switch [string tolower [HTTP::host]] { 
          "admin.xx.com" -  
          "tool.xx.com" { 
              log local0. "Potential header manipulation detected. Request for [HTTP::host] from [IP::client_addr]" 
             HTTP::header replace Host www.xx.com 
          } 
       } 
    } 
 }

    Aaron
  • Jace_45978's avatar
    Jace_45978
    Icon for Nimbostratus rankNimbostratus
    Nov 02, 2009
    the irule that I posted was literally the irule (exception of xx in place of client initials).

     

    the VS:

     

    b virtual atgip_xx.com cmp mode show

     

    VIRTUAL atgip_xx.com - CMP mode: all

     

     

    The only change that was made to watch cpu0 drop from 80% and the other 3 cores rise from 10% to meet in the 40% range was modifying the rule above from log local0. to log local0.

     

     

    If the F5 case comes back with something I will post it. thanks all.

     

  • hooleylist's avatar
    hooleylist
    Icon for Cirrostratus rankCirrostratus
    Nov 02, 2009
    So CMP shouldn't be an issue with that iRule. I'd guess it's a host daemon like alertd or syslog-ng eating up the CPU0 cycles. You could enable logging again during a low traffic period and check the top and/or ps output to see what's using CPU0.

     

     

    Aaron