Filter by:
  • Solution
  • Technology

answers

High memory usage from TMM

Updated 11/22/2010 • Originally posted on 22-Nov-2010 by rjordan 1

Hey Everyone,
I've been experiencing some weird issues on my LTM HA cluster. Whenever I make a config change through Web Admin or CLI (using b load), it see a hundred or so monitors flap up and down. The flapping only occurs on my active unit. If I make similar changes on the standby unit, I don't see any flapping. I've also failed over to the secondary unit and confirmed that it has the same issue. Increasing the timeouts to around 4 or 5 times the interval seems help, but I don't want to wait for 4-5 failures before a member is marked down.
I don't know when this issue started so I can't really associate it with a specific config change. We use our LTMs as a shared appliance to provide load balancing and redundancy to our enterprise customers. We don't do much with iRules other than pointing a hostname or URI to a specific member.
So far, I've done the following:
1. full_box_reboot on each unit
2. ran EUD - both came back clean
3. upgraded from 9.4.6 to 9.4.8 HF4

The host memory is pretty much maxed out at any given time on our active and standby units, so I don't think it is a bad iRule. I see the following process (ps -aux) on both units using ~40% CPU and ~80% memory:
/bin/tmm -m -s 1556 --command=rtc_frequency 4096

Has anyone had similar problems? Does anyone know what that process is?
0
Rate this Question

Answers to this Question

5 Answers:

placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
Updated 22-Nov-2010 • Originally posted on 22-Nov-2010 by hoolio 2066
TMM is the Traffic Management Microkernel and is responsible for handling load balanced traffic. See SOL8035 for details on TMM and other system daemons.

SOL8035: Overview of BIG-IP daemons
http://support.f5.com/kb/en-us/solutions/public/8000/000/sol8035.html

I suggest opening a case with F5 Support on the resource utilization issues. They'll be able to review your full configuration and recent logs to help troubleshoot this.

Aaron
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
Updated 23-Nov-2010 • Originally posted on 23-Nov-2010 by rjordan 1
Thanks for the suggestion. I opened the case a week ago, but progress has been very slow. I just wanted to see if anyone else has experienced similar issues. I'll be sure to provide an update as soon as I find a fix.
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
Updated 05-Dec-2010 • Originally posted on 05-Dec-2010 by L4L7 59
To add a bit: because TMM is a microkernel it'll occupy CPU - it's got real time priority (you'll see it as RT in the priority column in top). As a result it'll sit on idle cycles or has first right of refusal at the very least.I've not spent cozy time with 9.x for a good long while, but I do remember that TMM typically ran hot. TMM sitting on 40% CPU in 9.x doesn't sound alarming to me, for what it's worth. On a 10.x system it's common to see it sitting at around 8-11% utilization per core.

Despite all that, Aaron's advice is good, as usual - I'd open a case here too just to be sure.

-Matt
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
Updated 13-Feb-2012 • Originally posted on 13-Feb-2012 by Josh Becigneul 916
I hate to necromancer a thread like this, but I'm curious if there ever was a resolution to this issue. We're on 9.4.8 HF4 and experiencing similar issues. A case is open with F5 but it's progressing slowly.

Thanks!
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER
Updated 15-Feb-2012 • Originally posted on 15-Feb-2012 by nitass 12650
as Aaron suggested, please open a support case if you have not yet done.

additonally, i think providing data below could be helpful for support team.
1. generate qkview
2. turn on bigd debug
3. start tcpdump
4. reproduce issue i.e. change config
5. stop tcpdump
6. turn off bigd debug
7. generate qkview again

Troubleshooting Ltm Monitors by Aaron
http://devcentral.f5.com/wiki/AdvDesignConfig.TroubleshootingLtmMonitors.ashx
;