During that last year and a half that I’ve been on the F5 Management Pack team, we’ve had to do quite a bit of Systems Center Operations Manager debugging. I think it is certainly possible to develop management packs without needing to delve into Operations Manager internals at all, but much of what our pack does is outside the norm. For example:

• We have a custom-built agent on the Big-IP LTM box that our management pack talks to using a proprietary protocol
• We’ve built custom OpsMgr workflow modules in order to meet our performance objectives
• We just released a PRO pack which is fairly cutting-edge technology so there is very little documentation to go on

In this series of posts, I’ll share some of the techniques we’ve used for debugging and developing management packs. So, without further adieu, I present part I – the top 10 most useful tools.

10. Runtime Health Explorer

Troubleshooting state change problems can be tricky. There are several ways to view health and monitor states of your discovered instances, but not all in one place. This tool does just that, and has a synchronization feature as well.

9. DIR

Yes, the lowly DIR command. Simple, yet powerful.

Try running this command:

c:\Program Files\System Center Operations Manager 2007\Health Service State\>dir *.PS1 /s /b

If you have scripts included in your management pack, they will be placed in a subfolder of this directory before they are executed. Looking at these scripts can help troubleshoot selectors and script caching. A great explanation of this technique can be found here.

8. Operations Manager Tracing Configuration Scripts

By turning verbose logging on, you can see an amazing amount of detail around what OpsMgr is doing. I have found this technique most useful when debugging scripts, especially ones using the new powershell module. For example, you may have an error in your script because you forgot to add a snapin and the normal task window will not show you what the problem is. The TracingGuidsScript.log file will show the error.

7. Module Explorer

It is critical to understand what modules are available to you when developing new workflows. This tool shows you all the modules available in your current OpsMgr environment.

6. Workflow Tracer

If you have a workflow with several ConditionDetection modules and you’re losing data items somewhere, this tool is extremely useful. This module can be inserted in any of your workflows (between each module in a workflow is the best for debugging) and it will log the data items that are passing through. This is also useful for understanding what the raw data items look like.

Part II of this series will be a further exploration and extension of this concept.

5. Lineage Explorer

Much thanks to SCOMNIVORE for three of the tools on this list, including this one. This tool is similar to the Module Explorer but allows you to view class and relationship type lineage as well as module lineage.

4. Reflector

Operations Manager is not open source, but there is an enormous amount of .NET code that can be decompiled and examined. One of the posts in this series (#3 I think) will cover this topic. However, I’ll leave you with this little snippet from Microsoft.EnterpriseManagement.HealthService.DLL:

public enum ModuleType
{
    DataSource,
    Condition,
    ReadAction,
    WriteAction
}

3. SQL Server Query Analyzer

A good understanding of the OperationsManager and OperationsManagerDW databases goes a long way when troubleshooting MP issues. There are also an enormous number useful OpsMgr SQL queries available out there. This guy covers a couple dozen in just one post and our own Julian Balog just posted a couple queries for troubleshooting statistics.

2. System Center Operations Manager 2007 R2 Authoring Resource Kit (Authoring Console)

I wasn’t incredibly impressed with the original authoring console, but the just-released new version looks great. It also includes several new tools for debugging. This is the best (only?) tool that targets the beginner to intermediate level MP author. I’ve used the console several times to author a module and then save it to XML to learn about the XML schema and how things are supposed to be put together.

1. MP to XML Dumper

This simple tool is #1 because the best way to learn and debug MPs in my experience is to look through other MPs, especially the built-in OpsMgr ones. This can also be done with a powershell one-liner from the OpsMgr powershell console:

Get-ManagementPack | Export-ManagementPack –Path “C:\packs\”

Honorable Mention:

• MP Viewer: This didn’t make the list because the authoring console has all the same features and much more. The one nice thing about this tool however is that you can open a XML or MP file directly without needing to connect to OpsMgr, resolve references, etc.

If you have a great tool that is not on this list, please post a comment. I’ll make it “Top 11” if there’s something really good that I missed!

>> Part II: WorkFlows