[ Teknoglot ]

I’ve been building a little SNMP Management Pack in the past few days to discover and monitor a bunch of PowerWare UPS’s, which turned out to take quite a lot more energy and time than expected. Mostly due to the facts that I am really bad with SNMP and how it works, I’ve never really looked into the inner working of building an SNMP management pack and also because we ran into a couple of errors preventing the discovery process to work alright.

To make it clear right away, this is not going to be a “Building an SNMP Management Pack Tutorial” since there’s plentiful good ones out there already, and to be extra helpful I’m gonna include a few links right away:

• SNMP Setup and Simple Custom SNMP Discovery – Pretty much the basics
• SNMP Management Pack Example: NetApp Management Pack – Part 4 actually, but has the links to the other parts
• Creating SNMP Probe Based Monitors – No custom discovery, but a good and simple guide to SNMP Probes

It’s the second, the NetApp one, I’ve used as a guide to building the UPS management pack since it goes through the process of building your own filtered discovery using SystemOID to identify your hardware-classes and then building the monitors on top of those.

Let’s get to it

When building the discovery of my hardware classes I ran into problems. The discovery simply did not work. At first I got some strange errors about “invalid queries”, something that turned out to be related to me reading two guides–seriously though, pick one guide that is closest to what you want to achieve and stick to it–and mixing up the XPathQuery variables. Silly me.
I got those errors to go away and I was able to get a few objects to my base-class, but none of the hardware classes who was populated through the return value of an SNMP OID got discovered.
The only error I got this time was the following:

Log Name:      Operations Manager
Source:        Health Service Modules
Date:          2010-09-02 11:19:12
Event ID:      11001
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      CENSORED
Description:
Error sending an SNMP GET message to IP Address XX.XX.XX.XX, Community String:=CENSORED, Status 0x6c.

One or more workflows were affected by this.

Workflow name: CENSORED.MP.CLASS.DISCOVERY
Instance name: CENSORED_DEVICENAME
Instance ID: {5C7EFB30-D885-8843-0DD7-EA86B4FD2311}
Management group: CENSORED

<IsWriteAction>false</IsWriteAction>
<IP>$Config/IP$</IP>
<CommunityString>$Config/CommunityString$</CommunityString>
<Version>1</Version>
<SnmpVarBinds>
	<SnmpVarBind>
		<OID>1.3.6.1.4.1.534.1.1.1.0</OID>
		<Syntax>0</Syntax>
		<Value VariantType="8"></Value>
	</SnmpVarBind>
	<SnmpVarBind>
		<OID>1.3.6.1.4.1.534.1.1.2.0</OID>
		<Syntax>0</Syntax>
		<Value VariantType="8"></Value>
	</SnmpVarBind>
	<SnmpVarBind>
		<OID>1.3.6.1.4.1.534.1.1.3.0</OID>
		<Syntax>0</Syntax>
		<Value VariantType="8"></Value>
	</SnmpVarBind>
</SnmpVarBinds>

After a long wait (definitely more than 90 days) the management packs for MSMQ 4 (Windows 2008) and MSMQ 5 (Windows 2008 R2) are finally released.

Both seem to be fully Cluster aware and pretty much holds the same monitoring as the the latest MSMQ 3 MP.

Message Queuing 4.0 Management Pack for Operations Manager 2007

Quick Details

Version: 6.0.6700.83

Date Published: 4/5/2010

Language: English

Download here: http://www.microsoft.com/downloads/details.aspx?FamilyID=cfc103b8-7185-4721-8098-110885fe9e9e&displaylang=en

Message Queuing 5.0 Management Pack for Operations Manager 2007

Quick Details

Version: 6.0.6700.88

Date Published: 4/5/2010

Language: English

Download here: http://www.microsoft.com/downloads/details.aspx?FamilyID=28349b78-8329-44aa-8a1f-81f4e3f84d0c&displaylang=en

Microsoft has released an update to the MSMQ (version 3) management pack.

System Center Pack for: Message Queuing 3.0
Version: 6.0.6615.0
Released on: 12/14/2009

Message Queuing (also known as MSMQ) is a server application that enables applications to communicate across heterogeneous networks and systems that may be temporarily offline or otherwise inaccessible. Instead of an application communicating with a service on another computer, it sends its information to Message Queuing, which sends the information to a Message Queuing service on the target computer where it is made available to the other application. Message Queuing provides guaranteed delivery, efficient routing, security, and priority based messaging.

Now, what’s really interesting is what you will find in the MP Guide under “Supported Configurations”.

The Message Queuing Management Pack for Operations Manager 2007 is designed to monitor Message Queuing version 3 only.

The Message Queuing Management Pack supports the following platforms:

· Windows Server 2003

· Windows XP

The Message Queuing Management Pack also supports monitoring clustered MSMQ components.

Text coloration is obviously added by me to highlight the interesting part.

Finally MSMQ monitoring seems to be cluster aware, which might mean that the home-made pack i did to have those (numerous) queues covered could be passed on to the scrap-heap. This is also confirmed under “Changes in This Update”.

The December 2009 update to this management pack includes the following change:

· Fixed a problem when working with an instance of MSMQ in a Cluster. The MP is now able to discover and monitor public and private queues in a cluster.

· Fixed a problem when discovering the local and cluster instance of MSMQ. The MP is now able to discover and monitor both instances.

The confusing double RunAs profiles seems to have been cleaned up too (you only have to worry about one now) as well as fixing some sloppy mistakes in the previous scripts (no Option Explicit? C’mon Microsoft! You write the best practices, try to stick to them.) and generally improving display and documentation.

Gonna import this to our staging environment today and let it roll during the holidays.

Cheers! Oh, and happy holidays!

Download and documentation:
http://www.microsoft.com/downloads/details.aspx?FamilyId=1D2B4398-8BC2-4A43-850C-852EBB0D983B&displaylang=en&displaylang=en

Microsoft has released an updated MP for SCCM SP2 (v6.0.6000.2, released on 10/28/2009) for OpsMgr R2.

The update basically contains support for x64 that was missin in the previous release.

The Configuration Manager 2007 SP2 Management Pack adds support for monitoring Configuration Manager 2007 SP2 in a 64-bit environment with Operations Manager 2007 R2 or Operations Manager 2007 SP1 with hotfix (KB971541) installed. This enables the Configuration Manager 2007 SP2 Management Pack to work with either the 32-bit or the 64-bit Operations Manager 2007 agent. Except for the 64-bit support, the other features and guidance for Configuration Manager 2007 Management Packs remain intact.

(coloration added by me)

Read more and download here:
http://www.microsoft.com/downloads/details.aspx?FamilyID=a8443173-46c2-4581-b3b8-ce67160f627b

This update hasn’t showed up in the MP Catalog yet, but the System Center Operations Manager 2007 R2 Cross Platform Update can be downloaded here.

Besides SUSE 11 support, here’s the short overview.

The System Center Operations Manager 2007 R2 Cross Platform Update adds fixes for a defunct process issue on Unix/Linux Servers, as well as, adds support for SUSE Linux Enterprise Server 11 (both 32-bit and 64-bit versions) and Solaris Zone support.
Feature Summary:
The System Center Operations Manager 2007 R2 Cross Platform Update supports the monitoring of Unix/Linux Servers including:

• Monitoring of SUSE Linux Enterprise Server 11 servers (both 32-bit and 64-bit versions)
• Support of Solaris Zones
• Fix for defunct Process issue
• The Cross Platform Agent may not discover soft partitions on Solaris systems. Therefore, the disk provider may be unloaded, and the Cross Platform Agent may stop collecting information from the system disks.
• The Cross Platform Agent may not restart after the AIX server reboots.

The latest versions of all the Operations Manager 2007 R2 Unix/Linux agents are included in this update.

Perfect timing, I must say, since I really need this today.

Update:
This is no small MP-update, which probably is the reason that we do not find it in the MP Catalog, but a ~250MB OpsMgr R2 Software Update. You need to run this on all Operations Manager Servers (RMS/MS, GW?) since it actually updates many of the agent Cross Platform binaries. It does add a new MP för SUSE 11 that you have to import from disk if you need it.

So, the installation goes somewhat like this:

1. Install the Software Update (pick the right Architecture) on all OpsMgr R2 Servers
2. Import the SUSE 11 MP if necessary
3. Re-discover your Unix/Linux machines.

Files updated in this update for R2:

• .\Microsoft.Enterprisemanagement.UI.Administration.dll (Version 6.1.7043.1)
• .\AgentManagement\UnixAgents\scx-1.0.4-248.aix.5.ppc.lpp.gz
• .\AgentManagement\UnixAgents\scx-1.0.4-248.aix.6.ppc.lpp.gz
• .\AgentManagement\UnixAgents\scx-1.0.4-248.hpux.11iv2.ia64.depot.Z
• .\AgentManagement\UnixAgents\scx-1.0.4-248.hpux.11iv2.parisc.depot.Z
• .\AgentManagement\UnixAgents\scx-1.0.4-248.hpux.11iv3.ia64.depot.Z
• .\AgentManagement\UnixAgents\scx-1.0.4-248.hpux.11iv3.parisc.depot.Z
• .\AgentManagement\UnixAgents\scx-1.0.4-248.rhel.4.x64.rpm
• .\AgentManagement\UnixAgents\scx-1.0.4-248.rhel.4.x86.rpm
• .\AgentManagement\UnixAgents\scx-1.0.4-248.rhel.5.x64.rpm
• .\AgentManagement\UnixAgents\scx-1.0.4-248.rhel.5.x86.rpm
• .\AgentManagement\UnixAgents\scx-1.0.4-248.sles.10.x64.rpm
• .\AgentManagement\UnixAgents\scx-1.0.4-248.sles.10.x86.rpm
• .\AgentManagement\UnixAgents\scx-1.0.4-248.sles.9.x86.rpm
• .\AgentManagement\UnixAgents\scx-1.0.4-248.solaris.10.sparc.pkg.Z
• .\AgentManagement\UnixAgents\scx-1.0.4-248.solaris.10.x86.pkg.Z
• .\AgentManagement\UnixAgents\scx-1.0.4-248.solaris.8.sparc.pkg.Z
• .\AgentManagement\UnixAgents\scx-1.0.4-248.solaris.9.sparc.pkg.Z

Files added:

• Microsoft.Linux.SLES.11.MP

All in all, the update contains the following fixes:

• KB969342
• KB973583
• Q954049
• Q956240

I’ve wrestled a bit with a critical status on one of the Organization States at a clients site that wont go back to green despite all the underlying monitors have gone back to green. And apparently I am not alone on this one. Others, like me, has read and re-read the MP-guide i search for a monitor/rule/discovery for overrides forgotten, and I don’t know how many times I’ve made a small change and tried resetting the health once again. Anyhow.
Marius Sutara posted an answer on TechNet forums last week with a “fix” (-ish), or rather the acknowledgement that the problem is not a 40c. The problem might be related to other MP as well, but I’ve only seen it on the new Exchange MP so far. In that same post, Pete Zerger provided some links to two nifty little tools that will help you reset the health of the monitor.

In case you wonder why on earth I post when there’s allready a “solution” out there; Pagerank, baby!
Not for me, but for the forum post making it show up earlier on google.

Microsoft released an updated MP (v6.1.7533.0, released on 10/8/2009) for monitoring the health the Operations Manager components.

Most significant updates, according to me, would seem to be:

Fixed an issue that was previously preventing all rules related to agentless exception monitoring from generating alerts.

Added the rule “Collects Opsmgr SDK Service\Client Connections” to collect the number of connected clients for a given management group. This data is shown in the view “Console and SDK Connection Count” under the folder “Operations Manager\Management Server Performance”.

Updated a number of monitors and rules to ensure that data is reported to the correct management group for multihomed agents.

Fixed the configuration of the rule “IIS Discovery Probe Module Execution Failure” to so that the parameter replacement will now work correctly for alert suppression and generating the details of the alert’s description.

The rest is mostly polishing, fine-tuning and complementary updates. Nothing really ground-breaking here, but still a welcome update.

Download at: http://www.microsoft.com/downloads/details.aspx?FamilyID=61365290-3c38-4004-b717-e90bb0f6c148

According to the OpsMgr Team blog, Microsoft wants to know what you think about their SQL Server MP. It’s really hard to come by a better opportunity to express your feelings and desires about monitoring SQL Server, so don’t miss this one out.

http://blogs.technet.com/momteam/archive/2009/09/25/sql-management-pack-survey-live-on-connect.aspx

Don’t know how I missed this when writing the last post, but Microsoft released the MP for Windows Server 2008 NLB yesterday (28/4 -09). This is the initial release for Win2k8 NLB so I guess we just have to try it out then.

Quick Details

Feature Summary

• Monitor the NLB Node status.
• Based on the status of individual cluster nodes, determine the overall state of the cluster.
• Where an integration management pack exists, determine the health state of a cluster node by looking at the health state of the load balanced application, such as IIS.
• Alert on errors and warnings that are reported by the NLB driver, such as an incorrectly configured NLB cluster.
• Take the node out of the NLB cluster if the underlying load-balanced application becomes unhealthy, and add the node back to the cluster when the application becomes healthy again.

Requires OpsMgr 2007 SP1 or later, the Base Operating System MP for 2008, the QFEs for Windows Server 2008 and that you are not running the converted 2003 NLB MP. If you are running the old converted NLB MP, upgrade first. As an additional recommendation, Microsoft recommends in the MP Guide that you install the QFE for wmiprvse.exe problems on Windows Server 2008.

No support for Mixed-mode (2008 and 2003) clusters though.

I have seen this error popping up every now and then at multiple customer sites and haven’t really been able to solve it yet. It does not look like I am alone either.
The error message usually looks like this:

Error doing IIS Discovery

Error: 0x80070002
Details: The system cannot find the file specified.

One or more workflows were affected by this. 

Workflow name: Microsoft.Windows.InternetInformationServices.2003.DiscoverBase
Instance name: Microsoft.Windows.InternetInformationServices.2003.ServerRole
Instance ID: {A81E4808-4D05-9BFE-4043-DC668527F2D0}
Management group: MASKED

Or…

Error doing IIS Discovery

Error: 0x80070006
Details: The handle is invalid.

One or more workflows were affected by this. 

Workflow name: Microsoft.Windows.InternetInformationServices.2000.DiscoverWebSites26to50
Instance name: IIS Web Server
Instance ID: {D36DA76A-027F-8F3E-4160-115279A1E23A}
Management group: MASKED

I have been trying to figure out what file is missing and/or if the “invalid handle” is related. Possibly a file-handle? Could be but not neccesary since these two errors occur on different servers with increasing repeat-count (atleast once-a-day). The IIS MP does call the IIS*.VBS Scripts in %windir%\System32 but as far as I can tell, on the systems I have tried it on, the scritps return valid data. This does by no means mean that there is no error and evidently I am missing something. But what? Does anyone have a clue to this?

References and other victims:

• “Error Doing IIS Discovery” on Google
• “SCOM: IIS Discovery Probe Module Failed Execution” on Codejnki’s Blog
• “Error doing IIS Discovery Error: 0×80070006″ on TechNet Forums
• “IIS Discovery Probe Module Execution Failure” on Digital Support Forum
• “IIS Discovery Probe Module Execution Failure” on EggHeadCafe
• “IISDiscoveryProbe Error – any ideas?” on Microsoft Discusson Groups

And no, neither of these provides even a hint to a working solution.