Hardware Monitoring Using System Center Operations Manager

Hardware management is the one thing I am most worried about.  Sure, I could deploy the manufacturers management solution.  But do I want consoles to manage lots of different systems?  Really, you don’t.  You want one central point and that can be the Operations Manager console.

I’m most familiar with what HP does so I’ll explain it.  They provide and Insight Manager agent that detects health and performance issues of the hardware.  This includes all of the components, e.g. CPU, fans, disks, network cards, etc.  You can deploy and OpsMgr agent to this server.  If you install the HP Insight Manager management pack then, after discovery, OpsMgr will be aware of the Insight Manager agent.  All data collected by that agent will be detected by OpsMgr.  So now, if a disk fails you learn about it in OpsMgr.  If memory degrades, you learn about it in OpsMgr.  This is so handy – because this is where you also get performance and health alerts for Windows, SQL, Exchange, Red Hat Enterprise Linux, SUSE Linux Enterprise Server, etc.  You can extend with 3rd party solutions to include your Cisco network, etc.  Heck, there’s even a coffee pot management pack!!!

Back in the day, there appeared to be only support from HP and Dell.  But that has changed.

  • HP: Hewlett Packard has management packs for ProLiant servers, BladeSystem, and Integrity.  There is also a management pack for StorageWorks systems (e.g. EVA SAN).
  • Dell: I’ve never managed Dell machines with OpsMgr.  But I am told that Dell did a very nice job.  They are significant Microsoft partners.
  • IBM: I’m not the biggest fan of IBM – we have some X series stuff which I detest.  We had to get a IBM employee to download the management pack because all external links failed.  At the time, it appeared their “shared” download was only available from the IBM corporate network. A Dutch friend had the same issue and I ended up sending him what I was given by IBM.  I’ll be honest, the IBM Director management pack is poor compared to the HP one.  IBM wants you to spend lots of money on consultancy led Tivoli.  IBM Director is pretty poor too.  IBM Ireland employees have been unable to figure out how to monitor IBM DAS nor give me the documentation to do it.
  • Fujitsu: I have not seen a Fujitsu server since 2005.  Back then there was no MOM management pack for the Fujitsu Siemens servers; they wanted you to use a native solution only.  That has changed.  They have ServerView Integration for Microsoft System Center Operations Manager 2007 and System Center Essentials 2007 and ServerView Integration Pack for Microsoft System Center Operations Manager 2007.

That should get you started.  Each of the manufacturers seems to do things differently.  HP, for example, uses the above system for ProLaints.  But blade enclosures require a piece of middleware.  Make sure you read the accompanying documentation from the OEM before you do anything.

Thanks to fellow MVP Mark Wilson for finding the links for Fujitsu. 

Technorati Tags: ,

Operations Manager 2007 R2 Cross Platforms Cumulative Update 2

Microsoft released an update for OpsMgr 2007 R2 cross platform extensions last night. 

The System Center Operations Manager 2007 R2 Cross Platform Cumulative Update 2 includes System Center Operations Manager 2007 R2 Cross Platform Agent Update (KB973583) and additional bug fixes.

This updated release includes all features that were in the previous update release (KB973583) and additional fixes in this release:

Adds support for (in previous release – KB973583):

  • SUSE Linux Enterprise Server 11 (both 32-bit and 64-bit)
  • Zones (Whole and Sparse Zones) for all supported version of Solaris

There are a number of fixes included, all available to read on the MS download page.

New Operations Management Management Pack: Power

There’s a glut of Operations Manager 2007 management packs available from Microsoft.  One of them really stood out.  It is the Windows Power Management Pack for System Center Operations Manager 2007 R2.

The short description is:

“The Power Management Pack for Operations Manager 2007 R2 enables you to monitor and manage the power consumption of computers running Windows Server 2008 R2.

This management pack provides:

  • Visibility into power consumption
  • Visibility and control of power policy
  • Ability to lower power consumption during non-business hours to reduce overall power consumption
  • Ability to limit power consumption
  • Ability to detect excessive power consumption”

Microsoft is making a lot of effort in this space.  System Center Configuration Manager 2007 R3 could be considered as “System Center Configuration Manager 2007 Power Management Edition”.  Windows 7 and Windows Server 2008 R2 both include lots of power features.

This new management pack seems to be leveraging some of those features in Windows Server 2008 R2.  The basic concept is that it will retrieve details of power consumption for servers running that operating system.  It allows you to set power plans and use automatic recovery to switch power plans based on server usage.  You can set thresholds and raise alerts when servers consume more than an allocated amount of power.  You can force servers to use no more than a certain amount of power. 

This is a complicated management pack.  There is a Word document with rough instructions.  Please read it thoroughly before importing this management pack.  If you have a physical lab (the management pack won’t do anything with VM’s) then work with it there first.  You should also note that if you use SQL 2008 for your OpsMgr database then you should apply a hotfix first.

Some More Cross Platform Updates For OpsMgr 2007 R2

Microsoft released two more updates for Operations Manager 2007 R2 cross platform extensions.

First is the management pack to take advantage of cross platform Audit Collection Services (ACS).

Second is a new management pack for monitoring cross platform agents, i.e. UNIX and Linux.

Technorati Tags:

Audit Collection Services Adds Cross-Platform Support

I first heard about Audit Collection Services (ACS) at TechEd in 2004.  It was going to be a free download like WSUS.  The idea is that it would be an intelligent alternative to SYSLOG for Microsoft platforms/applications, gathering security logs into a central database.  Instead of gathering everything, it would gather the important alerts/events only.

Time went by and no beta appeared.  Then ACS appeared as a feature in System Center Operations Manager 2007.  OpsMgr 2007 evolved in OpsMgr 2007 R2 to add cross platform support, i.e. MS written native agents and management packs for Linux and UNIX.

Microsoft has now added an extension to this cross platform support to offer ACS to Linux and UNIX:

“System Center Operations Manager 2007 R2 Cross Platform Audit Collection Services enables the collection and audit of events from UNIX and Linux Servers. Using Cross Platform ACS, events are collected from the desired Unix/Linux servers and stored in the Audit Collections Services Database. Audit reports for UNIX/Linux Server collected events are included.

Feature Summary

Collection of Audit events from UNIX/Linux server, including:

  • AIX 5.3 (Power), 6.1 (Power)
  • HP-UX 11iv2 (IA64/PA-RISC), 11iv3 (IA64/PA-RISC)
  • Red Hat Enterprise Server 4 (x86/x64), 5 (x86/x64)
  • Solaris 8 (SPARC), 9 (SPARC), 10 (SPARC/x86)
  • SUSE Linux Enterprise Server 9 (x86), 10 (x86/x64), 11 (x86/x64)

Built in Audit Reports including:

  • Access violations – unsuccessful logon attempts
  • Account creation/deletion/password change
  • Administrator activity – su, sudo
  • Forensic – all events for a computer/event ID
  • User logons”
Technorati Tags:

Looking Into Other Ways To Automate Maintenance Mode

I’m going to be looking at alternative ways to put computers and other monitored resources (e.g. Web and port monitors) into maintenance mode in Operations Manager 2007 R2 this week.  We pushed out patches this weekend.  We warned customers that they might get one or two nuisance alerts.  Sure, each of them just got a couple of alerts but we got a LOT because we get all of them.  I’ve tried a few batch script and task scheduler approaches and each of them has sucked.

I’m going to have to do this in PowerShell I think.  I’ll see how this week goes.  Any non-customer engineering is frozen until the new year.  I don’t want to make changes that may cause unwanted faults over the holidays.  That gives me some time to do some work; I hope!  Pre-sales is still busy and I’m even going out on-site with some hosting customers to do some work with them.

Post a comment to let me know how you get around scheduling maintenance mode in OpsMgr.

Technorati Tags:

Cannot Delete Cluster Object From Operations Manager 2007

I recently decommissioned a Windows Server 2008 Hyper-V cluster.  It was monitored by OpsMgr 2007 R2.  When we shutdown the last cluster node I tried to remove both its agent object and the agentless managed cluster object from OpsMgr administration.  I couldn’t.  The cluster just refused to disappear.  The server agent would delete because there was a remaining dependency – the cluster object which relied on it as a proxy.

It had a red state (ruining my otherwise all green status view) and, more annoyingly, many of the migrated resources (VM’s) still seemed to be linked to the old cluster despite being moved to the new cluster.

I searched and found lots of similar queries.  The official line from MS is that there is no supported way to do this deletion.  There is a hack but the instructions didn’t work for me – I couldn’t find the key piece of info – plus it is unsupported.

So I uninstalled the agent manually.  No joy.  I waited.  No joy.  I rebuilt the server and added it to our Windows Server 2008 R2 Hyper-V cluster.  No joy.  I installed the OpsMgr agent and enabled the proxy setting.

That was yesterday.  This morning I logged in and the old cluster object is gone.  Vamoose!  I guess OpsMgr figured out that the server was now in a new cluster and everything was good.

Lots Of Operations Manager Updates

Microsoft released lots of updates for Operations Manager over the last couple of weeks.  There are lots of updates to management packs, too many for me to go posting them at this time of night.  Have a look on the catalogue and you’ll see them.  Or check your console if you’re using OpsMgr 2007 R2.

Most importantly is KB971541, Update Rollup for Operations Manager 2007 Service Pack 1.

“The Update Rollup for Operations Manager 2007 Service Pack 1 (SP1) combines previous hotfix releases for SP1 with additional fixes and support of SP1 roles on Windows 7 and Windows Server 2008 R2. This update also provides database role and SQL Server Reporting Services upgrade support from SQL Server 2005 to SQL Server 2008.

The Update Rollup includes updates for the following Operations Manager Roles:

  • Root Management Server, Management Server, Gateway Server
  • Operations Console
  • Operations Management Web Console Server
  • Agent
  • Audit Collection Server (ACS Server)
  • Reporting Server

The following tools and updates are provided within this update which may be specific to a scenario:

  • Support Tools folder – Contains SRSUpgradeTool.exe and SRSUpgradeHelper.msi (Enables upgrade of a SQL Server 2005 Reporting Server used by Operations Manager Reporting to SQL Server 2008 Reporting Server)
  • Gateway folder – Contains a MSI transform and script to update MOMGateway.MSI for successful installation on Windows Server 2008 R2
  • ManagementPacks folder – Contains an updated Microsoft.SystemCenter.DataWarehouse.mp which requires manual import

For a list of fixes and tools addressed by this update rollup, see KB971541.

This update is supported for application on System Center Operations Manager 2007 Service Pack 1 only.

Feature Summary

The System Center Operations Manager 2007 SP1 Rollup 1 contains:

  • All binary hotfixes released since Service Pack 1 release
  • Support for Windows 7 and Windows Server 2008 R2
  • Operational and DataWarehouse database support on Windows Server 2008 R2
  • Additional stability hotfixes”

Requirements

  • Supported Operating Systems: Windows 7; Windows Server 2003; Windows Server 2008; Windows Server 2008 R2; Windows Vista; Windows XP
  • System Center Operations Manager 2007 Service Pack 1

Instructions

This update must be applied to each computer that meets the following criteria:

  • Hosts a Microsoft Operations Manager Root Management Server
  • Hosts a Microsoft Operations Manager Management Server
  • Hosts a Microsoft Operations Manager Operations Console
  • Hosts a Microsoft Operations Manager Web Console Server
  • Hosts a Microsoft Operations Manager Reporting Server
  • Hosts a Microsoft Operations Manager Manually installed Agent
  • Hosts a Microsoft Operations Manager ACS Server

Before applying this update it is strongly recommended that Operations Manager databases, Management Server, Report Server and Web Console roles be backed up.

To extract the files contained in this update and installation of the update on the Operations Manager roles above:

  1. Copy the file – SystemCenterOperationsManager2007-SP1-KB971541-X86-X64-IA64-locale.MSI – To either a local folder or accessible network shared folder.
  2. Run the file – SystemCenterOperationsManager2007-SP1-KB971541-X86-X64-IA64-locale.MSI – locally on each applicable computer that meets the predefined criteria.
    You can run SystemCenterOperationsManager2007-SP1-KB971541-X86-X64-IA64-locale.MSI from either Windows Explorer or from a command prompt.
  3. Select the appropriate role to update from the Operations Manager 2007 Software Update dialog.

NOTE: To run this file on Windows Server 2008 you must run this file from a command prompt which was executed with the Run as Administrator option. Failure to execute this Windows installer file under an elevated command prompt will not allow display of the System Center Operations Manager 2007 Software Update dialog to allow installation of the hotfix”.

Use OpsMgr 2007 To Monitor …

Ask an IT Pro what’s the most important thing at work.  Servers?  SAN?  Exchange?  Active Directory?  Not even close.  It’s the coffee pot.  And no one likes and empty one.

Wilbour Craddock just sent me a link to an article with a demo video.  In the video one of the authors of the Operations Manager 2007 management pack created a solution to visually monitor a coffee pot using a digital camera.  It alarms on empty, below warning threshold and below error threshold.  It automatically changes to a healthy state using a monitor.  Superb!

Operations Manager 2007 SP1 Management Pack

Microsoft has released a updated MP for OpsMgr 2007 Service Pack 1. 

“The Operations Manager 2007 SP1 Management Pack alerts you to problems with components such as agents, management servers, the Operations Manager database, agents, modules used by workflows and services so that you can continue to monitor the servers and clients that your business depends on.
The management pack includes tasks that you can automate to get easy access to common diagnostic tools, such as restarting a health service or reloading an agent configuration.

The Operations Manager 2007 SP1 Management Packs includes the following features:

Local and Remote Monitoring of an Agent’s Health

  • Operations Manager agents monitor themselves for events and performance indicators that signal an issue with the agent’s health.
  • Management servers also maintain an external perspective of an agent’s health via the Health Service Watcher.
  • The ‘Agent Health State’ view provides a side-by-side dashboard of both perspectives on the agent.

Optional, Automatic Agent Remediation Capabilities

  • If the Health Service Watcher determines that an agent is unhealthy, a series of diagnostics and recoveries can be enabled to further diagnose the problem and event take actions to attempt to fix the problem (e.g. Ping the server to see if it is completely offline, start a stopped agent, trigger a reinstall, etc.). Refer to the management pack guide for more details.
  • Agents are monitoring their own process to ensure that memory utilization is not sustained at unacceptable levels. If this condition is detected then the agent will automatically restart itself to force the freeing up of memory.

Detection of Problems and Misconfigurations with Run As Accounts and Profiles

  • Checks are run on a regular basis to detect if any of the management group’s “Windows” type Run As Accounts have credentials which are about to expire. Alerts will be raised, and where possible this will be done in advance of the credentials expiring to avoid outages.
  • Alerts will be raised if any errors are encountered during the distribution of Run As Accounts.

Monitoring of problems with Running Workflows in Management Packs

  • Numerous rules are provided to detect if workflows within management packs are failing. Examples of workflows include discoveries, rules, monitors, etc. Failures can range from bad configurations on the workflows themselves, script failures, permissions problems, etc.”