What do I do for a Living? – Step 5: Discovery & Mapping

This is a follow up to series of posts titled, “What I do for a Living.”  That question was the precursor to lead me to write a series of blog posts detailing my perspective on a network & systems management (NSM) maturity model.  The NSM pecking order in terms of monitoring maturity:

1.     Availability Monitoring (discussed here)

2.     Performance Monitoring (discussed here)

3.     Fault/Event Management (discussed here)

4.     Correlation & Enrichment (discussed here)

5.     Discovery & Mapping

6.     Real Time IT Dashboarding

7.     Service Level Management

This  post will discuss Step 5 of my NSM pecking order – discovery & mapping.  I like to think of myself as a very pragmatic fellow.  I am a big fan of technology, but in my book it needs to work reliably to be considered useful.  Up to this point in my NSM pecking order it is safe to say that the previous steps are all tried and true.  They have proven themselves out through the years.

This brings us to the topic of this blog article – discovery & mapping.  There is no doubt that integrating discovery & mapping into your overall monitoring solution is beneficial.  Anything that can help the Ops team to more quickly identify and resolve an issue is helpful.  Let’s start with discovery.

Discovery has long been a highly sought after feature in the world of management and monitoring software.  The idea behind discovery is simple enough.  Let’s consider how Monolith leverages discovery to hopefully give a good example of its usefulness.  Prior to being able to monitor an IT infrastructure you need to know what you are going to monitor.  That traditionally means configuring the system to monitor a particular environment.  Within Monolith’s administration component organizations can configure discovery profiles.  The purpose of a discovery profile is to identify the devices that the Monolith server is able to reach.  Discovery profiles can be a seed file copied from another system; a LDAP, Radius or Active Directory scan, a CDP (Cisco Discovery Protocol) scan of the environment, or scan of an IP address range (e.g. 192.168.1.*).  Step one in the process identifies which devices in the discovery range the Monolith server can reach.

The next step is to type the devices.   This is done via either SNMP or WMI.  This lets us identify what type of device we are dealing with.  It will identify whether the devices is a Windows 2003 server, a linux Red Hat sever, a Cisco Catalyst 6509 switch or a Juniper router.  Typing of devices is beneficial for a number of reasons.  First, within the Monolith system you can view devices by Group, Type or Name.  From a navigational perspective it can be useful to see all of the same type of devices listed together.

Second, and perhaps most importantly, typing of devices can lend itself to auto-provisioning of device monitoring.  It is highly likely that you may want to monitor all of a certain class and type of devices in the same fashion.  Because we can auto-discover devices in a range that the management server can talk to, and because we can auto-type the devices to know what type of device that it is, then it logically makes sense to utilize that information to make your life a little easier.  Within the Monolith system we support the ability to create monitoring templates.  Monolith uses automation logic to automatically apply templates and provision monitoring for devices.  The value is more accuracy of monitoring and auto-provisioning of monitoring.  A big win for any large organization!

The third step in Monolith discovery process is to perform deep discovery.  Deep discovery allows us to understand more detailed information about the devices.  We discovery things like device info, interface information, ip routes, mac addresses, ARP table, VLANs, CDP neighbors and topology layout.  This information is then presented within our Topology Manager module.

Now that we have talked through the discovery portion of this  post I will dive into the mapping portion.  Users of monitoring systems, especially folks in the networking side of the house, have long desired the ability to visually see what is connected to what within their environment.  This can certainly streamline the troubleshooting process.  Let’s assume a user is having a network connectivity problem.  Wouldn’t it be nice to be able to dig into the monitoring system to see which switch and port the user is connected to and even see what the full path is between the end user and the server they are trying to access.  Hopefully that example shows you the benefit of providing visual mapping functionality.

Ever since I first started working with monitoring software there has always been a special fondness for software that could provide mapping capabilities.  This started out with software like WhatsUP Gold and HP’s Network Node Manager.  These applications would provide a nice little map showing what network devices are connected to what other network devices from a Layer 3 perspective.  This was pretty handy information; especially when you consider the fact that in the world of networking organization were just beginning to move from flat networks to routed networks.

Of course the industry continued to evolve quickly.  I can still remember back to 1996 when I sold a 24-port Bay Networks Lattis Switch for over $1,000 per port.  That same technology can now be bought to under $1,000 for the whole switch.  Network segmentation with CSMA-CD became all the rage.  Networks were sliced and diced all over the place to allow for faster performance.  As network infrastructures moved from using hubs to switches everywhere the need to be able to understand network connectivity via visual mapping became increasingly important.  Around this time we started seeing companies like Riversoft and Smarts coming out of the wood work.  Micromuse also had their Visionary product they were selling at this time.  This technology purported to do wonderous things.  They would auto-discovery your network, build a data structure to understand your networks connectivity model, provide dynamic maps of your network infrastructure and even do connectivity based correlation.

If my memory serves me correctly, this technology started popping up in the 2001/2002 time frame.  I was working for a large technology management consulting firm at the time.  As part of our reference architecture we started to layer in this functionality into client environments for NOCs that were looking for the latest and greatest capabilities. This was not, and, in my opinion, still isn’t the monitoring technology that most organizations should start out with.     These products have historically been difficult to install and configure.  They also seemed to often point to bad cables or connections as the issue an awful lot of the time.

By the way, I really cannot blame the problems solely on the technology.  In the world of management and monitoring technology the axiom “garbage in, garbage out” is incredibly appropriate.  Ever since I have been in this space one of the biggest challenges has been the lack of standards within customer environments that makes this type of monitoring so difficult.  I can count on one hand the number of networks we have run into over the years that have had a pristine environment.  Generally speaking, very few organizations have had a defined ip addressing scheme, device naming scheme, standard device login credentials, standard SNMP community strings, a single NTP clock source, etc.  These are all of the things that make this type of monitoring so difficult to recognize value with.  If, and it’s a big IF, organizations have a clean environment, then this type of technology can provide nice benefits to the organization utilizing it.

At Monolith Software we introduced our Topology Manager module with version 3.3.  We have the ability to perform out of the box network connectivity based mapping and correlation.  We have also included the ability to perform what we call hierarchical correlation via our Hierarchy Storage Engine (aka HSE).  The HSE allows us to perform parent : child correlation on any type of hierarchy.  A good example of this is the ability to perform SONET or DWDM correlation based upon an optical hierarchy.  Another example is customer : circuit correlation whereby a link outage can show circuits impacted and circuits impacted can show customer(s) impacted.  This type of correlation and associated mapping is often times accomplished by integrating our HSE with an organizations provisioning or billing information to do higher level correlation that was previously impossible to achieve.

Today’s post took you through the value and business case for utilizing discovery & mapping (& even some correlation).  As I hopefully described, there are many benefits to be gained by leveraging this type of monitoring logic.  The key to remember here is the ‘garbage in, garbage out’ paradox.  A clean, standards based infrastructure can be your best friend when utilizing this type of monitoring technology.

Next blog post – Step 6 Real-time IT Dashboarding.

Cheers!

Technorati Tags:
availability monitoring, performance monitoring, fault management, event management, correlation, enrichment, discovery, mapping, monitoring, application monitoring, IT monitoring, IT Ops, DNS, DHCP, HTTP, HTTPS, up down monitoring, IT operations, synthetic transaction, passive application response, performance monitoring, NSM software, monitoring software, monolith software, monolith-software.com