Network Monitoring with Integrated Data Normalization

By Dale Smith
20th September 2017

Overview of the Problem
Multi-vendor networking environments have become standard as enterprises deploy a diversity of networking devices to address the evolving needs of the business. While a heterogeneous networking environment makes sense from a business perspective, it also presents challenges from a network management perspective. In the absence of standards to guide network equipment vendors in their selection of units, scales and naming conventions for device parameters, they tend to invent their own. The result is a proliferation of vendor-specific representations of device parameters.

The burden then falls on network operators to process device parameters across vendors into a common frame of reference in order to make sense of what’s happening in the network. This often becomes an error-prone manual process involving spreadsheets and other work arounds to ensure that errors related to diverse scales, units and interpretations of the data are minimized.

Example: Device Status and Alert Management
To make this a bit more concrete, let’s look at a practical operational scenario. Let’s say that a network operator needs to know if a fan has failed in any device. This is a potentially serious problem because of the risk of overheating. Because a fan failure requires immediate attention, an alert would be the best mechanism to ensure attention is given in a timely manner. In a multi-vendor, multi-device network fan status will be reported in a number of different ways. Examples are shown below to illustrate this point:

A10 Networks:
Uses the axFanStatus variable with seven values to represent fan operational status:

  • failed(0)
  • okFixedHigh(4)
  • okLowMed(5)
  • okMedMed(6)
  • okMedHigh(7)
  • notReady(-2)
  • unknown(-1)

Cumulus Networks, Net-SNMP, others:
Uses the EntitySensorStatus variable with three values to represent fan operational status:

  • ok(1) – indicates that sensor value can be obtained
  • unavailable(2) – indicates that the sensor value cannot be obtained
  • nonoperational(3) – indicates that the sensor is likely broken

Now suppose that a network has devices reporting fan operational status in 5 different ways. This would ordinarily require building 5 different alerts that all do essentially the same thing but in 5 different ways. This presents a challenging problem because it means maintaining 5 different alerts. And, when it becomes necessary to change the fan status alert, such as who gets paged or notified, then it will be necessary to make the change in 5 different alerts within the alerting system. And, the larger the network, the more complex and challenging this problem becomes due to the increasing number of different alerts that must be managed for a single monitored condition. This is highly undesirable because of the risk of downtime due to human error and productivity lost to an inefficient alert management process.

A significantly more efficient, scalable and error-free approach would be to have a single fan status alert for the entire network regardless of device and regardless of vendor. This can be achieved with data normalization.

What is Data Normalization?
Data normalization enables network operators to utilize device parameters in a uniform manner regardless of the device type or vendor. Data normalization involves transforming units, scales and naming conventions of data originating from different sources (i.e. vendor devices) so that this data can be utilized or processed in a coherent manner. For example, memory utilization can be reported as total amount and amount of available memory or as a single metric that reports memory utilization as a percentage. Or, in our example of fan operational status, data normalization enables us to convert fan status data in one format into another format that is uniform across different device vendors and compatible for data analysis, reporting, troubleshooting and efficient alert management.

What is Integrated Data Normalization?
The reality of modern networks is that different vendors have very different data schemas to represent device parameters. A10 Networks provides 7 different fan operational state values. Juniper also provides 7 different state values but they are not the same as those provided by A10. Dell and Cumulus both provide 3 state values for fan operational status but they are also not the same. And finally, Cisco defines either 4 or 6 state values depending on the device model and the software the device is running.

Because these differences between vendor data schemas is known and is well-understood, it is possible to “integrate” this understanding into the network monitoring platform. And, this is what defines “integrated data normalization” as a NetSpyGlass platform feature that leverages detailed understanding of device data schemas to enable uniform handling of device parameters across vendors.

Since NetSpyGlass is a fully-programmable automation platform for network monitoring, users have the ability to programmatically create variables  to represent any values of interest. Users also have the option to use a number of variables available with the platform “out-of-the-box”. With specific regard to fan operational status, NetSpyGlass creates a variable called “fanStatus” with potential values:

  • 1 = normal
  • 0 = failure

Thus, a key benefit of integrated data normalization is the ability to use a variable, like fanStatus, to create a single, reliable and efficient “good”/“not good” alerting mechanism that will respond to any fan failure occurring in the network. The figure below illustrates how integrated data normalization significantly reduces the complexity of managing fan status alerts. Only a single alert is needed for all fans in the network regardless of device type and regardless of device vendor.

How Integrated Data Normalization Works


With reference to the figure above, the fanStatus variable provides a simple mechanism for mapping vendor specific complexity into a simple state variable perfectly suited for use in any kind of alert generation and management logic. For example,

A10 Networks:
fanStatus would have the following values under the specified circumstances:

  • 1=normal: when axFanStatus returns values: 4,5,6,7
  • 0=failure: when axFanStatus returns values: 0, -1, -2

Cumulus Networks, Net-SNMP, others:
fanStatus would have the following values under the specified circumstances:

  • 1=normal: when EntitySensorStatus returns values: 1
  • 0=failure: when EntitySensorStatus returns values: 2, 3

The data normalization shown above with A10 and Cumulus as examples, applies broadly to any device and any vendor – Cisco, Juniper, Arista, Dell and many more.

The Value of Integrated Data Normalization
The preceding discussion illustrates the complexity involved in answering a relatively simple question, specifically – “Are all of my device fans operating properly so that I can avoid an expensive replacement due to overheating damage?” The value of integrated data normalization is that it enables network operators to answer this question quickly and efficiently with significantly reduced risk of human error.

NetSpyGlass’ integrated data normalization capability greatly simplifies the inherent complexity of monitoring multi-vendor networks. This is accomplished by integrating into the NetSpyGlass platform a detailed understanding of the MIBs, OIDs and related variables for supported devices. And, this capability can be extended to additional devices beyond those currently supported in response to customer requests.

Concluding Thoughts
The example of fan operational status discussed above is easily generalized to practically anything a network operator might wish to monitor in their network. Integrated data normalization is simple to grasp but is also a powerful feature that enables NetSpyGlass users to conquer network complexity with impunity. However, multi-vendor networking environments represent just a single dimension of complexity challenging today’s networking professionals. Enterprise infrastructure deployed across cloud, storage, virtualized, software-defined and hybrid environments means that monitoring and managing must trend toward leveraging the kind of automation and programmatic flexibility that NetSpyGlass delivers to its users. Share your experiences with monitoring multi-vendor networks in the comments below.

About the Author

Dale Smith

Dale Smith enjoys operating at the intersection of web strategy and digital infrastructure. With a particular interest in monitoring automation for 21st century networks, he aims to be a helpful resource for networking professionals preparing for the software-defined future.

Share this Post

Leave a Reply

avatar
  Subscribe  
Notify of