Companies who have busted their budget getting an IT architecture in place frequently try to cut costs anywhere they can, and all too frequently, expenses trimmed are the monitoring components of the network infrastructure. As a network management weenie from way back, I’ve always claimed “I get no respect.” It seems that few care about network management until the network goes down. Then you should see how interesting SNMP, syslog, protocol analyzers and performance management tools become. Suddenly they are very hot!

Symbolic of your data going up in flames if you can’t troubleshoot fast enough for real-time analytics use cases (stock trading, retail web site, etc.).

And this is where it gets interesting depending on where you sit.  The nascent, emerging and legacy monitoring tools manufacturers who have to come up with insightful analytics harvesting meaningful information out of a mountain of moving, changing data are probably pretty stressed.  But it’s not as bad for the network traffic cops.  To them, “big data” really isn’t anything new, it still has switches, routers, servers, databases, pictures and video –just a ton more of it to monitor!  And that’s the real the tricky part; how do you harvest all that “stuff”?

How does an enterprise, over-burdened service provider or even a successful SMB deal with this conundrum? And what about those weary warriors manning the spaghetti factories called data centers armed with Wireshark and iReasoning on their laptops or praying their routing, application, server upgrade, etc. change goes successfully in their 2-5AM outage window when no mere mortal is particularly “sharp” at that hour? How many days are spent praying that you capture enough data off a SPAN port in a protocol trace to track down the bad guys trying to steal your big data or analyzing why errant server sessions aren’t cooperating in your piece of cloud like they should be? Big Data is a tough enough proposition even when life is good; when things go bad, it can go very bad. (e.g., Amazon’s EBS outage 4/21/11) All those sexy Big Data analytics aren’t worth much if you’re storming the northeast sector of your cloud, now are they?

At some point, everyone has to pick and choose the juicy bits on what to keep and what to throw away because it’s nearly impossible to harvest everything the way we used to; that’s where a network monitoring switch comes into play. As the plumbers of the data highways, IT departments must prioritize, filter and synthesize information to analyze it in one way or another be it data analytics or security and capacity planning.

Network monitoring switches make this job significantly easier by pre-cabling all those monitoring points from SPANs and TAPs to them instead of directly to the monitoring tools. Your tools are also connected to the network monitoring switch and then all that wonderful info can be shared, filtered, sliced, diced, de-duplicated and directed from one-to-many inbound feeds to one-to-many tools. All with a drag-and-drop control panel that can be automated if you want to sleep in instead of troubleshoot the latest MOP change overnight. Depending on quantity of connections and potential amounts of data in/out (capacity of total ingress traffic vs. total actual processing capacity of your egress monitoring tools), you can filter out huge amounts of unnecessary overhead traffic and/or aggregate tool ports into load balanced groups to keep up with your monitoring requirements. Filters can be changed on-the-fly, thereby avoiding “time-sucking” change board procedures and problem resolution to begin immediately without waiting for permission. This can get you many more months, or even years, of extra performance out of that huge monitoring CAPEX expenditure you laid out only yesterday, i.e. investment protection and another tool in your arsenal.

Bottom line: focusing on your business objectives will tell you what choices to make on how much data is critical to collect and keep, and what you can afford to ignore or look at later.