GXMLG 2.0

I finally broke down and rebuilt my GXMLG tool.  Given the complexity of the task, I originally wrote the applet using MS Access.  However, to make things easier to distribute and easier to troubleshoot and use, version 2.0 uses perl and is run from the command line.  To illustrate the difference, the old utility was 6.5MB.  The new script is 11KB.  That's what a graphical interface gives you.



You'll have to install perl (I use strawberry perl on windows boxes) and run the script like this:

>perl gxmlg.pl

Running it without any arguments shows you the help file:

This script outputs any combination of configuration files for the NetQoS
suite of products. You must install strawberry perl and Text::CSV,
Text::CSV_XS, and Getopt::Long. To install CPAN modules, run cpan [module name]
from the command prompt.

Example: cpan Text::CSV

This script was created by Stuart Weenig (C)2012.  For more information visit
http://stuart.weenig.com. This script may be redistributed as long as all the
files are kept in their original state.


Current Version: 2.0

Usage: PERL gxmlg.pl [-outnpcxml] [-outsacsv] [-outnvcsvnv] [-outucmcsv]
                     [-infile NAMEOFINFILE] [-npcinspath INSERTPATH]
                     [-npcxmlname NPCXMLFILE] [-sacsvname SACSVFILE]
                     [-nvcsvname NVCSVFILE] [-ucmcsvname UCMCSVFILE]

    -infile NAMEOFINFILE        Specifies the name of the Sites file to be
                                imported. (If omitted: sites.csv)
    -outnpcxml                  Output an NPC XML groups definition file.
    -npcxmlname NPCXMLFILE      Name of the NPC file to be output. (If
                                omitted: NPCGroups.xml)
    -npcinspath INSERTPATH      The path to the group that will serve as
                                the insertion point.  Required if using
                                -outnpcxml option.
    -outsacsv                   Output a SA networks CSV file.
    -sacsvname SACSVFILE        Name of the SA file to be output. (If omitted:
                                SANetworks.csv)
    -outnvcsv                   Output a NV discovery scopes file.
    -nvcsvname NVCSVFILE        Name of the NV file to be output. (If omitted:
                                NVScopes.csv)
    -outucmcsv                  Output a UCMonitor locations file.
    -ucmcsvname UCMCSVFILE      Name of the UCMonitor locations file to be output.
                                (If omitted: UCMLocations.csv)

You must install strawberry perl and Text::CSV, Text::CSV_XS, and Getopt::Long.
To install CPAN modules, run cpan [module name] from the command prompt
Example: cpan Text::CSV

If the server you will be running this on doesn't have access to the internet, you
won't be able to install the modules automatically (since they have to be downloaded
from the internet).  The solution is to download the tarballs from www.cpan.org and
extract them using winzip or winrar or 7z.  You might need to extract several times
until you get just the folder with the files in them.  Then copy them to your server.
Open a command prompt and cd to the directory containing Makefile.pl (you'll have to
do this for each module).  Then execute the following:

     perl Makefile.pl && dmake && dmake test && dmake install

For the text modules, do Text::CSV first, then Text::CSV_XS.

The script itself is pretty simple.  Specify an input file.  This input file is the same sites/networks file referenced in my previous blog post.  Here's a sample sites list to get you started.  Then just decide which output files you want.  If you specify the npc output file, you'll also need to specify the insertion point (more information about the insertion point).

If you don't have internet access on the box, you won't be able to install the Text::CSV modules to install (since they come from the internet).  The solution is to download the Text::CSV and Text::CSV_XS tarballs and extract them using winzip or winrar or 7z.  You might need to extract several times until you get just the folder with the files in them.  Then copy them to the NVMC.  Open a command prompt and cd to the directory containing Makefile.pl (you'll have to do this for each one).  Then execute the following:

perl Makefile.PL && dmake && dmake test && dmake install

Do Text::CSV first, then Text::CSV_XS.

NPC Sites and Groups 7 of 7: GXMLG - Exporting the Configuration Files

Part of a series of posts.  To see all posts, click here.

Once the sites and networks are completely configured in the tool, configuration files can be generated for NPC, NetVoyant, SuperAgent, and UCMonitor.

Before an NPC configuration XML file can be generated, the tool need s to know the path to the ‘All Sites’ group. If an XML file was imported that was exported from an existing ‘All Sites’ group, the tool extracted this information from the XML. If a CSV was imported or the Sites/Networks list was built manually within the tool, this setting must be input manually on the ‘Import’ tab of the tool.

Three other settings of importance exist on the ‘Import’ tab: NVDatasource, SADatasource, and UCMDatasource. If the values of these fields are set to ‘true’ the tool will include rules in the NPC XML for that type of datasource. The tool will also include CSV outputs for those data source types. Setting the value to anything other than ‘true’ will exclude that type of data source output.

There are two options for exporting the configuration files. On the ‘Export Config Files’ tab, the ‘Export All Config Files…’ button will prompt for a location to output the files. The tool will save the XML and CSV files in that target folder. Those files can then be imported into their corresponding products.

The other option for export is to click the individual generation buttons. These will generate the output and place it into a text box in the tool. The text could then be copied and pasted into a text editor to save the files.

NPC Sites and Groups 6 of 7: GXMLG - Automatic Config File Generation

Part of a series of posts.  To see all posts, click here.

At this point, this is all the information strictly needed to start with a Sites/Networks list and end up with a fully populated Sites group in NPC, a NetVoyant that should be discovering all devices at all sites, a SuperAgent that only needs an active monitor port, and a UCMonitor that only needs an active monitor port (if monitoring only Cisco telephony).

The obvious next step is to implement this into the product in such a way that this configuration is done in NPC and synchronized to the sub-products. However, since CA is currently sans an NPC Product manager, we’ll have to suffice ourselves with a third party tool.

NPC Sites and Groups 5 of 7: UCMonitor Locations and SuperAgent Networks

Part of a series of posts.  To see all posts, click here.


UCMonitor Locations
As mentioned before, the relationship between NPC Sites and UCMonitor Locations should be a 1-to-1 relationship. Locations should be named the same way the locations are named in NPC. All that remains is to ensure that the networks that correspond to each site are added to each corresponding location.

UCMonitor locations can be imported via a CSV file. The syntax is described in the Installation Steps documentation and is shown below:

NPC Sites and Groups 4 of 7: NetVoyant Discovery Scopes

Part of a series of posts.  To see all posts, click here.

I have a philosophy about NetVoyant and discovery scopes. I believe that NetVoyant should be opened up to all segments of the network. Most people disagree saying there’s no need to discover workstations and printers.

While I agree that there’s not much value in discovering or polling people’s desktops or laptops nor network attached printers, there are mechanisms built into NetVoyant that allow partial discovery of those devices without using licenses. In fact, NetVoyant is setup by default to ignore workstations and printers. During discovery, NetVoyant will attempt to get the sysObject OID. Based on this sysObject OID, NetVoyant knows what type of device is being discovered. If it is a workstation or printer, it will be classified as such and placed into the workstations or printers group and discovery will halt for those devices.

The best scenario will depend on each individual situation. I’ll detail a middle-of-the-road solution that should work in indecisive situations.

NPC Sites and Groups 3 of 7: Building the Groups

Part of a series of posts.  To see all posts, click here.

Back when NPC underwent a dramatic set of upgrades, one of the new features was the group tree. Administrators could now create groups of monitored objects that could be applied to pages as filters. Recently, the NPC developers added a context page so that users could actually drill into a site. Instead of applying a filter, NPC would take the user to a set of pages specifically designed to show a set of site overview reports for the objects in the group. These site groups are crucial to implementing the best group structure.

Calling back to a Best Practice document written by NetQoS several years ago, NPC administrators should create five groups under the root, /All Groups: All Sites, Regional, Line of Business, Reporting Groups, and zExcluded Items. Optionally, a sixth group could be added, zRetired Groups, where groups that should no longer be used can be retired without actually deleting them.

NPC Sites and Groups 2 of 7: Building the Sites/Networks List

Part of a series of posts.  To see all posts, click here.

Before any thought can be dedicated to configuring the products, a good Sites/Networks list should be compiled. The Sites/Networks list should represent the smallest building blocks for reporting groups. Ideally, if you wanted to report on anything smaller than one of these sites, you’d report on an individual device or poll instance.

This list may already exist in some form or another. You’ll probably need to clean it up a bit. Whether you start over fresh or take an existing list and clean it up, you should have a list similar to the one shown below. Columns could be added to the right to include metadata about the site like Line of Business or Regional membership. This information is not required to implement this best practice, but can help when filling out other groups. The information I’ve provided below will be used as an example throughout this document and should hopefully exemplify most of the complex situations possible in corporate networks.

Site Name	Network	Line of Business	Region
New York	10.1.0.0/16	Sales Office	NA
Los Angeles (Bldg 1)	10.2.0.0/16, 10.3.0.0/16	Corporate	NA
Los Angeles (Bldg 2)	10.4.0.0/16, 10.5.0.0/16, 172.16.1.0/24	Corporate, Sales Office, Data Center	NA
London	10.100.0.0/16	Sales Office	EMEA
Singapore	10.150.0.0/16, 192.168.150.0/24	Sales Office	APAC

Table 1: Raw Sites List

Now that the sites list is compiled, it needs to be cleaned up a bit. The Los Angeles site is actually two sites. Also, Bldg 2 of the Los Angeles site is comprised of three different network segments, each serving a different line of business (LOB). I recommend treating each of these network segments as separate locations. Most reporting will want to segregate data center operations from the corporate network infrastructure monitoring. Also, since Corporate and Sales are two different LOB’s, splitting them into different sites will allow easier reporting by LOB later on. After cleaning up the raw Sites/Networks list, we end up with the list shown below:

Site Name	Network	Line of Business	Region
New York	10.1.0.0/16	Sales Office	NA
Los Angeles (Bldg 1)	10.2.0.0/15	Corporate	NA
Los Angeles (Bldg 2 - Corp)	10.4.0.0/16	Corporate	NA
Los Angeles (Bldg 2 - Sales)	10.5.0.0/16	Sales Office	NA
Los Angeles (Bldg 2 - DC)	172.16.1.0/24	Data Center	NA
London	10.100.0.0/16	Sales Office	EMEA
Singapore	10.150.0.0/16, 192.168.150.0/24	Sales Office	APAC

Table 2: Final Sites/Networks List

You’ll notice that I’ve added to the names of Bldg 2 to make them unique. Also notice that I’ve simplified the network definition for Bldg 1 since the two subnets were consecutive and could be summarized using a smaller subnet mask.

To be continued...

NPC Sites and Groups 1 of 7: The Philosophy

Part of a series of posts.  To see all posts, click here.

A while ago, I started building a document that detailed the best practices I had developed around groups and sites in NPC.  I applied to present this at CA World this November, but the new direction of CA World wasn't compatible with this type of presentation.  As a result, I'm going to post my findings here.  I'll post the sections individually then maybe post the whole thing later as a single document.  I've also built a little applet (using my limited programming skills, so be nice) that takes a majority of the manual work out of following these best practices.


The Philosophy

After having setup countless instances of the NetQoS suite of products and after the most recent release of SuperAgent, I’ve stumbled upon a discovery that I think is significant.  Frankly, I’m surprised no one at NetQoS has realized or built on this relationship.  Perhaps they have, but due to the recent evacuation of NetQoS talent from CA, no one has stepped in to fill the gap.  This is why I’m writing these posts.

This discovery came after being tasked to revamp the group structure in a NetQoS Performance Center (NPC) installation.  At the same time, a request came in to clean up the locations in Unified Communications Monitor (UCM).  It was then, that I noticed that the information required for both was essentially the same.  I needed to know what locations we had and what subnets were in use at those locations.  Not long after that, I realized that the same information is needed, although in a slightly different form, for SuperAgent and NetVoyant configurations.

Therefore, in order to configure four of the NetQoS products, one set of core data is required.  There are obviously other pieces of the configuration that are required to get the best monitoring.  In these posts, I’ll detail how the Sites/Networks list can be used to best configure the core NetQoS products.

To be continued...