Thursday, July 25, 2013

Google's new Chromecast dongle sells out on Play Store

UPDATE: Several reviews have come out about the Chromecast. Since I haven't dedicated the $35 to purchasing one myself, I'll defer to David Pogue.
Chromecast: is it a game changer?  The public is eating it up.  It's definitely a game changer.  But to understand how it's going to affect things, you have to think about how Apple has been going about the same thing.  For years now, Apple has had a 'pet project' called AppleTV.  It has never been on the front line of Apple's advertising.  There have even been several analysts over the years that have predicted the doom of AppleTV.  Apple has persisted though without really highlighting AppleTV.  AirPlay was available through the iOS devices, but was mainly used for streaming music.  It was a cool feature but not a wave maker.

When the iPhone 5 and iPad 3 came out, they both had AirPlay, which meant that anyone who already had an AppleTV could mirror their screen to their big screen TV.  This was a significant event and clearly showed Apple's desire to get into the living room.  They've since released newer updated hardware and software for the little device.  You can watch Hulu, Netflix, HBO, and a bunch of other content, as long as your have an account.  Recently, an iOS game developer released a game that really doesn't work without mirroring.  It's a tennis game much like the game that comes with the Wii.  The difference is that you play with your phone as the controller using its internal accelerometers and gyroscopes to detect your motion.  The video is displayed on your phone, but that doesn't really work when you're swinging your phone around like a tennis racket.  However, if you mirror your phone to your AppleTV, you essentially get the same game that came with the first generation Wii.  This game opens the door for other games that can be built in the same way.  All the work is being done on the phone/controller in your hand, while the video is mirrored up to the big screen.  The next advance I expect to see from Airplay is the ability to mirror multiple devices to the same AppleTV.  Putting two people's phones' screens on a single TV gives multiplayer games a chance (imagine Mario cart but using your phone as the steering wheel).

Then Google released Chromecast.  It's 1/3 the cost of the AppleTV and seems to work across different platforms.  If you only look at the Chrome browser mirroring capabilities, this is huge.  All the things that can be done in a Chrome browser can now be done on a big screen TV while not requiring any extra remote controls much in the same was as Airplay does for Apple phones' screens.  While Chromecast appears to compete directly with AppleTV given all the current features, there's more to it than that.  AppleTV and Chromecast are on the same trajectory.  While Google was late to the phone game joining in only after Apple already had a tight grip on the market, they came to the table much sooner with Chromecast.  
The other major factor here is that Airplay for Apple iOS devices only works with mobile devices.  Chromecast promises to work not only using any mobile device but also PC computers with the Chrome browser installed.  This means that all the content that people currently consume using their PC can now be consumed on their TV.  This may not seem big, but given the cheap entry point, Chromecast could easily be used as a secondary monitor for every device in the house.  
On top of that, since Chromecast can mirror anything from the Chrome browser, much content that has had a hard time breaking into the living room now has a direct link.  For example, Hulu has two services, free and paid (Hulu Plus).  The paid service doesn't have much content that the free service doesn't.  There's a little, but it's not really what subscribers are paying for.  Hulu Plus subscribers have the ability to stream Hulu content on just about any device they can get their hands on.  Hulu free users can only get content through their browser (but not a browser on a mobile device).  With Chromecast, users can easily use the free Hulu service but still view it on their TV without hooking up a PC.  This means that Hulu will need to reevaluate what users are really paying for.  At $8 a month, a one time investment of $35 for Chromecast not only will pay for itself in 5 months, but will get me pretty much the same content with little extra hassle.
Yes, Chromecast is a game changer.  At $35, it's cheap enough to give it a chance even if it doesn't eventually work out.  It's not like the $99 investment in an AppleTV.  
at 0 comments More posts like this: ,

Tuesday, July 23, 2013

BTSync: A new alternative to cloud based drive services

In a previous post, I wrote about how to connect your system folders (My Documents, My Pictures, etc.) to a Google Drive/Dropbox/Skydrive account.  The benefit of this is that you always have a backup available on the web.  If my hard drive were to die today, I wouldn't be too bad off since all I need to do is download the Google Drive desktop app and redirect my system folders to my Google Drive folder.  All my stuff would come back like it was never gone.  I'd still have to install applications but that's not too bad.  It's nice every once in a very long while to lose all my programs.  It forces me to trim the fat or look for better versions of the apps I use (Like Paint.Net or Notepad++).

However, I've always been a fan of the Bittorrent protocol.  It's a peer to peer file sharing program that most people use to download illegal movies or music.  While those days are behind me, I've tried to help people understand that P2P is no the same as illegal.  There are perfectly legal uses of P2P protocols like Bittorrent (see this, this, and this).  

And so my becoming a fan of Google Drive (and also Google Apps that allow me to work on a spreadsheet or document with many other people simultaneously) only makes sense.  If you're happy with that, or you only have one computer and no one you would ever want to share anything with, stop reading this blog post now (check out my most popular post instead).

Last week I stumbled upon BTSync (it has since been spun off and renamed to Resilio Sync).  This is a little app created by the same people that designed the Bittorrent protocol.  While there are several uses of BTSync, the main use is to compare it to the features and functionality of products like Google Drive.  Here is my comparison matrix:

FeatureGoogle DriveSkyDriveDropboxBTSync
Size Limit115GB27GB18GB∞
Shareable ContentMust be in Google Drive folderMust be in SkyDrive folderMust be in Dropbox folder3Any number of existing folders
Your files stored on a corporate serverYesYesYesNo
Online File EditorsYesYes4Viewer: Yes Editor: NoNo
Web Access to FilesYesYesYesNo
SharingOnly with Google users5Only with MSN Passport usersOnly with Dropbox usersAnyone with the app
1Without paying anything
2Shared with Gmail and Google+ Hi Res Photos
3Although pretty easy
4If you have a paid subscription to Office360
5Not necessarily Gmail users, but anyone with a Google account

So, let me highlight some of the reasons that BTSync intrigues me.  First of all, there is no limit to the amount of content that can be synchronized.  This is mainly due to the fact that your sync'd files are not stored on some limited corporate server somewhere, they're stored only on the systems where your files are synchronized.  This is a double edged knife, however.  While Cloud based drives can be used for backup, BTSync doesn't back your content up to the internet.  If you add a folder to BTSync only on one computer, the files aren't copied anywhere.  So, unlimited size but no storage on any corporate servers.  That may be two advantages in some peoples' books.

Unlike most of the corporate cloud based drives, BTSync is only about the file transfers.  As such, there are no online editors for your files.  However, since there is no web access to your files (because they're not on any corporate server) you'll only be accessing your files from your desktop.  You can install any office productivity suite locally (or even use google docs in a roundabout way).

Sharing is another feature that is different from other offerings.  While other offerings essentially require you to have an account in order to have RW access to a shared folder, BTSync will allow anyone with the app to access/sync your folder as long as they have the secret, a special, very long, very complicated password.  If you give another person the secret to your folder, they can sync your folder with a folder on their computer.  Any changes either of you make will be reflected in the other's sync'd folder.

You can however, give out two other types of secrets: a read only secret and a one time secret.  The RO secret allows the person to whom you give it to sync your folder to a folder on their system, but they won't be able to make changes to your folder.  This is a good way of distributing files to friends.  For example, you could setup your pictures folder and hand out the RO secret to family members.  They would then get copies of any pictures you put in your pictures folder (think of doing this with your iCloud Photo Stream).

Have you started thinking about the possibilities yet?

Another thing I did right away with BTSync was to synchronize my Dosbox working directory across all my PCs.  I play retro DOS games every once in a while.  By synchronizing the working directory for Dosbox (a DOS emulator) I can access the games, save files, and anything else on any of my computers.  This allows me to play a game on my desktop then save the game, exit Dosbox, go to the living room and launch Dosbox and pick up the saved game right where I left off.

I'm hosting a LAN party this weekend.  I setup a folder where I intend to put all the installers and files needed (including my dosbox folder).  I've added it to BTSync and will hand out the RO key via Facebook and email to everybody coming to the party.  That way they can install the games ahead of time to make sure they work.

Another idea I had was to use BTSync to replace NQSync (which I had originally intended to write using the bittorrent protocol anyway).

One more feature then I'm done.  I promise.
In my previous post, I talked about changing the default location of system folders.  This is necessary since most cloud drives require the sync'd files to be in a particular folder.  So I have to move my system folders to that sync'd folder in order to get them to sync.  With BTSync, I don't have to move the folders.  I can setup each folder in BTSync without moving it.  This means that I don't have to move anything, I don't have to change Windows configurations, or anything.

I haven't done it yet, but I will probably only run Google Drive on one of my PCs.  The rest will use BTSync to stay in sync.

So, that's my initial review of BTSync.  So far, I don't see it replacing Google Drive, but I do see myself using it to distribute pictures and home videos to my family, keeping my games in sync across all my PCs, possibly synchronizing recorded TV shows, making backups, and using it at work.
at 0 comments More posts like this: , ,

Thursday, June 20, 2013

Tailing a File in Windows

There is an easy and very useful command in most Minix systems called tail.  Essentially tail shows the last bit of a file.  This can be really handy when you're trying to read a log file with 10,000 lines in it.  You can use tail to view the last 10 lines in the file.  While no such command exists for the native Windows command interpreter, there is an option in Windows PowerShell.  It's pretty easy to use and since PowerShell is built into all Windows systems XP and later, it's a handy way to get similar functionality without having to download or install anything.
However, the command isn't called tail, it's called Get-Content.  Here are some good examples and some of the options.  The one I like is:

This returns all lines of the file specified in [filename] and waits for the file to be changed.  When the file changes, the display is updated.  This is great for when you want to watch a log file for new entries.
at 0 comments More posts like this:

Friday, June 7, 2013

NetVoyant Trap Processing

I decided to go through some of the content I've garnered over the years and make a few videos.  This one shows how NetVoyant goes through its processing of incoming traps.  Enjoy!

at 0 comments More posts like this: ,

Monday, June 3, 2013

AirPrinting with an HP printer

If you have an iOS device and an HP printer, you might have been disappointed to find out that even though the HP printer uses AppleTalk (the protocol behind AirPrint), you can't actually print to the printer from you iOS device.  This has been one of the main reasons my wife hasn't completely abandoned her desktop PC.

Here's how I fixed it:
Use a little utility called AirPrint installer.  It installs a windows service that reshares the printers from your Windows PC using AppleTalk (the version that is compatible with iOS 6).  I tried doing it without the reg fix, but the printers wouldn't work.  After the reg fix, sure enough, I saw the two printers I have connected to my PC but they both had padlocks.  I used the recommended fix (enabling the guest account on my pc) and they started working.

Here's the utility: http://forums.macrumors.com/showthread.php?t=1293865
at 0 comments More posts like this: , ,

PiTunes

Previously, I posted my experience automating iTunes for use in the nursery.  That has been working very well and I wouldn't normally see any reason to change it.  However, since I recently acquired a Raspberry Pi, I've had several ideas about what to do with it.  The idea that won was to replace the nursery room computer (which was already a Dell Studio Hybrid small form factor PC).  The advantages would be that the Pi is fanless and quite a bit smaller.  I could also then reuse the Dell Hybrid for other things.

The same objectives apply:
  1. I want the music to come on and turn off by itself
  2. I don't want the music to start off at full blast in case we've put the boys down early.    
  3. I want the music to play whenever we put the boys to sleep, which could be as early as 7pm but as late as 9pm.
  4. I want to be able to rotate playlists so it's not the same thing every night.
  5. I want to be able to kick off the music any time.
  6. I want to be able to manually override either the current song or the current volume level.
So, of course the first thing I did was download the Rasbian image and get it booted up, connected via SSH, and downloading updates.  I'll assume if you're also trying to do this that you'll use the instructions at RaspberryPi.org to get you started with that.  To get the most recent updates, type sudo apt-get update && sudo apt-get upgrade.  This may take a while, depending on your internet speed.  The Raspberry Pi doesn't have that much CPU either, so you might just kick this one off and go to bed.

I also purchased a Netgear G54/N150 Wireless USB Micro Adapter and plugged it in.  I'm a Windows guy, so most of what Linux does appears magical to me.  I had a suspicion that the Raspberry Pi would get on my wifi all by itself.  Of course it didn't because at the very least it didn't have my SSID and WPA pre-shared key (the password).  I was lucky enough that the Raspbian distribution recognized it and fired it up without requiring me to load any drivers (thank the stars!).  I verified this by issuing an iwconfig command.  This showed me that it was all ready to go.  Since my network uses WPA2 encryption, I had to configure a built in utility called supplicant.  In the end, all I had to do was edit /etc/wpa_supplicant/wpa_supplicant.conf and add a couple lines:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

 network={
    ssid="Hobbiton"
    proto=RSN
    key_mgmt=WPA-PSK
    pairwise=CCMP TKIP
    group=CCMP TKIP
    psk="yeahright"
}

Much documentation online says all you have to do to get it working is bring the interface down and back up.  I wasn't able to get it to grab a DHCP address without rebooting the Pi. To get the interface to come online at boot time, I added ifup wlan0 to /etc/rc.local.

After I got the Pi up and running, it was time to install a few things that aren't included in the base image.  Mainly, I needed mplayer (that's the media player that will actually play the music) and ffmpeg (a bunch of codecs that will allow mplayer to play all kinds of files including mp3 and m4a).  I also wanted to be able to access the log file from wherever, so I decided to install a lightweight web server.  I'll log to index.html in the www folder.  Type sudo apt-get install lighttpd mplayer ffmpeg and sit back for a while.    

The next thing is to get the actual music onto the Raspberry Pi.  At first, I started out by putting all the music (and my scripts) on a USB stick.  In order to give the Pi access to a USB drive, just follow the instructions here.  I eventually went instead with a network attached hard drive, so I followed the instructions here to create a persistent mount to my NAS.  After a while, I noticed some jitter in the music, so I moved the data back to the SD card (where the OS is also installed).  I'm trying to listen now to see if that makes a difference to the jitter.

Now, on to the script.  The script runs as a cron job.  The comments should explain exactly what's happening.

By the way, the contents of /mnt/nas/PiTunes are: christmas/  nothing/  music/  play.sh  waves/.  Version 2 of the script will play the Christmas directory instead of the music directory during the month of December.

The next part involves running the script from CRON, which is equivalent to Windows' Scheduled Tasks.  To add jobs, just run crontab -e.  The format is one job per line, parameters separated by spaces:


Mine looks like this:


So, let's look at the objectives again:
  1. I want the music to come on and turn off by itself - check!
  2. I don't want the music to start off at full blast in case we've put the boys down early - check!
  3. I want the music to play whenever we put the boys to sleep, which could be as early as 7pm but as late as 9pm - check!
  4. I want to be able to rotate playlists so it's not the same thing every night - check!
  5. I want to be able to kick off the music any time - this can be done, although it's not as easy.  I have to ssh to the Raspberry Pi and kick off the script manually.  However, if it's a silent day, nothing will happen.  I'll probably look at a way to adjust the script so that when it's called from CRON it does its normal thing but when called from command line it always plays music.
  6. I want to be able to manually override either the current song or the current volume level - This can also be done, but it's not as easy.  While in ramp up or ramp down mode, the volume doesn't increase/decrease by 1, it gets set to a particular level.  Overriding the volume during ramp up/down won't make any difference because as soon as the script loops it will reset the volume to right back where it was.  I may be able to change this in the future.  While playing, I can change the volume by logging in via SSH and issuing amixer -q set PCM X% where X is the volume I want to set to.  However, during ramp down, the volume starts back at 100% before dropping down.  It may not take much, but the next version may have an increment/decrement instead of fixed volume levels.  If I get this working, this objective will be met.
Some of the plans I have for the future are an automatic switch over to Christmas music instead of regular music during the month of December, web service based controls, & incremental/decremental volume controls.  We'll see when I have time to work on it.
at 0 comments More posts like this: , ,

Monday, May 20, 2013

Setting a default group for an NPC report page

UPDATE: An additional tool now available: autorefresh.html.  This widget uses its own script to handle automatically refreshing a page.  It does not call the built in Auto-Refresh capability of NPC.  You provide the number of minutes between refreshes in the browser view url: /custom/autorefresh.html?interval=5

UPDATE: An additional tool now available for download lets you set the default timeframe for a page. Do you have a page that you always want to show the last 24 hours of data? Just add this widget.

UPDATE: The newest version is out.  I updated the script to fix a problem when going back to NPC when a page using the default page context setter was used.  I also have released a modified version that allows you to set the default IP SLA test type on an IP SLA test report page.

Page IPSLA Type Default


Page Group Default


Post truncated. Click here to view the rest »
at 0 comments More posts like this: , , , ,

Friday, May 17, 2013

Interface Summary Table Ultimate Tweak

A while back, I did a major customization of the Top Least Interfaces Table in NPC.  This is a NetVoyant view that normally shows interface availability and utilization in and out of every interface.  There's no reason, however, that that table can't contain many more metrics.  That's essentially what I did with this customization.

In order to implement this run the following command on the NPC server:


That should do it. Now the default definition for that view should contain all the advanced metrics shown above. The view also has a new title when in the view list: "Interface Utilization Summary". The way you can know it'st he right one is by hovering over the view in the view list and it should pop up a description with my name in it.

This can also be applied directly to the NV view.  You can do it through the wizard, or just run the following command on the NV server:
at 0 comments More posts like this: , , , ,

Wednesday, May 8, 2013

NV Default Tweaks

To go along with my post about the default tweaks that I do to a vanilla SuperAgent (ADA) installation, I decided to go ahead and document my default tweaks for NetVoyant.  Note the disclaimer at the bottom of this page.  All of these tweaks should be done before the first discovery cycle begins.
  1. Add discovery scopes by network, not individual IP address.  This is a hot topic, but I maintain that using networks is better than individual IP addresses, if only for the sake of administration.  If you've configured DNS and discovery properly (see point 5 below) IP address changes won't require any intervention.  If you'd rather keep a super tight grip on your stack, go right ahead.
  2. Enable daily Periodic Discover: just a checkbox under discovery
  3. Tweak SNMP Timeout: Change the timeout from 5 seconds to 2.  If it hasn't responded after 2 seconds, it's not going to respond after 5.
  4. Enable Reachability Only Monitoring: If you want to monitor devices in scope but not SNMP capable, you can by only using ICMP.  Enable this by unchecking the box that says 'Ignore Non-SNMP Devices'.  You'll also need to go to Config>>Discovery>>Device Models and check the 'Enabled' checkbox on the 'NonSNMP Devices' model.
  5. Update Device Naming: This one takes some thinking.  If you know you will have DNS entries for all of your devices, the best would be to let NV poll via FQDN (vs. polling by IP address).  That way, if your discovery scopes include networks instead of individual IP addresses you won't have to change anything in NV when the IP address of a device changes.  Since NV will be polling via FQDN and the new IP address is still in scope, NV won't know any different.  Set Default device name to 'DNS Name'.  If there isn't one, NV will poll via IP address.
  6. Give NV more resources: Slide the resource usage slider up to its max.  If NV isn't the only thing on the server, do this carefully.
  7. Disable Undesired Classes: Under Discovery>>Device Classes disable any device classes you won't want to monitor.  This is one way you can prevent NV from monitoring everything on your network even though you've added scopes by network.  I typically disable Printers and workstations.  You will need to keep an eye on any SNMP capable devices that show up in the other group.  This means NV doesn't know what class the device belongs to.  Right click the device and click change classification.  If you need a new class, come to Config>>Discovery>>Device Classes and create it.  After you make a classification change, make sure your undesired classes still say "No Device Models Enabled Upon Discovery'.
    Tip: when you're reclassifying devices, you can set the icon that gets used by the NV console when displaying the device.  This is only for the console, but it can make things easier to troubleshoot.  You can either use one of the built in images (found at D:\netqos\netvoyant\classes\redpoint\images) or store your own there (keep it to less than 20x20 pixels) by entering the image name (without the .gif) in the change classification dialog box.
  8. Disable polling of the System Idle Process: If the Host Resources Software Performance (hrswrun) dataset is going to be used, setup a discovery rule called 'Default' with expression: 
    hrSWRunName <> 'System Idle Process'
    It's also a good idea to go ahead and set the poll event severity to none.  Otherwise you'll get an alarm every time a process fails to poll.  This can be a good thing, since it indicates that a process has gone down.  However, if NV is polling a process that is being run by a user, when the user logs off, the process will disappear.  In fact, I usually go through and disable poll events for all datasets.  This should be done understanding what is lost when not getting poll events.
  9. Disable Host Resource Device Table (hrdevice): Create a discovery rule called 'None' with expression:
    1==2
    If you've already discovered some/all of your devices, set the poll instance expiration to 0 and enable the 'None' discovery rule.  Then run a full rediscovery.  After that's done, disable polling and periodic discovery on that dataset.
  10. Disable VMware datasets: You will only get data for these datasets if you own CA Virtual Assurance.  If you do, skip this step.  If you don't, disable polling and periodic discovery for VMware Datacenter Element (aimdc), VMware Host (aimhost), and VMware Virtual Machine (aimvm).
  11. Disable NBAR and RMON2: if you have NBAR or RMON2 probes and want to poll them from NV, skip this step.  Otherwise, disable polling and periodic discovery for Protocol Distribution (NBAR) (nbarstats) and Protocol Distribution (RMON2) (protodist).
  12. Disable polling of optical, removable, and floppy drives: Add a discovery rule to the Host Resource Storage (hrstorage) dataset called 'Default' with expression:
    hrStorageType NOT IN ('1.3.6.1.2.1.25.2.1.7','1.3.6.1.2.1.25.2.1.5')
    If you've already discovered some/all of your devices, set the poll instance expiration to 0 and enable the 'Default' discovery rule.  Then run a full rediscovery.  After that's done, set the poll instance expiration back to something reasonable like 28.
  13. Disable polling of various interface types: Add a discovery rule called 'Default' with expression:
    ifInOctets+ifOutOctets<>0 AND ifType NOT IN (1, 18, 24, 134, 37, 100, 101, 102, 103, 104) AND ifSpeed<>0
    If you're curious about which interface types this excludes, look on the Config tab under Discovery>>Interface Types.
  14. Enable Verbosity on the Topology service: Go to Services>>Topology and change the drop down from 'Normal' to 'Normal (Verbose)'.  There's no save button.  Turn this back to 'Normal' after NV is up and running and stable in production.
  15. Disable Traps: If NV isn't going to be your trap handler, prevent stray traps from getting logged into the database by going to Services>>Traps and setting start mode to 'Manual'.  Then click 'Stop' to stop the service.
  16. Configure your view options: Under the View menu, make sure everything is enabled.
That's it for now.  Make sure the discovery monitor is open and kick off discovery.  That should get you started.

Here's a picture:

at 0 comments More posts like this: , ,

Tuesday, May 7, 2013

Default SuperAgent Tweaks

Whenever I'm setting up a new SuperAgent system, there are always a few things I go through and do before I start data collection.  So, here's my list so I don't have to remember it:

  1. Add collectors by DNS - I like to add by NetBIOS name then click the IP button.  This helps me make sure I get the right IP address.  Then, after SA has done its check of the collector, I click the DNS button, which finds the DNS name from the IP address it previously resolved from the NetBIOS name.  This  double check makes sure I have the right server since the FQDN name should be fairly similar to the NetBIOS name.
  2. Add a port exclusion - Given the troubles I've had with large deployments and auto-discovery, I've decided to start adding a huge port exclusion from the get go.  I add 1025-65535 for the whole domain.  When/if I need to monitor an application in that range, I can always add an exception.  This can be done via the GUI or through a query:
    insert into application_rules
    (application_id,exclude_port_begin,exclude_port_end,rule_type)
    values (0,1025,65535,0);
    New in ADA 9.3! - This option can be enabled on each collector.  For standard collector or virtual collector, create a new text file: drive:\CA\bin\saConfigInterfaceRuntimeOptions.ini with the following line:
    /force positive config
    Then restart the CA ADA Monitor service.
  3. Add actions to the default incident responses - add all the possible actions to the incident responses. If I have the email address of the person/group that will be monitoring the SA for problems, I put an email action in the collection device default incident response.
  4. Create a 'No Response' network incident response - create this incident response with no actions.
  5. Adjust network types - I like to have only 4 network types: Internet - VPN, LAN, Not Rated, WAN.  I delete all the other network types.  Assign the Internet - VPN and Not Rated network types to the 'No Response' incident response created earlier.
  6. Edit the 'Weekends' maintenance schedule - Change the name to 'Full Time Maintenance' and change the period to all day, every day.  If there is a standing maintenance window that affects every server everywhere, add that period to the default.
  7. Change the data retention settings - Bump everything up to their max.  If it becomes a problem later on, I can always tune it down.
  8. Change the free space alarm - Change this from 5GB to 20GB and put somebody's email address in there.
  9. Import a networks list - I prefer to use the GXMLG, but at least understand regions if doing it by hand.  You can also use the standard private networks list if you have nothing to start with.
  10. Bump up the default observation numbers
    New in ADA 9.3! - You don't have to do this via direct database manipulation any more.  Just go to Administration >> Policies >> Performance Thresholds.  The middle table allows manipulation of the default threshold settings.  You can also setup the default threshold settings for the 'Not Rated' and 'Internet - VPN' network types; set them up for no thresholds on the network and combined thresholds.
  11. Import servers as subnets instead of individual servers - This just makes sense.  If possible, try to group servers together into subnets by application.  This makes it easier to assign groups of servers to an application.  If this isn't possible, enter the entire subnet.  
Those are all the tweaks I can think of at the moment.  If I think of any others, I'll add them to this list.
at 0 comments More posts like this: , ,

SuperAgent Application Thresholds

SuperAgent thresholds are comprised of several different components.  The most critical part of the thresholds is the baseline sensitivity.  Out of the box, SA thresholds are applied to every application and are set with the sensitivity values dictated by NetQoS.  There are actually two types of thresholds that can be applied: sensitivity and milliseconds.

Doing anything in the database directly isn't supported by CA and you may break your stuff.  If you do, i'm not responsible and CA will probably have you revert to a db backup before even considering talking to you.  So either don't tell them that you did this or make sure you can backup and restore your database as needed.  There, you have been warned.

Sensitivity 

Sensitivity is a unit-less scalar number between 0 and 200. This type of threshold looks for deviations from baseline. A higher number (think 'more sensitive') will alert on a slight deviation. A lower number will not alert until the deviation is more extreme. Think of it as that sensitive co-worker who goes to HR for everything. If that person is very sensitive, any little thing will cause them to go to HR. If they were less sensitive, it would take something more extreme for them to march over and report you. Sensitivity baselines are really handy since the actual numbers involved in the threshold change as the baseline changes.  This means that if one day of the week is typically different than the other days, the baseline would reflect that.  Since the baseline reflects that, so do the thresholds for that day.  SuperAgent baselines take into consideration hour of the day and other factors to get very good baselines.  The other thing that SuperAgent does with regards to baselines is that it baselines every combination individually.  Since every combination has its own baseline, a single set of thresholds that refer only to the baseline can be set across the board.  This is how things come out of the box.


Milliseconds

The second type of threshold is a more traditional threshold that looks at the value and determines if it is over a specified value.  This threshold is much harder to set since you'd have to track data and understand what values you should set.  This type of threshold does have one advantage: baseline creep protection.

Baseline creep is when the baseline increases over time because of a slowly degrading performance.  Thresholds tied to that baseline would also slowly increase.  This is like boiling a frog.  You start out with a live frog in cool water and heat it up gradually.  By the time the water is hot enough to kill and boil the frog, it's too late for the frog to jump out.


Minimum Observation Count 

SuperAgent also takes into consideration the fact that a single observation of a transaction that exceeds a threshold (either sensitivity or millisecond) is nothing to pay attention to.  The problems really come into play when many observations are seen exceeding the threshold.  The minimum observation count is the number of observations that must exceed the threshold within a 5 minute period before the whole 5 minute period is marked as degraded or excessively degraded.  These numbers are quite low out of the box.  It is common practice to bump these numbers up (usually by a power of 10) in order to reduce the amount of noise that is reported by SA.  More on this later.


Default Application Thresholds 

When an application is configured, either by a user or by the system, a default set of thresholds is applied.  The same settings are used for all applications.  This can be a problem with newer SA systems since auto-discovery tends to create many applications.  If they are all using the default thresholds, it can result in much noise.  This is not because the thresholds are too low.  Remember, the default thresholds are tied to the baseline.  The real problem is that the default minimum observation numbers are too low.  Luckily, these numbers can be changed.


Changing Thresholds Through the Web GUI 

The thresholds and minimum observations can be changed in the GUI through two different places.  In the applications list or under policies.  The applications list is the better place to be if you want to change more than one application/network type set at a time.  In the applications list, multiple applications can be selected (maximum of 100 applications selected at a time) and the thresholds edited for all those applications.  This may be handy at least for editing the thresholds of the user created applications.

New in ADA 9.3! - A new option has been added to the GUI that allows the modification of the default threshold for new applications (new system discovered applications and new user defined applications).  Go to Administration >> Policies >> Performance Thresholds.  The middle table allows modification of the default threshold set.  You should also go back to applications that have already been defined and update those thresholds.  Once an application is discovered by the system or created by the user, the thresholds are independent of the default set.

Changing Thresholds Through a MySQL Query 

When changing the thresholds for the system applications, there are several tactics.  The first involves increasing the minimum observation count.  This can be done with a fairly simple query that both increases the minimum observation count for all defined applications but also modifies the default application thresholds so that all future applications use the same settings.

 --run this query to increase the minimum observation count by a power of 10.
update performance_incident set observations = observations * 10;

 You shouldn't have to reload the collectors to get this change to take effect, however if you do experience problems seeing the updated threshold values, reloading the collectors should fix it.


Setting Thresholds for Internet/VPN Network Type 

A best practice when configuring SuperAgent is to configure a special network type for all the network definitions in SA whose network performance is not entirely within your control.  Alarming on networks like this is ineffective since the resulting alarms are inactionable.  I usually create a network type called 'Internet - VPN' to indicate any networks that are entirely or partially out of my domain of control.  In other words, I set the network type to 'Internet - VPN' for any client IP address ranges across the internet or on another organization's network.  If I were to detect a problem with the network metrics to a user within one of these networks, I wouldn't know if the problem were within my portion of the network or out on the internet.  If it were out on the internet, I wouldn't be able to do much about it.

So, first of all, create the 'Internet - VPN' network type and assign all your non-internal IP address ranges to it.  This would include VPN IP addresses since a portion of their conversation occurs over the internet.

The next step is optional, since the third step negates its necessity.  However, if you don't want to go ahead with the third step, implementing this step will at least prevent you from getting alerts on the network metrics for those networks.  All that you need to do is create a new network incident response for the 'Internet - VPN' network type and don't assign any actions to it.  This should weed out email notifications from issues detected for networks where you can't help the network performance.

New in ADA 9.3! - A new option has been added to the GUI that negates having to perform step three using direct database manipulation.  Instead, go to Administration >> Policies >> Performance Thresholds.  Click 'Add Custom by Network Type' in the second table.  Pick the 'Internet - VPN' network type.  Change the Network and Combined thresholds from 'Use Default' to 'Customize' then change the now enabled drop downs from 'Sensitivity' to 'None'.  You'll want to do this for NRTT, RTD, NCST, ERTT, DTT, and TTT.

Step three involves a little database manipulation.  Essentially, you will need to add a record to the performance_incident table for every metric/app combo you want to ignore.  Since you'll need to ignore NRTT, RTD, NCST, ERTT, DTT, and TTT, you'll need to add 6 rows for every application.  Luckily, this isn't too hard.  The only downside is that this doesn't set things up for any future applications.  You'll have to repeat the process.  If you do, the query will fail unless you do a complete undo of everything else first.  This first query undoes all the threshold sets for the network type containing the string 'VPN'.  Make sure your network type has this string or modify the query below.

 -- run this query to remove any thresholds currently tied to that network type
Delete from performance_incident where agg_id = (select max(agg_id) from aggregates where agg_type=1 and agg_name like '%VPN%');

 Once you've done that, or if this is the first time you're running this, run the following query.  Again, make sure your network type has the string 'VPN' in the name.  Essentially, this inserts a row ignoring thresholding for the VPN network type (hence the 0's in the query below right after m.metric_type) for every application and for each of the metrics we want to ignore (hence the last set of numbers).

-- run this query to disable network and combined metrics for the network type whose name contains the string: VPN
INSERT INTO performance_incident (app_id, agg_id, metric_type, thres1, thres1_type, thres2, thres2_type, observations)
SELECT a.app_id, (select max(agg_id) from aggregates where agg_type=1 and agg_name like '%VPN%'), m.metric_type, 100, 0, 90, 0, 50 as observations
FROM applications as a, metric_types as m where m.metric_type in ( 0 , 1 , 2 , 3 , 4 , 9 );
at 0 comments More posts like this: , ,

Tuesday, April 16, 2013

Finding the data source for a particular device in NPC

Recently, we needed to know which data source was contributing to the report data for a particular device in NPC.  This was fairly easy to find out given a simple query:

mysql -P 3308 -D netqosportal -e "select a.itemname as Device, v6_ntoa(a.address) as Address, b.consolename as DataSource from dst_device as a, data_sources2 as b where a.sourceid=b.sourceid and itemname like '%devicename%' order by a.itemname;"

Simply replace devicename with the device name and execute this at a command prompt on the NPC server.  The result should look something like this:
+------------------------+-----------------+----------------------+
| Device                 | Address         | DataSource           |
+------------------------+-----------------+----------------------+
| center                 | 192.168.100.2   | NetVoyant            |
| nacogdoches            | 192.168.100.3   | NetVoyant            |
| nacogdoches            | 192.168.100.3   | ReporterAnalyzer     |
| houston                | 192.168.100.4   | ReporterAnalyzer     |
| houston                | 192.168.100.4   | NetVoyant            |
| dallas                 | 192.168.100.5   | ReporterAnalyzer     |
| dallas                 | 192.168.100.5   | NetVoyant            |
| sanfelipe              | 192.168.100.6   | ReporterAnalyzer     |
| sanfelipe              | 192.168.100.6   | NetVoyant            |
| austin                 | 192.168.100.7   | ReporterAnalyzer     |
| austin                 | 192.168.100.7   | NetVoyant            |
| elpaso                 | 192.168.100.8   | NetVoyant            |
| brownsville            | 192.168.100.9   | NetVoyant            |
| beaumont               | 192.168.100.10  | NetVoyant            |
| lufkin                 | 192.168.100.11  | NetVoyant            |
| ftworth                | 192.168.100.12  | NetVoyant            |
| ftworth                | 192.168.100.12  | ReporterAnalyzer     |
| tyler                  | 192.168.100.13  | ReporterAnalyzer     |
| tyler                  | 192.168.100.13  | NetVoyant            |
| henderson              | 192.168.100.14  | NetVoyant            |
| amarillo               | 192.168.100.15  | NetVoyant            |
| amarillo               | 192.168.100.15  | ReporterAnalyzer     |
| sanantonio             | 192.168.100.16  | NetVoyant            |
| bexar                  | 192.168.100.17  | NetVoyant            |
+------------------------+-----------------+----------------------+

It would be nice if NPC or the new CAPC had some kind of feature that showed the datasource(s) for a particular object on the device details page.
at 0 comments More posts like this: , , , ,

Thursday, April 4, 2013

Raspberry Pi

Today I joined the latest craze sweeping the Internet.  No, not Vadering (although I should look into how that could be done with the boys).  I bought two brand new computers today.  It cost me a total of $70 + shipping.  Well, actually, I didn't buy the whole computer, I just bought the motherboard, which has the processor, RAM, video card (with HDMI and analog RCA out), audio card, SD card reader, onboard NIC and 2x USB ports.  It's all built into a special little PCB (printed circuit board) that takes advantage of SOC (system on a chip: basically the main guts of a computer all on one chip, similar to most tablets and phones).

It's called Raspberry Pi and is produced by a non-profit organization that designed them to be used in developing countries for education.  Since it's a charity, they designed the board to be extremely cheap and can sell it for $35!  They're currently being distributed through three reseller partners officially.  Each reseller also sells in bulk to other companies so you can find them elsewhere.  The main drive for resellers is the accessories that go a long with it (more about this later).  RS (only in Europe), Allied Electronics, and Element14 (through Newark) are the official resellers.  For more information, check out the video below or read this article.

I ordered two of them today because for a long time both model A and model B were out of stock everywhere.  My dad reminded me last night that they were back in stock so I got one for him and one for me this morning.  Here's a good video to introduce what's really behind it.
It should be here within a few days.  All that actually comes in the box is the board, no case, no cables, no nothing.  So, I'll go dumpster diving into my crates of extra parts in the garage for a 700mA USB power supply (I wonder if the dual USB plug for my portable hard drive would allow this to be powered from my USB hub...), HDMI to VGA converter so I can plug it into my desk monitor, a USB keyboard and a USB mouse (maybe I'll use my Bluetooth dongle and the Logitech diNovo Mini keyboard...), and an SD card.  I think I'll start by running Rasbian Wheezy, just to see how well it performs normal desktop tasks.  After that I'm going to use it as a test bed for XBMC with RaspBMC to potentially replace my current Windows Media Center dependent installation.

Anyway, I'll try to post my progress as things go along.  Eventually, I'll be looking into putting this thing into a case and see what happens.  Here are some of the cases I'm looking at

I might even consider making a competitor to the iMac:
 versus 

at 0 comments More posts like this: , ,

Wednesday, March 6, 2013

Page Retirement Methods

Continuing my effort to document the various ways I've used the ODBC connector for the NetQoS products, here's my next query and controls I've built and that I use in production.  Today's query comes from a need to notify users that a particular page is being retired. Instead of immediately deleting a page and having to deal with confused users, I used the browser view to display a banner telling the users that they have X days until the page is retired.  This could be done manually by creating a different static HTML page display a different date for each page, but I wanted a way to make it easy.
First of all, I didn't want to have to rebuilt the banner for each page that is going to be retired.  So, I needed a way that each instance of the banner have a way to store an individual expiration date.  I also wanted a way for users who don't want their precious pages deleted to notify me.  A simple email would have done, but that would have almost always involved an immediate ping back to the user asking which page.  So, the banner needs a way to start an email and automatically put the URL to the page.
The implementation is pretty easy.

  1. First of all, get this first section of code, update 'email@company.com' to your email address on the fourth to last line and save it as pageexpiring.html in your custom content directory.  Download a copy of the code here.
  2. Then put a browser view on the page to be retired and set the URL to point to pageexpiring.html in your custom content directory and save the view.  When the page refreshes, you should see a message saying that the page has been scheduled to be retired on an unknown date.  
  3. To add a specific date, edit the browser view and append ?expirationdate=3/31/2013 to the URL (replacing 3/31/2013 with the page retirement date).
  4. Repeat steps 2 and 3 for any remaining pages to be retired.


Now that you've marked a couple pages to be retired, you can use an ODBC connection to display all those pages.  Here's the SelectCommand and OdbcConnection String to put in the configuration.xml:


To create the view, run the following SQL commands against the NPC server:
at 0 comments More posts like this: , , ,

Thursday, February 14, 2013

NetVoyant Device Inventory through ODBC

Continuing my effort to document the various ways I've used the ODBC connector for the NetQoS products, here's my next query and controls I've built and that I use in production.  Today's query comes from a need to export a list of all devices polled by NetVoyant to CSV.  This can usually be done by going to Inventory >> Devices and exporting to CSV from there.  However, that control has a limit of 5000 devices and we're monitoring more than that.  Luckily ODBC doesn't have that limitation.  Here's the SelectCommand and OdbcConnection String to put in the configuration.xml:


To create the view, run the following SQL commands against the NPC server:

at 0 comments More posts like this: , , ,

Wednesday, February 13, 2013

The Law of Averages

The Law of Averages is not actually a law, but really just a lay term for some mathematical principles either completely unfounded in any mathematical law or borrowed from the Law of Large Numbers.  For more information, see this Wikipedia article.  However, this post is titled 'The Law of Averages' because I wanted to go into some detail about averaging averages.

In particular, I was asked this question: "If I have a data point for every minute of the data, I also have an average of 15 data points for every 15 minute period of the day, and I wanted to calculate the daily average, it would be better to use the 1 minute data versus the 15 minute averages (aka rollup data).  The average of the 15 minute averages would really be an average of averages and is therefore not accurate."

The reality is actually contrary to that thinking.  The average of averages of data is exactly the same as the average of the same source data.  To illustrate this, let's look at some data.

Below, we have random numbers between 0 and 100 to represent our data.  We have one data point for each minute of the day.  Every 15 minutes, an average is calculated.  For example, at 0:30 an average is calculated averaging the points from 0:30-0:44.

Time1 Minute Data15 Minute Average
0:003751.6666666666667
0:0174
0:0249
0:0321
0:0459
0:0515
0:0632
0:0769
0:0840
0:0926
0:1087
0:1138
0:1296
0:1379
0:1453
0:156025.8
0:1635
0:177
0:1824
0:1946
0:203
0:2122
0:221
0:2339
0:249
0:2534
0:260
0:2742
0:2834
0:2931
0:30433.8
0:315
0:3281
0:3336
0:3435
0:359
0:3636
0:3741
0:3876
0:3993
0:4021
0:4112
0:4210
0:4321
0:4427
0:45548.8
0:4642
0:4783
0:4848
0:4918
0:5079
0:5124
0:5278
0:5319
0:545
0:5552
0:56100
0:5739
0:5870
0:5970
1:001444.0666666666667
1:0145
1:0282
1:0334
1:0474
1:0590
1:0633
1:0710
1:0843
1:0924
1:1096
1:1112
1:1217
1:1362
1:1425
1:159654.7333333333333
1:1619
1:1791
1:1858
1:1976
1:2024
1:2186
1:2235
1:2348
1:2431
1:2567
1:2672
1:279
1:2845
1:2964
1:307940.4
1:3168
1:320
1:3310
1:3430
1:3523
1:3665
1:3774
1:3853
1:3978
1:404
1:413
1:4238
1:4378
1:443
1:459147.6
1:4685
1:4744
1:484
1:496
1:5013
1:5139
1:5293
1:5352
1:5435
1:5549
1:5669
1:5767
1:5864
1:593
2:001156.8666666666667
2:0172
2:0254
2:0345
2:0416
2:0517
2:0675
2:0796
2:0896
2:0962
2:1080
2:1155
2:1296
2:1334
2:1444
2:157346.6666666666667
2:1657
2:1721
2:1861
2:1928
2:2011
2:2116
2:221
2:2386
2:2423
2:2564
2:2689
2:2746
2:2838
2:2986
2:304245.8
2:3115
2:327
2:333
2:3440
2:3541
2:368
2:3766
2:381
2:3985
2:4059
2:4189
2:4294
2:4342
2:4495
2:456457.6
2:4658
2:476
2:4814
2:4917
2:50100
2:5185
2:5272
2:5393
2:5458
2:55100
2:5632
2:5753
2:5836
2:5976
3:00438.7333333333333
3:0119
3:0232
3:0314
3:0484
3:0526
3:0617
3:0765
3:0820
3:0915
3:1026
3:1152
3:1279
3:1331
3:1497
3:15855.6666666666667
3:1647
3:1787
3:1866
3:1923
3:2088
3:2128
3:2253
3:2387
3:2466
3:2532
3:2671
3:2787
3:2810
3:2982
3:308454.9333333333333
3:3181
3:3254
3:3388
3:3477
3:3598
3:362
3:3715
3:3892
3:3972
3:4016
3:4145
3:424
3:4360
3:4436
3:458463.4666666666667
3:4697
3:4787
3:4829
3:4953
3:5011
3:5184
3:5283
3:5376
3:5464
3:5550
3:5636
3:5783
3:5838
3:5977
4:00364.1333333333333
4:0179
4:0290
4:0391
4:0446
4:0553
4:0626
4:0777
4:0846
4:0962
4:1060
4:1193
4:1278
4:1392
4:1466
4:153537.1333333333333
4:1681
4:1742
4:1850
4:1914
4:2026
4:2172
4:224
4:2315
4:2465
4:2514
4:269
4:2710
4:2838
4:2982
4:301451
4:3148
4:3262
4:3336
4:3489
4:353
4:3653
4:3758
4:3886
4:3971
4:4049
4:4167
4:4220
4:4396
4:4413
4:45047.3333333333333
4:4691
4:4764
4:4828
4:4996
4:5023
4:5137
4:5237
4:5377
4:5415
4:5513
4:5698
4:577
4:5858
4:5966
5:004848.6
5:0122
5:025
5:0376
5:0496
5:055
5:0610
5:073
5:0834
5:0956
5:1082
5:1196
5:1258
5:1380
5:1458
5:154848.1333333333333
5:1630
5:1750
5:1871
5:1912
5:2012
5:2198
5:2224
5:2341
5:2421
5:2566
5:2624
5:2799
5:2873
5:2953
5:30837.3333333333333
5:3167
5:3217
5:330
5:340
5:3537
5:3651
5:3733
5:3828
5:3958
5:403
5:4174
5:4279
5:4390
5:4415
5:455655.7333333333333
5:4682
5:4766
5:4864
5:4969
5:5094
5:5181
5:5223
5:5322
5:5441
5:5516
5:5689
5:5747
5:5874
5:5912
6:007049.2666666666667
6:0172
6:0276
6:0328
6:0462
6:0528
6:0646
6:0762
6:0861
6:0934
6:1052
6:1169
6:1212
6:1337
6:1430
6:157951.8666666666667
6:1635
6:1774
6:1871
6:1912
6:2052
6:2172
6:226
6:2389
6:2416
6:2517
6:2630
6:2764
6:2876
6:2985
6:304742.8666666666667
6:3144
6:3246
6:3346
6:342
6:3583
6:3612
6:3724
6:3847
6:3994
6:4032
6:4170
6:4270
6:4324
6:442
6:457754.2
6:461
6:4727
6:4847
6:4918
6:5075
6:5146
6:5283
6:5358
6:5424
6:5558
6:5681
6:5765
6:5857
6:5996
7:003466.8666666666667
7:0199
7:0279
7:0375
7:0489
7:0592
7:0663
7:0747
7:0849
7:0993
7:1045
7:1140
7:1297
7:1341
7:1460
7:155652.0666666666667
7:1671
7:1792
7:1831
7:1968
7:201
7:213
7:2264
7:2321
7:247
7:2582
7:268
7:2799
7:2896
7:2982
7:309751.4
7:315
7:321
7:3382
7:3467
7:3530
7:3629
7:3789
7:3853
7:3980
7:4014
7:4188
7:4273
7:4336
7:4427
7:455347.4666666666667
7:4647
7:4755
7:484
7:4946
7:5065
7:5147
7:5221
7:5325
7:5451
7:5587
7:5624
7:5746
7:5875
7:5966
8:006341.5333333333333
8:0146
8:0269
8:0358
8:0414
8:0521
8:0683
8:0775
8:0823
8:0912
8:1060
8:1163
8:126
8:139
8:1421
8:155340.6
8:1642
8:1722
8:180
8:1965
8:206
8:2166
8:2216
8:2385
8:248
8:2578
8:2644
8:2786
8:2830
8:298
8:309868.3333333333333
8:3181
8:3293
8:3366
8:3460
8:3527
8:3684
8:3762
8:3881
8:3976
8:4094
8:4119
8:4260
8:4338
8:4486
8:459247.7333333333333
8:4639
8:4718
8:4849
8:4936
8:5057
8:5159
8:5284
8:538
8:5436
8:5514
8:5648
8:5751
8:5867
8:5958
9:007344.7333333333333
9:0190
9:0259
9:032
9:041
9:052
9:0633
9:0751
9:0833
9:0981
9:102
9:1193
9:1279
9:1318
9:1454
9:158748.8666666666667
9:1675
9:1733
9:1819
9:1924
9:2014
9:2145
9:2213
9:2393
9:2454
9:2569
9:2612
9:272
9:2899
9:2994
9:304244.4666666666667
9:3170
9:3283
9:3320
9:3466
9:3510
9:3635
9:3774
9:3818
9:3935
9:407
9:4164
9:4267
9:4355
9:4421
9:454853.4
9:4691
9:4788
9:483
9:4911
9:5045
9:518
9:5220
9:5396
9:5417
9:5576
9:5678
9:5754
9:5899
9:5967
10:001853.5333333333333
10:0152
10:0233
10:0319
10:0465
10:0588
10:0656
10:0799
10:0818
10:0998
10:1050
10:1146
10:1242
10:1353
10:1466
10:156158.5333333333333
10:1684
10:1766
10:1844
10:1943
10:208
10:2126
10:2277
10:2355
10:2485
10:256
10:2691
10:2766
10:2866
10:29100
10:307446.3333333333333
10:3170
10:3210
10:3372
10:3435
10:3519
10:3686
10:3772
10:387
10:3931
10:4095
10:4135
10:4246
10:432
10:4441
10:454546.3333333333333
10:4685
10:473
10:4811
10:4948
10:5090
10:5159
10:5240
10:5326
10:549
10:5594
10:567
10:5770
10:5831
10:5977
11:006747
11:0178
11:0259
11:030
11:0470
11:0577
11:0671
11:0749
11:0851
11:0963
11:1021
11:1121
11:1250
11:1311
11:1417
11:15446.9333333333333
11:1654
11:179
11:1849
11:1930
11:2082
11:2141
11:2222
11:2333
11:2429
11:2578
11:2688
11:2737
11:2884
11:2964
11:302347.4666666666667
11:3150
11:328
11:3350
11:3424
11:3584
11:3653
11:3778
11:3880
11:392
11:4061
11:4189
11:4277
11:4316
11:4417
11:451936.2666666666667
11:4664
11:4785
11:489
11:4930
11:5024
11:515
11:5210
11:5359
11:547
11:5542
11:5621
11:5769
11:5855
11:5945
12:003748.0666666666667
12:0160
12:0285
12:0344
12:0428
12:0589
12:0635
12:0718
12:0848
12:0964
12:103
12:1138
12:1296
12:135
12:1471
12:157146.4
12:1693
12:1737
12:187
12:1938
12:2085
12:2128
12:2286
12:2312
12:2465
12:2538
12:266
12:2766
12:2824
12:2940
12:302642
12:3198
12:3274
12:3313
12:3446
12:3552
12:3639
12:3714
12:3838
12:3925
12:4028
12:4130
12:4244
12:4363
12:4440
12:459641.0666666666667
12:4613
12:4750
12:488
12:4943
12:5027
12:5138
12:5297
12:530
12:5411
12:5584
12:5673
12:5711
12:5820
12:5945
13:003960.1333333333333
13:0194
13:0295
13:0382
13:0439
13:0526
13:0680
13:0770
13:0819
13:0979
13:1074
13:1178
13:1216
13:1320
13:1491
13:155846
13:1667
13:1752
13:1855
13:1947
13:2025
13:218
13:2278
13:2353
13:2455
13:2543
13:269
13:2728
13:2856
13:2956
13:302627.7333333333333
13:315
13:3217
13:332
13:3453
13:359
13:365
13:3763
13:3891
13:3946
13:407
13:4123
13:4225
13:433
13:4441
13:459449.0666666666667
13:4670
13:4749
13:4860
13:4959
13:5051
13:5171
13:5216
13:5337
13:5431
13:5513
13:5664
13:5776
13:5830
13:5915
14:00032.5333333333333
14:016
14:027
14:0313
14:0452
14:0525
14:0653
14:0715
14:0865
14:0970
14:1072
14:1112
14:1223
14:1323
14:1452
14:158750.9333333333333
14:1698
14:1724
14:1860
14:1932
14:2031
14:2110
14:2263
14:2328
14:2420
14:2542
14:2690
14:2743
14:2881
14:2955
14:306053.7333333333333
14:3145
14:3298
14:3396
14:3411
14:3514
14:363
14:3797
14:3863
14:3929
14:4023
14:4196
14:4235
14:4389
14:4447
14:456758.4666666666667
14:4641
14:4766
14:4870
14:4998
14:5047
14:5137
14:5268
14:5361
14:5416
14:5570
14:5619
14:5746
14:58100
14:5971
15:009756.7333333333333
15:0165
15:0299
15:0360
15:0454
15:053
15:0660
15:0779
15:0830
15:0950
15:1037
15:1154
15:1251
15:1363
15:1449
15:152743.0666666666667
15:1610
15:1777
15:1882
15:1915
15:2055
15:2168
15:2274
15:2311
15:240
15:2572
15:2649
15:2762
15:2841
15:293
15:305464.8666666666667
15:3186
15:3295
15:3331
15:3445
15:3571
15:36100
15:3759
15:3843
15:3915
15:4088
15:413
15:4297
15:4388
15:4498
15:457649.7333333333333
15:4617
15:4782
15:4834
15:4964
15:5085
15:5152
15:5229
15:5360
15:5476
15:5516
15:5612
15:5764
15:5852
15:5927
16:008442.8
16:0110
16:0211
16:0313
16:0499
16:0529
16:0664
16:0729
16:0820
16:0964
16:1036
16:1114
16:1238
16:1369
16:1462
16:159652.0666666666667
16:1639
16:1710
16:1880
16:1988
16:2014
16:212
16:224
16:2354
16:2425
16:2578
16:2688
16:2799
16:2889
16:2915
16:307356
16:3168
16:3229
16:3343
16:3426
16:3554
16:3613
16:37100
16:3889
16:3933
16:4048
16:4135
16:4292
16:4356
16:4481
16:453050.1333333333333
16:4699
16:474
16:48100
16:495
16:5010
16:5137
16:5212
16:5392
16:5499
16:5595
16:5669
16:5720
16:5852
16:5928
17:004544.3333333333333
17:0127
17:0220
17:0335
17:0457
17:0588
17:0691
17:077
17:0866
17:0941
17:1020
17:1132
17:1277
17:138
17:1451
17:153853.6
17:1636
17:1739
17:1840
17:1915
17:2091
17:2145
17:2237
17:2399
17:2498
17:2554
17:2664
17:2797
17:2848
17:293
17:305550.6
17:3133
17:3271
17:3340
17:3465
17:3580
17:3646
17:3757
17:3843
17:393
17:4070
17:4130
17:4273
17:4363
17:4430
17:457967.4
17:4667
17:4749
17:4875
17:4998
17:5078
17:5183
17:5283
17:5354
17:5477
17:5546
17:5652
17:5766
17:5864
17:5940
18:009558.8666666666667
18:01100
18:0283
18:0361
18:049
18:0511
18:0635
18:0740
18:0875
18:0980
18:1014
18:1182
18:1237
18:1372
18:1489
18:154845.6666666666667
18:1640
18:1790
18:1827
18:1910
18:2065
18:2152
18:224
18:2329
18:2423
18:2529
18:2690
18:2792
18:281
18:2985
18:306263.2
18:3146
18:3299
18:3350
18:3461
18:3574
18:3690
18:3745
18:3891
18:39100
18:4093
18:4167
18:4219
18:4316
18:4435
18:458658.9333333333333
18:4664
18:4799
18:4826
18:4999
18:5019
18:5131
18:5293
18:5311
18:5474
18:5588
18:5636
18:5756
18:5868
18:5934
19:001869.3333333333333
19:0197
19:0226
19:0384
19:0498
19:0553
19:0679
19:0795
19:0872
19:0992
19:1024
19:1195
19:1286
19:1396
19:1425
19:155058.8
19:1688
19:1795
19:1820
19:1985
19:2041
19:2139
19:2245
19:2367
19:2439
19:2563
19:2698
19:272
19:2861
19:2989
19:309057.8666666666667
19:3174
19:3284
19:3398
19:3483
19:3525
19:3692
19:3772
19:3826
19:3934
19:4048
19:412
19:4270
19:437
19:4463
19:453853.7333333333333
19:4612
19:4744
19:4823
19:490
19:5080
19:5188
19:5288
19:5393
19:5434
19:5569
19:5615
19:5795
19:5885
19:5942
20:003735
20:01100
20:023
20:0317
20:045
20:0544
20:0699
20:0729
20:08100
20:092
20:105
20:1111
20:1225
20:1324
20:1424
20:156340.7333333333333
20:1681
20:1790
20:1844
20:1961
20:2018
20:2137
20:226
20:2385
20:241
20:2553
20:263
20:2754
20:285
20:2910
20:30954.4666666666667
20:3168
20:3289
20:3369
20:3474
20:3550
20:3684
20:3715
20:3858
20:3980
20:4030
20:4169
20:4286
20:4311
20:4425
20:45045.4666666666667
20:4649
20:4729
20:4865
20:4926
20:5062
20:5141
20:5265
20:5327
20:5478
20:5570
20:5617
20:5790
20:585
20:5958
21:002449.4
21:0195
21:0223
21:0316
21:040
21:0525
21:0670
21:0770
21:0885
21:0981
21:1081
21:1190
21:1244
21:139
21:1428
21:156753.1333333333333
21:1689
21:1754
21:1811
21:1917
21:2063
21:2197
21:2288
21:2395
21:2463
21:2565
21:2656
21:2713
21:283
21:2916
21:301455.8
21:3183
21:322
21:3373
21:3481
21:3570
21:365
21:3784
21:3863
21:3988
21:4033
21:4149
21:4290
21:4375
21:4427
21:454353.7333333333333
21:4667
21:4761
21:4817
21:4949
21:5032
21:51100
21:5293
21:5393
21:5453
21:5523
21:5616
21:574
21:5876
21:5979
22:004739.6
22:0146
22:029
22:0392
22:0467
22:052
22:0639
22:0738
22:0852
22:091
22:1015
22:1117
22:1278
22:1314
22:1477
22:157043.8666666666667
22:1658
22:1754
22:1871
22:190
22:2068
22:2110
22:2279
22:2350
22:2457
22:2549
22:260
22:2750
22:285
22:2937
22:305046.8
22:3127
22:3254
22:3339
22:341
22:3588
22:3623
22:3781
22:3870
22:3954
22:4039
22:4152
22:4233
22:432
22:4489
22:45558.9333333333333
22:4678
22:4739
22:4882
22:4979
22:5073
22:5136
22:5275
22:5381
22:5471
22:5540
22:5633
22:5790
22:5838
22:5964
23:001451.2666666666667
23:0127
23:0293
23:0380
23:0412
23:0598
23:069
23:0767
23:0887
23:0910
23:1015
23:1169
23:1260
23:1360
23:1468
23:152159.4666666666667
23:1686
23:1776
23:1821
23:1981
23:2031
23:2171
23:2268
23:2390
23:2463
23:258
23:2685
23:2795
23:2829
23:2967
23:304546.2666666666667
23:3171
23:3273
23:3358
23:3423
23:3516
23:3616
23:3779
23:3884
23:39100
23:404
23:4123
23:4255
23:4325
23:4422
23:45845.4
23:4627
23:4749
23:4822
23:4984
23:5046
23:5124
23:5219
23:5353
23:5432
23:5572
23:5669
23:5796
23:5820
23:5960
Here's the funny part.  The question that was posited essentially asked if the average of the 1 minute data would be more accurate than the average of the 15 minute averages.  Here are the results:

Average of all the 1 minute data points: 49.76319444
Average of all the 15 minute averages: 49.76319444

They're exactly the same!  The reason for this comes from the associative property of addition.  Think about the formula for an average:
x̄ = (a1+a2+a3+...+an)/n

If you had multiple sets (i.e. 15 minute averages):
x̄ = {(a1+a2+a3+...+an)/na + (b1+b2+b3+...+bn)/nb + (c1+c2+c3+...+cn)/nc}/3

Since in this case na=nb=nc, then substitue n and factor out:
x̄ = {(a1+a2+a3+...+an) + (b1+b2+b3+...+bn) + (c1+c2+c3+...+cn)}/3n

Associative property of addition:
x̄ = (a1+a2+a3+...+an+b1+b2+b3+...+bn+c1+c2+c3+...+cn)/3n

Since in this case 3n = n + n + n and na=nb=nc=n, then substitue for n:
x̄ = (a1+a2+a3+...+an+b1+b2+b3+...+bn+c1+c2+c3+...+cn)/(n + n + n)
x̄ = (a1+a2+a3+...+an+b1+b2+b3+...+bn+c1+c2+c3+...+cn)/(na + nb + nc)

Knowing that na + nb + nc is actually the total number of original points gets us back to the original equation for calculating the average of the 1 minute data.  So the average of all the data points is the same as the average of evenly spaced averages of all the data points.  If you're wondering if this works, go ahead and mock it up in Excel.  I did.
at 0 comments More posts like this:
Subscribe to: Posts (Atom)