A new feature that ships with NextGen 9.8 is the historical stats database. The basic idea is almost every stat for each NextGen service will be stored in a database that can be queried for later retrieval. The queries you can perform are all time based. So, for instance, you can query for Decoder's line rate during the night and chart it. Each query can include or exclude stats to reduce the amount of data that is returned.
Currently, the clients that ship with NextGen 9.8 do not use the stat db. But a future version of Security Analytics will use it for charting and other innovative uses.
It's very easy to query this database either over our REST interface (http://<hostname>:<port>/sys?msg=statHist) or using NwConsole or Administrator's explorer view. Everything is done using the statHist message on the /sys node. A simple way to experiment with REST is to grab my python script here:
https://community.emc.com/message/662711#662711
An example is worth a thousand words, so let's show how to grab a Decoder's line rate for one minute last night. This results are coming from our internal decoder:
timeFormat=simple time1="2012-09-21 02:00:00" time2="2012-09-21 02:01:00" include=/decoder/stats/capture.rate,time
[ /decoder/stats/capture.rate=19 time="2012-Sep-21 02:00:22" ]
[ /decoder/stats/capture.rate=20 time="2012-Sep-21 02:00:23" ]
[ time="2012-Sep-21 02:00:24" ]
[ time="2012-Sep-21 02:00:25" ]
[ /decoder/stats/capture.rate=19 time="2012-Sep-21 02:00:26" ]
I removed some of the records to keep this post brief, but I wanted to highlight something during seconds 24 and 25. Notice how capture.rate was not returned for those seconds? The reason is the capture rate didn't change from second 23. Since it remained at 20 for a couple of seconds after second 23, the stat db did not record it to save space. Please keep that in mind when plotting or using stats that are missing from various time records.
What if a stat rarely changes, how do you find the value for all those missing entries? Every 5 minutes on the 5 minute mark, all stats are recorded to the database. Start your queries at a 5 minute mark and you will be able to plot stats that rarely change.
Here's the help for this command:
statHist: Retrieve historical stats from the stats db. Don't send time1/time2 to get bounding times about stats db. Supported wildcards are ? to match any single char, * to match zero or more characters, not including slash /, ** to match zero or more characters including slash /.
security.roles: sys.manage
parameters:
time1 - <date-time, optional> The starting time for retrieving stats
time2 - <date-time, optional> The ending time for retrieving stats
timeFormat - <string, optional, {enum-one:posix|simple}> Specify the time format for each stat snapshot, default is posix (seconds since 1970)
include - <string, optional> Comma separated list of stats to include (wildcards allowed)
exclude - <string, optional> Comma separated list of stats to exclude (wildcards allowed, has precedence over include)
reduce - <bool, optional> If true, reduces data transmission by replacing stat pathnames with a shorthand and provides a lookup table as first result
If you have any questions, please reply to this post.
Thanks,
Scott
