Retention calculation via Rest API for Log Data

[Anonymous]

All,

Most of the posts on this forum are Packet Centric, I'm posting this to help others on the Log Data side that are trying to get REST apps to work.

I ran this in a cygwin environment but this should work well in any UNIX environment too.

#!/usr/bin/perl
use Time::ParseDate qw(parsedate);

@stat_string =
`/usr/bin/curl --user "xxx:xxx" "http://concentrator_iphttp://concentrator_ip:50105/database/stats:50105/database/stats"`;
foreach $date (@stat_string) {
    if ( $date =~ /meta.oldest.file.time/ ) {
        ( $Year, $Month, $Day ) =
          ( $date =~ /([0-9]{4})-([A-Z]{1}[a-z]{1,4})-([0-9]{2})/ );
        print "Year is $Year Month is $Month Day is $Day\n";
    }
}

$d1 = "$Day-$Month-$Year";
$d2 = `date  "+%d-%B-%Y"`;

print "The oldest file on record  is '$d1' current date is '$d2'\n";

printf "Currently retaining %d days worth of log data\n",
  ( parsedate($d2) - parsedate($d1) ) / ( 60 * 60 * 24 );

This should print out the amount of days that the SA 10.x environment is retaining.

[NaushadKasu1]

For a Concentrator, you cannot simply look at the meta.oldest.file.time, because the Concentrator has several DBs that could have less data than what is available in the metadb.  For example, there is the 'index', 'sessiondb' and 'metadb' -- you have to take the shortest timeframe from each of the 3 metrics and that is your retention as the lowest one is your bottleneck.

A clear example is when you have 60+ days of metadb but the customer is indexing so many keys with high volume values that their index may only have 30 days of indexed against the metadb.  Thus, for queries in the Investigator view -- you are limited to searches (fast) against that unless you do a full metadb search which takes forever (raw log search etc...).