2017-03-17 11:15 AM
Archiver retention days. I want to generate an alert if archiver not retaining logs for XXX days.
I am new to RSA and from Server monitoring background.
I know perl and PHP as well
RSA is all new so if someone can tell where to start from that will be helpful.
I can see this information by going into each archiver and see no of days its retained.
2017-04-13 06:05 PM
Your best bet is to use the REST API on the Archiver service. You can use any scripting language to retrieve the information you are looking for, such as perl, python or PHP. When it comes to retention it is important to realize that data retention is made up of four parts: oldest packet, oldest meta, oldest session and oldest index. Which ever has the youngest date is how far back your retention actually goes.
So if you had the following:
packet (logs): 2017-Feb-01 14:25:00
meta: 2017-Feb-10 14:25:00
session: 2017-Feb-03 14:25:00
index: 2017-Feb-02 14:25:00
Your overall retention would go back only to 2017-Feb-10 14:25:00 as it is the youngest start date.
I'll show you how to get the default stats for these four. You'll need to look over your own environment to determine the exact path to find these numbers for yourself.
Oldest Packet (Log): https://<archiver ip>:50108/archiver/collections/default/database/stats/packet.oldest.file.time?msg=get&force-content-type=text/plain&expiry=600
Oldest Meta: https://<archiver ip>:50108/archiver/collections/default/database/stats/meta.oldest.file.time?msg=get&force-content-type=text/plain&expiry=600
Oldest Session: https://<archiver ip>:50108/archiver/collections/default/database/stats/session.oldest.file.time?msg=get&force-content-type=text/plain&expiry=600
Oldest index: https://<archiver ip>:50108/archiver/collections/default/index/stats/time.begin?msg=get&force-content-type=text/plain&expiry=600
These commands will produce a result in the format like: 2017-Feb-07 14:25:00
When running the commands above it will request a username and password to log in. The credentials you use are stored on the service itself. They can be found under Administration -> Services -> <service> -> Actions -> View -> Security. I highly suggest that if you are going to use this kind of script connection that you create a under privileged account on the service so it has limited access. Also know that your REST API may not use SSL by default. If this is the case and you want SSL enabled you will need to Explore the service in the UI and go to rest -> config -> SSL and change this to on. You'll then need to restart the service for it to take effect.
All Netwitness core services (broker, concentrator, log decoder, packet decoder, archiver, log collector) has this REST API that can be used. To access it you need to change the service port in the HTTP statement from above.
Broker: 50103
Concentrator: 50105
Log Decoder: 50102
Log Collector: 50101
Packet Decoder: 50104
Archiver: 50108
I hope this helps.
2017-04-24 12:50 PM
I appreciate the suggestion of John. However, my opinion is that having a builtin monitoring system (H&W) you would probably want to get an alarm in there rather than running some external scripts.
It is a shame that prior 10.6.3 collectd is not configured to read the mentioned stats such as meta.oldest.file.time, packet.oldest.file.time, session.oldest.file.time. I had to add these metrics to the collectd config manually and write a custom collectd module to calculate a database retention.
However, RSA has improved that in 10.6.3, so these stats along with the Overall (Meta/session/packet) Oldest File Time Retention of core services have been introduced. You can setup a monitoring policy OOB.
2017-04-24 12:55 PM
Thanks Maxim for updating on this.
I do read this in new section of release notes for 10.6.3.
I will work towards this solution and alarm as well once we upgrade our RSA devices to 10.6.3.
Thanks
Kapil Batra
Sr. Analyst, Operating Systems
Infrastructure Management Framework Support
Desk:513-784-2647
Mobile:904-610-4230
Email:Kapil.batra@convergys.com