Data Retention Scheduler best practices

[RichardB]

What are the best practices when using the Data Retention Scheduler for NW Packet decoders/concentrators? We typically set the retention to 30 to 90 days. The default for "Run" is every 15 minutes which seems quite low/often in our case.

The documentation at https://community.rsa.com/docs/DOC-80855 does not explain the impact of choosing Interval or Date & Time. Is there a difference in performance? When set to 15 minutes, I imaging the process runs every 15 minutes and removes 15 minutes of old data every time. When set to Weekends the process will run once (twice really) a week but will remove 7 days of old data. Is one of these strategies preferred over the other?

[JohnSnider]

For most customers we generally don't even configure it, as they want to keep as much data as possible with available storage, and the data will roll off the storage as it nears capacity (~95% used)

However there are specific situations where it is useful, especially where there are legal limitations on how long you can keep data.  

If you are required by law to only keep active data for no more than 14 days, then you need to schedule at least a daily roll-off of the data, but may need to be more specific and run it every 15 minutes to make sure you are not violating any data privacy laws.

If you are not using it for legal matters, but to keep your data "aligned" on all devices so they have the same retention length, then it's purely up to you as to when and how often you run it, the timeroll process does not cause any real detriment to the operation of the system and if there are no "full files" to delete, does nothing.

[KevinArunski]

There are no strict best practices around how often the scheduler when doing time-based retention policies.   It's designed to accommodate the retention policies the customer needs to implement, rather than being a portion of the product that requires performance tuning.

There's no need for concern around checking for rollover every 15 minutes.  The Netwitness services are designed to roll-out files more often than that.  Furthermore, there's no need to wait for the weekends before checking the time retention, unless of course that is the policy your organization wishes to implement.

There is no meaningful performance difference between choosing an interval or date/time.

[RichardB]

Thanks for clarifying this! Yes, we do need to explicitly set the retention due to GDPR. I'll leave the rollover to 15 minutes.