RSA likes to proclaim that the Security Analytics system can do "real-time alerting and correlation". I want examples/samples for EXACTLY how this is done in a meaningful way.
- Correlation rules exist on decoders but I've had PS resources tell me they're not very efficient and extremely limited. Thus we have them all removed.
- ESA Series IV gear lacks sufficient memory for our goal of hundreds of ESA alerts.
- ESA uses isolated streams and thus really only works well with events that have all the meta contained in them for the alerting.
Here is an example, say I want to look at incoming IDS alerts provided via logs. Now say that I want to alert when we get this IDS event but only if I don't see specific packet events that correlate with the source and destination IP's seen in the IDS log. Also I want to do this within a "reasonable" amount of time like 5 minutes.
One definite challenge I see is that if we process the packet data before we process the log data, it'll fail as the query in ESA looks at the log event FIRST. Either way the flow of data needs to be real-time for this to pan out and that assumes no congestion/processing delays of any kind.
If I try to do app rules or anything of that nature, again, I have to correlate across the infrastructure and potentially within 5-10 minute time windows.
A Hadoop cluster with sufficient nodes and hardware could do queries in parallel, but then I'd still need a method to join that data and compare.
Thoughts?
