Esper Named Windows - Do they survive stopping and starting the ESA Service

[david_waugh]

Hello

I am trying to implement

https://community.rsa.com/community/products/netwitness/blog/2016/08/18/trend-analysis-with-the-netwitness-suite 

but we have a backup job that stops the ESA Service every night, takes a backup and then restarts the service.

I thought that having @RSAPersist in the windows eg:

CREATE WINDOW baselineEvery5MinutesData.win:time(60 minutes) (metaKey string, metaValue string, eventCount long);
@RSAPersist
CREATE WINDOW baselineEvery1HourData.win:time(30 days) (metaKey string, metaValue string, eventCount long, timeframe string);
@RSAPersist
CREATE WINDOW baselineEvery1DayData.win:time(30 days) (metaKey string, metaValue string, eventCount long, timeframe string);
/* data structure to store the baseline statistics. The last stat only is kept and is updated at each iteration */
@RSAPersist

Would mean that when the ESA service stopped the last snapshot of data would be saved and reused again when starting up.

However, when I look in my named windows before and after stopping the service, then the results are different.

Can anyone confirm if named windows contents ado persist between service restarts?

I am using 10.6.4.1

[EricaChalfin]

cyBsFLJHwdf0d3ZrBhE5v6oJm6n4YwQ0dT3KoI34m3M=‌,

I've moved your question to the https://community.rsa.com/community/products/netwitness?sr=search&searchId=a39d6f85-f940-49a4-8327-626314bb8ce5&searchIndex=0‌ product space so you can get an answer to your question.

Regards,

Erica

[VolodymyrRozhni]

Hi David,

Its kind of off topic, how did you solve the issue with sessions behind? or you don't have this problem in your environment

[NikolayKlender]

I am not from RSA but from my point of view @RSAPersist is do what you need

[AnkushBaveja]

As per documentation - "All windows definitions are not suitable for persistence. @RSAPersist annotation must be used with care. If the window has timed-records or if it depends on time based constraints it is very likely that the reverted snapshots will not restore it to the correct state."

I am not sure how you're inserting into these window(s) but i guess since you are base-lining that is why snapshot does not revert to exact same values before the snapshot. 

[david_waugh]

Hi I solved this.

This was due to a mistake in my understanding.

The original context was:

CREATE CONTEXT contextEvery1Day INITIATED @now AND PATTERN [every timer:at(0, 0, *, *, *)] TERMINATED AFTER 1 day;

And the rule said something like:

/* every day aggregate data from the baselineNONBusinessHoursEvery1HourData window for each meta value */
@Name('baselineNONBusinessHoursAggregateEvery1Day') CONTEXT contextEvery1Day
INSERT INTO baselineNONBusinessHoursEvery1DayData
SELECT metaKey, metaValue, SUM(eventCount) AS eventCount, 'day_'||current_timestamp().getDayOfWeek().toString() AS timeframe , current_timestamp().format() AS timestamp
FROM baselineNONBusinessHoursEvery1HourData
GROUP BY metaKey,metaValue
OUTPUT snapshot WHEN TERMINATED;

What this means is that the context would start at 00:00 and then run for one day. At the end of this period, the results snapshot would be created.

However during this one day window I had my backup job running which meant that the results never got produced as it would never reach the end of the context.

To solve this I changed the context to

CREATE CONTEXT contextEvery1Day INITIATED @now AND PATTERN [every timer:at(0, 0, *, *, *)] TERMINATED AFTER 5 min;

This means that the context would start at 00:00 and then only run for 5 mins. At this point all the data for the previous day had already been collected, so there is no need to wait a further full day before carrying out the calculation.

This assumes that your ESA is aggregating in realtime and is not more than 5 mins behind.

You could change the length of the window to take into account any delays.

[david_waugh]

Hi I don't have an issue with sessions behind in my environment. I'd open a ticket with RSA Support for assistance.

[david_waugh]

Hi I have opened a case. @RSAPersist does not seem to be working in 10.6.4.1

This is being tracked under internal reference  SACE-8301.

I can see the snapshot files being written to disk, but on restarting the rsa-esa service they do not seem to be being read.

[david_waugh]

Hi @RSAPersist does not seem to be working.

Basically the issue is that if I disable aggregating from devices on the ESA, and then restart the ESA Service the named windows are restored correctly. If aggregating from the devices on the ESA is enabled, then restarting the ESA service does not always restore the named window.

I think that if events are received before the named window is restored then the named window is recreated without the old information.