ESA rules - adding events to alert

[BohdanR]

I am creating ESA Rules, but I see that alert generated by these rules usually contains only one event, not all events that participated in creation of the alert. I would like to add all related events into the alert for some of the rules.

Example of one of the rules:

@RSAAlert(oneInSeconds=0)
@Hint('reclaim_group_aged=90000')
SELECT * FROM Event(
<some filters here>
)
GROUP BY (esa_time).withTime(0, 0, 0, 0), user_src
HAVING COUNT(*) = 10 ;‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This rule triggers an alert when we receive 10 events of specific type in one calendar day. The alert contains only 10th event.

How can I add all 10 events to the alert?

I know that one of the possibilities is using batch window or time batch window for accumulating the events until specific amount or time is reached and then releasing them all into the alert. Is there any other way how to achieve that?

[LeeKirkpatrick]

Hey Bohdan,

You will need to use the window(*) syntax to output all events collected for the grouping:

@RSAAlert(oneInSeconds=0)
@Hint('reclaim_group_aged=90000')
SELECT window(*) FROM Event(
<some filters here>
)
GROUP BY (esa_time).withTime(0, 0, 0, 0), user_src
HAVING COUNT(*) = 10 ;‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Cheers,

Lee

[BohdanR]

Hi Lee,

Thank you, that is useful.

Although, if I use it like that, I get error during deployment: ExprValidationException: Failed to validate select-clause expression 'window(*)': The 'window' aggregation function requires that the aggregated events provide a remove stream; Please define a data window onto the stream or use 'firstever', 'lastever' or 'nth' instead

It seems that it needs adding something like .win:time(1 day) before GROUP BY to define the window. Is it the most optimal way how to write such rule? I mean, if 1 day windows won't eat up too much resources?

[LeeKirkpatrick]

Ah yes, that is correct.

The time a window is open is not of much concern, it is how much data is inserted into that window that is. As long as you are restrictive on what you insert into the window there will be no issue.

Cheers,

Lee

[BohdanR]

OK, I see.

So when I use the timed data window, I can remove "@Hint('reclaim_group_aged=90000')", is that correct?

[LeeKirkpatrick]

That depends on whether you want that functionality or not, that can still be applied here if needed.

Cheers,

Lee

[BohdanR]

OK, what functionality does it bring? Does it have any advantages to have reclaim_group_aged defined in this case?

[LeeKirkpatrick]

It instructs the Esper engine to discard grouped data that has not been updated for the number of seconds supplied. It is there to help with performance as it removes data from the window when it has not been updated for the number of seconds supplied.

Cheers,

Lee