Folks
While working with engineering to troubleshoot some ESA
weirdness we found that in 10.5 there is a default setting that is wrong.
If it is not changed you may experience ESA restarts as well as the output from
rules that may not have the expected results.
First a little background:
The Esper engine lives in a state of ‘NOW’. The Esper
engine has no sense of time, everything happens ‘now’. The only way
the Esper engine knows what time it is, is by SA telling it. Esper
does NOT read and use the system clock.
The way SA tells Esper what time it is, is by instructing
the Esper engine to read and use the ‘EventTime’ field in the sessions. The
sessions are ordered by time as they are sent into the Esper engine so life is
mostly good.
To clarify what
exactly this EventTime is… When ESA receives an event from a
concentrator, it stamps it with current time and adds a esa_time meta to the
session. By default, that’s the meta we pass to Esper as time and referenced in
the explorer as EventTime
When you create a rule, for example, looking for 5 failed
logins in 5 minutes, it is the event time field that causes Esper to know that
5 minutes later is really 5 minutes later.
If the Esper engine was not told what time it was (by using
the Event Time) it would never advance the clock. The result would be
Esper would never know when 5 minutes had passed. In addition to this,
since Esper would not know when 5 minutes had elapsed it would continue to grab
and hold onto all the failed logins until it ran out of memory and reset.
Let’s take the above scenario and assume that 5 million
failed logins had been seen and held on to since Esper doesn’t know about time,
even though 10 hours had passed. If you ‘asked’ esper what time the
first event got seen, the answer would be ‘Now’, and then if you asked what
time the 5 millionth event got seen it would also answer ‘Now’.
The other issue is ‘Followed By’ never happens as all events
seen by the Esper engine happened ‘Now’ as well.
As you can see, using the Event Time to advance the Esper
Clock is very important.
This brings me to the point of the email.
In 10.4.x and before the setting of ‘UsingEventTime’ under
ESA -> Explore -> CEP ->Engine
-> cepEngine has always been
set to ‘true’. However in 10.5 the default was inadvertently set to
‘false’. This ‘false’ setting unfortunately was maintained through the
upgrade process and even into ver 10.6.
You should check all of your systems to make
sure that a ‘true’ settings is Set.
After making this change you need to restart the ESA
service.
Hope this helps
Dave
