ESA Rule - event A not preceded by event B

[TamasSzilagyi]

Hi, I have a case when I want to create an alert for a specific event only if another event did not preceded that specific event. To give more context: if an email gateway 'reputation' event happened for an IP, the firewall alert rule for that same IP should not trigger. Or, looking from the other way around, the firewall rule should only trigger if there weren't any email gateway 'reputation' events before for the same IP.

I used the Esper reference - event patterns docs and found this:

A pattern that takes all A events that are not preceded by B within 5 minutes:

every (timer:interval(5 min) and not B -> A)

So I wrote a rule like that:

SELECT * FROM PATTERN [Every (
		timer:interval(1 min)
		AND NOT
		B=Event(
		device_type.toLowerCase() IN ( 'email_gateway' )
		AND
		result.toLowerCase() IN ( 'reputation' )
		)
		->
		A=Event(
		device_class.toLowerCase() IN ( 'firewall' )
		AND
		ip_src = B.ip_src
		AND
		isOneOfIgnoreCase(action,{ 'accept' })
		)
	)
];‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The syntax is valid, but the alerts did not trigger.

What could be the issue?

Edit: small addition, NW version 11.2.

[JoshRandall]

I really like the idea here, but the logic in your A event is preventing the match and alert, specifically the attempt to correlate A.ip_src=B.ip_src.

This is failing because your B event never occurs. And because there is no B event, there is no B.ip_src value for the A.ip_src to match.

In order to alert using this kind of correlation, one option is to hold the B.ip_src values in a window and then compare the A.ip_src value to the B.ip_src in order to (not)match/alert.

e.g.:

@Name('create emailGateway window')
@RSAPersist
CREATE WINDOW emailGateway.win:time(1 minutes) (ip_src string);

@Name('fill emailGateway window with ip_src')
INSERT INTO emailGateway
SELECT
	ip_src as ip_src
	FROM Event(
	medium IN ( 32 ) AND device_type IN ( 'email_gateway' ) AND result.toLowerCase() IN ( 'reputation' )
	);

@Name('alert when firewall events ip_src does not match emailGateway window ip_src')
@RSAAlert
SELECT * FROM Event(
    medium IN ( 32 ) AND device_class.toLowerCase() IN ( 'firewall' ) AND isOneOfIgnoreCase(action,{ 'accept' })
    ).win:length(1), emailGateway
    WHERE Event.ip_src NOT IN ( emailGateway.ip_src )
;‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[TamasSzilagyi]

Thank you Josh! It really does make sense - of course there isn't a B.ip_src, that's what I wanted in the first place!  Silly me.

I was playing around with the rule you wrote, and it does work technically - but aesthetically... The alert contains all the information that was used for the trigger, here is an example:

So the firewall log that supposed to fire the alert has the IP of 181.111.111.114

The unknown .111 (two logs had this ip_src) and .113 were the email_gateway logs before.

It seems that the alert includes each attempt when email_gateway IP that was in the window is being compared to the firewall log. I do understand this, but I have to find a way to make this presentable for a lvl1 analyst. The real alert will likely have much more than a couple email_gateway IP-s in the window.

[TamasSzilagyi]

Now that I had some time to test this method thoroughly I've realized the problem that causes the issue is not in the presentation of the data, but rather how it is being created (rule issue). Looking at the previous example I used, if I had .111, .112 and .113 IPs in the "emailGateway window", not just the firewall log with .114 source IP will trigger it but... anything even if it had .111, .112 or .113. as a source. In these cases the output will only contain the "comparisons" that are "true" so every IP in the "emailGateway window" except one.

So basically the engine does not consider this as one true/false statement

SELECT * FROM Event(
    medium IN ( 32 ) AND device_class.toLowerCase() IN ( 'firewall' ) AND isOneOfIgnoreCase(action,{ 'accept' })
    ).win:length(1), emailGateway
    WHERE Event.ip_src NOT IN ( emailGateway.ip_src )
;
‍‍‍‍‍

...but rather one true/false statement for each comparisons when using emailGateway.ip_src.

Also, i want to leave this here as a note: limiting the output rows is actually not that hard using things like "output first every 1 events" or "limit 1" at the end of the alert.

[TamasSzilagyi · Accepted solution]

I since figured it out:

SELECT * FROM Event(
	medium IN ( 32 ) AND device_class.toLowerCase() IN ( 'firewall' ) AND isOneOfIgnoreCase(action,{ 'accept' })
    ).win:length(1)
    WHERE NOT EXISTS ( SELECT ip_src FROM emailGateway WHERE emailGateway.ip_src = Event.ip_src);‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This will result with only the event that triggered the alert.

Thanks Josh for the original idea! Really helped me! 

[JoshRandall]

Nice! I didn't even think to use the window for output limiting.