ESA Rule - replace user_src in Alert

[BohdanR]

I am creating EPL rule, where I want to take 2 type of events - one type has user.src and the second type has different identification of user in custom meta, for which I add user.src using a custom feed, but it can have also empty user.src if the feed doesn't match on it.

Then I need to group generated alerts into incident grouped by user.src.

The problem is that no incidents are generated for alerts having no user.src when I am grouping by user.src in the incident. Thus I want to use the custom meta (secondary user identification) in cases when user.src is empty.

I got to an idea to set user_src to value of custom meta in EPL in case when user_src doesn't exist, but I don't know how to achieve that. Is there any way how to do it?

I tried for example this:

SELECT *, coalesce(user_src,custom_meta_name,'UNKNOWN_USER') as user_src  FROM Event( some filters);

But that won't deploy, as user_src is returned twice in the result. Listing also all possible keys from Event instead of *, while replacing user_src with coalesce expression also doesn't sound like the best way.

Then I tried saving the result of SELECT to EventStream and update the user_src in EventStream (using UPDATE command) before using it to create event by SELECT * from EventStream, but that didn't reploy too (problem was in UPDATE command, it probably doesn't work in EPL like in SQL)

Any solution for this?

[JoshRandall]

I think it might be better to stick with your initial approach, and just make sure your alerts always have the user_src (and/or custom_user) meta that you need to group your incidents with.

To that end, I believe modifying the respond engine's javascript and aggregation schema files could be a good solution here.  A general how-to on this process: https://community.rsa.com/community/products/netwitness/blog/2018/06/21/creating-custom-group-by-and-match-condition-fields-for-respond-server

For your use case, there's an existing function within the normalize_core_alerts.js script that you can copy and repurpose for your custom_user and user_src metas.  The function that you'll base yours off of is this one:

/**

* get normalized domain from event.fqdn event it exist. Otherwise it will get domain from event.alias_host

* @param event

* @returns {*}

*/

getDomain = function (event) {

    return event.fqdn ? event.fqdn : event.alias_host;

};

…and your version could look like:

/**

* get normalized user from event.custom_user if it exists. Otherwise it will get user from event.user_src

* @param event

* @returns {*}

*/

getUser = function (event) {

    return event.custom_user ? event.custom_user : event.user_src;

};

Add this function to the file, and then within the generateEventInfo function in the same normalize_core_alerts.js file, you would add:

generateEventInfo = function (headers, rawAlert, event) {

    var normalizedEvent = {

        timestamp: getTimeStamp(event),

        ….snip….

        custom_user: Utils.stringValue(getUser(event)),

        ….snip….

    };

Next up, in the normalize_alerts.js file, add a line into the normalizeAlert function, like so:

function normalizeAlert(headers, alert) {

    var transformer = null;

…snip…

        normalized.groupby_source_ip = Utils.generateFlattenedColumnValue(normalized.events, "source.device.ip_address");

        normalized.groupby_source_country = Utils.generateFlattenedColumnValue(normalized.source_country);

        normalized.groupby_custom_user = Utils.generateFlattenedColumnValue(normalized.events, "custom_user");

        ….snip….

        return normalized;

    }

}

And then, in the aggregation_rule_schema.json file, add:

  {

    "value": "alert.custom_user",

    "name": "Custom User",

    "type": "textfield",

    "operators": [0, 1, 8, 9, 10, 11, 12, 13],

    "groupBy": true,

    "groupByField": "alert.groupby_custom_user"

  },

Once you've made all these changes, restart the respond-server:

# systemctl restart rsa-nw-respond-server

After it restarts and you refresh your browser, you will be able to select "Custom User” in either/both the matchCondition and groupBy drown-down menus in your Incident Rule. This "Custom User” field should now always have a value, and so your incident grouping should not fail.

[BohdanR]

Thank you for your reply Joshua, but that is not what I was hoping to find. I don't want to modify JS files as it is quite painful to maintain that - they get replaced with every NetWitness update. And I want to have just one rule for this.

I hope there is a way how to replace Event.user_src in results of EPL query in case when it is empty, or eventually do it by some other way (like LUA parser, if they get executed after Feeds to be able to determine whether the feed successfully matched to the user or not).

I suppose there should be some simple command to do it in EPL, just don't know which one...

[LeeKirkpatrick]

Hey Bohdan,

Here is where we could make use of an expression. We pass the values from an EPL query to said expression, and run over a CASE statement to check if the value is NULL and populate the associated SELECT clause accordingly:

CREATE EXPRESSION checkUser
{i =>(
	SELECT case Event.user_src
	WHEN NULL THEN 
		Event.my_meta
	ELSE 
		Event.user_src
	END
	FROM Event.win:length(1))
};

SELECT checkUser(checkThis) as user_src FROM Event checkThis‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The below shows this logic in action, you can see the first Event outputs the user_src as admin, whereas the second outputs the my_meta custom value as there is no user_src contained within the Event:

Cheers,

Lee

[BohdanR]

Hi Lee,

I believe that your solution gives similar output like coalesce function. The problem here is that when I create an alert using your command:

SELECT checkUser(checkThis) as user_src FROM Event checkThis

The resulting alert will have only user_src on it and all other metas from the event get lost. When I use SELECT * from Event, I will have all metas in the alert, but can't replace user_src together with *. Otherwise I would need to list all metas instead of * individually, with using the expression on user_src there, but I don't know what all metas may be in the event to list them all here...

[LeeKirkpatrick]

Hey Bohdan,

What if you directly manipulated the istream like this? That way you could keep the SELECT * and manipulate user_src on the fly:

@Name('Modify')
update istream Event
set user_src = my_meta
where user_src IS NULL;‍‍‍‍‍‍‍‍

Cheers,

Lee