Event Stream Analysis (ESA) rule is disabled after being deployed in NetWitness 10.6 and higher

[#1]

Article Number

000001866

Applies To

NetWitness Product Set: NetWitness Platform
NetWitness Product/Service Type:  Event Stream Analysis (ESA)
NetWitness Version/Condition: 11.3 and Higher version
Platform: CentOS 7 / Alma

Issue

In 11.X version, An ESA rule is disabled after being deployed to the ESA service and reports the error below.
 

ESA was unable to deploy one or more rules, and these rules were disabled.  Common issues include:  missing metadata, invalid rule syntax, and unavailable external connections at the time of deployment. 

 

Image description
 

The ESA log level WARN contains the following message:
 

Implicit conversion from datatype 'String' to 'String[]' is not allowed

 

Image description
 

Image description

In 12.3 version, The deployed rule with an array error, shows the error in the UI under the ESA Deployment stats. Navigate to Policies>Content>hover over the policy status and it will show below errors.
Image description

ESA  /var/log/netwitness/correlation-server/correlation-server.log shows below errors:

Caused by: com.espertech.esper.compiler.client.EPCompileException: Failed to validate filter expression '"google"=any(alias_host) or alias_h...(51 chars)': Collection or array comparison and null-type values are not allowed for the IN, AN
Y, SOME or ALL keywords [@RSAAlert(oneInSeconds=0)         SELECT * FROM Event(             /* Statement: finding array */             (( 'google' = ANY( alias_host ) ) OR alias_host IN ( 'login' ))                      )]

 

Cause

Within the ESA service, some meta keys were changed from a string type to a multi-valued type. This affected the following rules:
 

Rule #	Rule Name	Array Type Meta Keys in 11.3
1	RIG Exploit Kit	threat_category
2	AWS Critical VM Modified	alert
3	Multiple Successful Logins from Multiple Diff Src to Same Dest	host.src and host.dst
4	Multiple Successful Logins from Multiple Diff Src to Diff Dest	host.src and host.dst
5	Multiple Failed Logins from Multiple Diff Sources to Same Dest	host.src and host.dst
6	Multiple Failed Logins from Multiple Users to Same Destination	host.src and host.dst
7	User Login Baseline	host.src and host.dst

Resolution

To change the string type meta keys to string array type meta keys in NetWitness Platform 11.3, see “Configure Meta Keys as Arrays in ESA Correlation Rule Values” in the ESA Configuration Guide for RSA NetWitness® Platform 11.3 

NetWitness 11.3:

To deploy custom ESA rules using the above listed meta keys, the rules must be updated to use the array syntax and then redeployed.  For example:
 

String Syntax	Array Syntax
threat_category = 'rig'	'rig' = ANY(threat_category)

If you had any of the above listed rules deployed before 11.3, note any rule parameters that you have changed in order to adjust the rules for your environment. Download the updated rules from Live. Reapply any changes to the default rule parameters and deploy the rules. (For instructions, see “Download RSA Live ESA Rules” in the Alerting with ESA Correlation Rules User Guide for RSA NetWitness® Platform 11.3 

NetWitness 11.2 and Prior:

To deploy Live ESA rules using these keys, the meta keys must be added to the ESA service using the multi-valued type.  In addition, any custom ESA rules using these meta keys must be updated to use array syntax.  The steps below explain how to add the meta keys to the ESA service with the multi-valued type.

1. In the NetWitness UI, go to Administration > Services > ESA > Explore > Workflow > Source > nextgenAggregationSource > ArrayFieldNames.

Image description

2. In the ArrayFieldNames property, enter the meta keys separated by commas.  Be sure to use underscores for multi-word meta keys.Image description
3. Restart the ESA service using below command.

service rsa-nw-correlation-server restart