Configure Advanced Settings for an ESA Correlation Service

These procedures are optional and they apply only to ESA Correlation Rules.

In the Explore view for an ESA Correlation service, you can manage sending ESA rule alerts to the Respond view, turn on debugging for all rules, configure the events to preserve for rules with multiple events, and configure meta keys as string array values on ESA.

Access Advanced Settings for an ESA Correlation Service

  1. Go to netwitness_adminicon_25x22.png (Admin) > Services.
    The Services view is displayed.
  2. In Services view, select an ESA Correlation service and then select netwitness_ic-actns.png > View > Explore.
    The Explore view is displayed.

netwitness_121_correxplvw_1122_672x315.png​

Enable or Disable Sending ESA Rule Alerts to the Respond View

ESA gathers data, runs ESA Correlation rules against the data, captures events that meet rule criteria, and creates alerts for those captured events. You can view those alerts in the Respond view.

Before an ESA Correlation rule alert can go to the Respond view, both of the following settings must be enabled:

  1. For all rules, the ESA Correlation service must have the respond-enabled parameter set to true. (The default is true.)
  2. For an individual rule, the ESA Correlation rule must have the Alert option selected in the rule builder for that rule.

To enable or disable alert forwarding to the Respond view for ALL ESA Correlation rules:

  1. In the Explore view node list for an ESA Correlation service, select correlation > alert.
    netwitness_121_correxplresp_1122_672x350.png
  2. To allow all ESA Correlation Rule alerts to go to the Respond view, set respond-enabled to true. Alerts for ESA rules that have the Alert option selected are visible in the Respond view.
  3. To stop all ESA Correlation Rule alerts from going to the Respond view, set respond-enabled to false.
    ESA Correlation Rules do not go to the Respond view, even if you select the Alert option in the rule.

    4. The changes take effect immediately.

Note: The respond-enabled parameter is equivalent to the Forward Alerts On Message Bus option in the Event Stream Analysis service in NetWitness Platform version 11.2 and earlier.

To send or not send alerts to the Respond view for a single ESA Correlation rule:

Content experts managing the ESA Correlation rules can decide whether to send alerts to the Respond view for each rule.

  1. Go to netwitness_configureicon_24x21.png (Configure) > ESA Rules > Rules tab.
    The Rules tab is displayed.
  2. In the Rule Library, select the rule you want to edit and click netwitness_ic-edit.png.
    Depending on the rule type, the respective rule tab is displayed.
    netwitness_121_rulealertresp_1122_768x434.png
    • To turn on Respond alerts for a rule, select the Alert checkbox.
    • To turn off Respond alerts for a rule, clear the Alert checkbox.
  3. Click Save.

    For more information, see the Alerting with ESA Correlation Rules User Guide.

Enable ESA Correlation Service Debugging for All Rules

You can turn on debugging for all ESA rules to see if rules are creating (firing) alerts and data is being processed properly by the ESA Correlation service. This can also be helpful when writing or fixing global notification templates, such as syslog or email. You can see the actual content of an alert before sending the notification.

When you disable ESA Correlation service debugging for all rules, you can still turn on debugging for an individual rule at any time.

  1. In the Explore view node list for an ESA Correlation service, select correlation > rule.
    netwitness_121_logfiredrules_1122_672x333.png
  2. Set log-fired-rules to true to print alerts to the /var/log/netwitness/correlation-server/correlation-server.log for troubleshooting. This is the same as the Debug option in the rule builders for individual ESA rules except that this option enables debugging for all rules.
  3. When you are ready to turn off debugging for all ESA rules, set log-fired-rules to false.
    The changes take effect immediately.

Note: The log-fired-rules parameter is equivalent to the Debug Rules? option in the Event Stream Analysis service in NetWitness Platform version 11.2 and earlier.

Configure Maximum Events per Alert for All Rules

  1. In the Explore view node list for an ESA Correlation service, select correlation > rule.
    netwitness_121_maxconstevents_1122_672x328.png
  2. For rules that contain multiple events, in max-constituent-events, enter how many of the associated events to preserve. For example, if a rule fires an alert with 200 associated events and this parameter is set to 100, only the first 100 are preserved by ESA, the rest are dropped. The default value is 100.
    The changes take effect immediately.

Note: The max-constituent-events parameter is equivalent to the Max Constituent Events option in the Event Stream Analysis service in NetWitness Platform version 11.2 and earlier.

Adjust Maximum Sessions for the ESA Data Source Filter

Note: This procedure applies only to NetWitness Platform version 11.5 and later.

In NetWitness Platform 11.5 and later, you can add an optional data source filter to the data sources in your ESA rule deployments to improve performance. This allows your data sources to be filtered further so that only the data relevant to the deployment is forwarded to ESA. The filter is comprised of application rules, which are applied to the Decoders mapped to your selected data sources.

Caution: The data source filter is intended for advanced users familiar with Decoder application rules. Improper filtering can cause the required data to not be forwarded to and analyzed by ESA.

Using a data source filter can be performance intensive for data aggregation. A filter slows the event aggregation rate, but when you are filtering a large amount of traffic, it can have performance benefits on ESA Correlation server. However, if you use a complex filter and do not filter a large amount of traffic, the event aggregation rate may be lower than expected.

11.5 Adjust Maximum Sessions for the ESA Data Source Filter

When filtering out a large portion of the traffic, you may see an "Invalid header size" error while communicating with Core services in the ESA Correlation log file. (You can use SSH to get in the system and go to: /var/log/netwitness/correlation-server/correlation-server.log). Lower the max-sessions parameter until you no longer see the error in the log. The more you filter out the traffic, the lower you should set the max-sessions parameter.

  1. In the Explore view node list for an ESA Correlation service, select correlation > stream.
    netwitness_121_esamaxsessions_1122_672x380.png
  2. In max-sessions, lower the value until you no longer see the error in the ESA Correlation log file. The default value is 10000.
  3. Restart the ESA Correlation service. Go to netwitness_adminicon_25x22.png (Admin) > Services, select the ESA Correlation service, and then select netwitness_ic-actns.png > Restart.

IMPORTANT: If an application rule linked to a data source filter is modified on a Decoder, the filter must be removed, added again, and redeployed. The changes take effect on ESA after the deployment is redeployed.

For more information, see “(Optional) Add a Data Source Filter” in the Alerting with ESA Correlation Rules User Guide.

Configure Meta Keys as Arrays in ESA Correlation Rule Values

A common reason for an ESA rule to generate an error during deployment is because a meta key in the rule is a string array type, but it shows as a string type on ESA. To prevent or fix this issue, do the following:

Caution: Changing string to string array type is not necessary for all fields. To support Endpoint, UEBA, and RSA Live content, specific string array (multi-value) and string (single-value) meta keys are required. See Update Your ESA Rules for the Required Multi-Value and Single-Value Meta Keys.

Determine if a Meta Key is a String Array Type on ESA

  1. Go to netwitness_configureicon_24x21.png (Configure) > ESA Rules and click the Settings tab.
    netwitness_121_metakeyreferences_1122_672x375.png
  2. In the Meta Key References, for each meta key that is a string array type, locate the meta key in the Name field and then check the value.
    • If it shows string[], it is configured as a string array type on ESA. This is fine.
    • If it shows string without the brackets, it is configured as a string type and you need to fix it on ESA. Go to Add the String Array Type Meta Key to ESA.

Caution: Changing string to string array type is not necessary for all fields. To support Endpoint, UEBA, and RSA Live content, specific string array (multi-value) and string (single-value) meta keys are required. See Update Your ESA Rules for the Required Multi-Value and Single-Value Meta Keys.

Add the String Array Type Meta Key to ESA

Caution: Any changes that you make to the multi-valued parameter may cause an error when you deploy your existing rules. If you add a meta key to the multi-valued parameter field that you use in other ESA rules, ensure that those rules are using the string array syntax.

  1. In the Explore view node list for an ESA Correlation service, select correlation > stream.
  2. Add string array meta keys to the multi-valued list to allow them to be used as an array in ESA rules.
    netwitness_121_correxplmultired_1122_768x435.png
  3. Verify the configuration on ESA. Go to Verify that the String Array Type Meta Key is Configured Correctly on ESA.

    4. Note: The multi-valued parameter is equivalent to the arrayFieldNames parameter in the Event Stream Analysis service in NetWitness version 11.2 and earlier.

Verify that the String Array Type Meta Key is Configured Correctly on ESA

  1. Go back to netwitness_configureicon_24x21.png (Configure) > ESA Rules and click the Settings tab.
  2. In the Meta Key References, click the Meta Re-Sync (Refresh) icon (netwitness_ic-refresh.png).
  3. Verify that the meta keys with a string array type show a value of string[].

Required String Array Meta Keys on the ESA Correlation Service

To use the latest Endpoint, UEBA, and Live content rules, the following default multi-valued meta keys are required on the ESA Correlation service in NetWitness Platform version 11.3 and later:

action , alert , alert.id , alias.host , alias.ip , alias.ipv6 , analysis.file , analysis.service , analysis.session , boc , browserprint , cert.thumbprint , checksum , checksum.all , checksum.dst , checksum.src , client.all , content , context , context.all , context.dst , context.src , dir.path , dir.path.dst , dir.path.src , directory , directory.all , directory.dst , directory.src , email , email.dst , email.src , eoc , feed.category , feed.desc , feed.name , file.cat , file.cat.dst , file.cat.src , filename.dst , filename.src , filter , function , host.all , host.dst , host.orig , host.src , host.state , inv.category , inv.context , ioc , ip.orig , ipv6.orig , netname , OS , param , param.dst , param.src , registry.key , registry.value , risk , risk.info , risk.suspicious , risk.warning , threat.category , threat.desc , threat.source , user.agent , username

The following default single-valued meta keys are also required on the ESA Correlation service in NetWitness Platform 11.3.0.2 and later:

accesses , context.target , file.attributes , logon.type.desc , packets

Note: Check the default-multi-valued and default-single-valued parameters on your ESA Correlation service for the latest required fields. For more information, see Update Your ESA Rules for the Required Multi-Value and Single-Value Meta Keys.

Remove Sensitive Meta Keys Globally from All Alerts for Data Privacy

11.4 Remove Sensitive Meta Keys from Global Alerts for Data Privacy

Note: This procedure applies only to ESA Correlation Rules in NetWitness Platform 11.4 and later versions.

For data privacy reasons, it may be necessary to remove some sensitive meta keys from the alert output globally, regardless of the data source. In the ESA Correlation service, you can set the global-private-fields parameter to remove the meta keys from all alert output.

  1. Go to netwitness_adminicon_25x22.png (Admin) > Services, and in the Services view, select an ESA Correlation service and then select netwitness_ic-actns.png > View > Explore.
  2. In the Explore view node list for the ESA Correlation service, select correlation > data-privacy.
  3. In the global-private-fields parameter, add the sensitive meta keys that you want removed from all alerts.
    netwitness_121_esacorrglobalprivatefields_1122_768x360.png
    The changes are effective immediately.

For more information, see "How ESA Handles Sensitive Data" in the Alerting with ESA Correlation Rules Configuration Guide. For more information on the strategy and benefits of obfuscating data, see the Data Privacy Management Guide.