NetWitness Community

3 Likes

The Investigation feed generates metadata in order to assist analysts with threat hunting and content generation such as reports and alerts. This is useful for front line analysts, because it minimizes the time dedicated to mining logs or sessions in support of their findings. The content within the feed is a list of application rules or network Lua parser logic and their associated tags according to the Investigation Model.

Each piece of content has this feed listed as a dependency for deployment through Live and, therefore, no manual deployment is necessary. If you need to manually deploy, or are running a version of NetWitness Platform prior to 10.6.2, see the Deployment section within this document.

The feed is multi-callback, meaning it uses multiple keys to trigger the meta generation. Each application rule or piece of Lua parser logic will output to one of the six hunting keys, or for older content, the meta key called alert.id. On trigger of the content, the feed will fire based on a match to one of these keys and output the additional meta data according to the Investigation Model.

Note: Content that currently registers alert.id values will be updated to instead register directly to the appropriate hunting key instead.

An example output of matches to application rules and Lua parser logic is below. The content writes to one of the six Hunting Keys and the feed generates the ATT&CK Tactic and ATT&CK Technique meta.

Feed Structure

The Investigation Feed has the following structure:

Index or Trigger Meta Keys
- Indicators of Compromise
- Behaviors of Compromise
- Enablers of Compromise
- Session Analysis
- Service Analysis
- File Analysis
Registered Meta Keys:
- ATT&CK Tactic
- ATT&CK Technique
- risk
- Feed Name

Index/Trigger Meta Keys

These are the keys that trigger the feed callback. All application rules and Lua parser logic developed since the creation of the hunting keys writes to one or more of these six keys.

Hunting Compromise Keys

Compromise keys provide insight and narratives into the varied attributes of an attack. These can be atomic or computed indicators.

Indicators of Compromise (ioc) Datatypes used in Threat Indicator Portals, or known signature-type resources should be pushed here. Anything worthy of analysis which denotes high confidence.
Behaviors of Compromise (boc) Tactics or techniques employed by malware and/or adversaries. Sometimes this observed behavior could be an anomaly or just poorly written code, or simply administrator activity. Use this key when there is no datatype indicator present but signifies potential cause for concern if high value hosts or parties are involved.
Enablers of Compromise (eoc) Reserve usage of this key for activities or policies that may contribute to an incident such as servers running default credentials or shared access amongst administrators.

Hunting Analysis Keys

Analysis keys provide an operational workflow for investigative operations through natural language descriptors. They quickly dissect collections based on features observed in network and security information.

Session Analysis (analysis.session) Client-Server communication summations, deviations, conduct and session attributes.
Service Analysis (analysis.service) Core application protocols identification. An underlying powerhouse of service-based inspection.
File Analysis (analysis.file) A large inspection library that highlights file characteristics and anomalies.

Registered Meta Keys

These are the keys that are generated based on a callback to an indexed meta key.

The ATT&CK Tactic key (attack.tactic)
The ATT&CK Technique key (attack.technique)
The Investigation Category (inv.category) key
The Investigation Context (inv.context) key

The model on which the inv.category and inv.context keys are based and the possible values is described in the NetWitness Investigation Model.

The risk meta key.
The Feed Name key (feed.name) will always be populated with the value investigation.

Discontinued and Deprecated Feeds

The Hunting feed has been discontinued, and the Alert ID feeds are deprecated.

Hunting Feed Discontinued

The Hunting feed has been discontinued with the release of the Investigation feed. The Hunting feed generated the same text-based name of the content logic into one of the six Hunting meta keys. Since content now directly writes to one of the six Hunting meta keys, the Hunting feed is no longer necessary.

Note: RSA recommends removal of the Hunting feed from the Log Decoders and Decoders.

To remove the legacy feeds from your Decoders:

Depending on your version:
- For RSA Security Analytics 10.x: In the Security Analytics menu, select Administration > Services and select a Decoder service.
- For RSA NetWitness 11.x: From the NetWitness UI, select ADMIN > Services, then select a Decoder service.
In the Actions column, select View > Config.
Select the Feeds tab.
Search for and select the Hunting feed.
Click – to remove the selected feed.

Alert ID Feeds Deprecated

Going forward, Content will only write to one of the six Hunting meta keys. Previously, Content wrote to the Alert ID meta key. The value of that meta key was a unique identifier, for example nw132520. The three Alert IDs feeds use the Alert ID meta key to generate a text-based name of the content to one of the meta keys of Risk: Informational, Risk: Suspicious, or Risk: Warning. These keys are deprecated in favor of the Hunting keys, making the Alert ID feeds legacy content.

Note: If you actively use the Risk: Informational, Risk: Suspicious, and Risk: Warning meta keys within your system, NetWitness recommends you leave the Alert ID feeds in place. Once your content and processes have been updated to use the Hunting and Investigation keys, the feeds should be removed from the Log Decoders and Decoders. Until the Alert IDs feeds are removed, you will get some duplication of meta values across the Risk and Hunting keys (as described in more detail in the following section).

Effects of Continuing Use of Legacy Content

The Alert ID feeds are deprecated, but are still active in Live. They are still in Live for the following reasons:

NetWitness might make future updates to the Alert ID feeds, and
Some customers are still using the generated meta from these feeds (the Risk * and alert.id keys).

Note the following duplication of data issues:

You will see exact duplicate meta keys and values if you have the Hunting feed and Investigation feeds enabled.
You will see some duplication of meta values generated by both the Risk* and Hunting keys. Thus, NetWitness recommends that customers begin moving from the Risk* keys (Risk: Informational, Risk: Suspicious, Risk: Warning) and Alert ID* feeds (Alert IDs Info, Alert IDs Suspicious, and Alert IDs Warning) in your content to the Hunting keys.

Example of duplicated data:

Generated from the Alert ID Info feed

risk.info = "http post missing content-type"
Generated from the Investigation feed:

analysis.service = "http post missing content-type"
Generated from the Hunting feed:

analysis.service = "http post missing content-type"

Once you have updated your content so that it is not dependent on the Risk keys and Alert ID feeds, you can remove the Alert ID feeds from your system, and thus you will stop receiving duplicate meta values.

Deployment

This section discusses:

How to add the meta keys used by the feed to the Index file, and
How to deploy the feed.

Note: Manual deployment of the feed should not be required, since it will be downloaded and deployed as a dependency with the content through Live.

About the Meta Keys

To get value from the Investigation feed, the following meta keys are required. The below keys should be added to your index-concentrator-custom.xml.

Note: These keys are now being delivered out of the box in index-concentrator.xml with NetWitness version 10.6.2 and newer. If your installed version is prior to 10.6.2, you must add the keys to index-concentrator-custom.xml.

To add keys to index-concentrator-custom.xml:

If your installed version of NetWitness is 10.6.2 or newer, you can skip this procedure.

In the Security Analytics menu, select Administration > Services, and select a Concentrator.
Select View > Config from the Actions menu.
Select the Files tab, then select the index-concentrator-custom.xml file.
Add the following lines:

<key description="Investigation Category" level="IndexValues" name="inv.category" format="Text" valueMax="10000"/>
<key description="Investigation Context" level="IndexValues" name="inv.context" format="Text" valueMax="10000"/>
<key description="Session Analysis" level="IndexValues" name="analysis.session" format="Text" valueMax="10000"/>
<key description="Service Analysis" level="IndexValues" name="analysis.service" format="Text" valueMax="10000"/>
<key description="File Analysis" level="IndexValues" name="analysis.file" format="Text" valueMax="10000"/>
<key description="Indicators of Compromise" level="IndexValues" name="ioc" format="Text" valueMax="10000"/>
<key description="Behaviors of Compromise" level="IndexValues" name="boc" format="Text" valueMax="10000"/>
<key description="Enablers of Compromise" level="IndexValues" name="eoc" format="Text" valueMax="10000"/>
Click Apply.

This screen shows the lines being added to the file in NetWitness:

Deploy the Feed

Depending on your version, see: