Configure Data Obfuscation

[#1]

Configure Data Obfuscation

This topic provides the procedures for configuring data obfuscation in NetWitness. In a single deployment, all Core service configurations for a data privacy solution must be the same; be sure to use the same hash and salt across all Decoders and Log Decoders.

Note: In order for data obfuscation to work, user accounts need to be configured as described in Configure User Accounts for Use in Data Privacy.

Configure the Decoder Hash Algorithm and Salt

Value hashing accomplished as part of the data privacy solution occurs at the time of meta key creation on the Decoder and Log Decoder. Both services have default settings for use with all meta keys whose values are transformed without a specified hash algorithm type or salt value. The initial NetWitness values for defaults are: hash algorithm (SHA-256) and salt (none).

Note: The SHA-1 algorithm and it is not available.

If you want to change the default settings, you can edit them in the Services Config view > Data Privacy tab or in the following nodes in the NetWitness Services Explorer view:

• /decoder/parsers/transforms/default.type

  The algorithm to use for any keys with a transform defined that does not specify type. The supported algorithms are: duplicate and sha-256.

• /decoder/parsers/transforms/default.salt

  The salt value prepended to any value that is hashed (sha-256). This value is optional, an empty salt is valid and produces an unsalted hash. The salt is not defined by default so that you can create a unique salt for your environment. In general, the longer and more complex the salt, the better the security. A salt of up to 60 characters can be used without any major impact. A salt of at least 16 characters is recommended.

To edit the default hash algorithm and salt

1. Go to (Admin) > Services.

2. In the Services view, select a Decoder or Log Decoder service, and select > View > Config > Data Privacy.

   

3. In the Configure Hash Algorithm and Salt section, select a Hash Algorithm to use for any meta keys with a defined transform that does not specify type: sha-256. (A second algorithm, duplicate, is available for administrators to use in validating that expected hashing behavior is occurring in the network.)
4. (Optional) In the Salt field, enter a salt value to be prepended to any value that is hashed. This value is optional, an empty salt is valid and produces an unsalted hash.​ The salt is not defined by default so that you can create a unique salt for your environment. In general, the longer and more complex the salt, the better the security. Best practices for security purposes dictate a salt value that is no less than 100 bits or 16 characters in length. If a unique salt is required for each individual meta key, that needs to be configured in the index file as shown in example 3 below.

5. Click Apply.

   The new settings become effective immediately.

Configure Language Keys

The NetWitness Core Language has several language key attributes to facilitate data privacy. You can edit these attributes in the custom index file for each Decoder or Log Decoder. The custom index file (for example, index-decoder-custom.xml) is editable in the Services Configuration view > Files tab. After making changes in the index file, like the ones shown in the examples below, a service restart in a specific sequence is required.

Based on the data privacy requirements for your site, configure individual meta keys to be protected using the following key attributes:

• protected

  This attribute specifies that NetWitness should consider the values as protected and tightly control any release of the value. When propagating the protected attribute, NetWitness ensures that any downstream trusted system treats the values accordingly. Add this attribute to all services that create the protected values (that is, Decoder or Log Decoder) and any services that will provide trusted access (software development kit (SDK) query/values, aggregation) outside of Core services. The exception to this rule is that a Broker with no index file specified does not need to have the attribute added.

• token

  This attribute specifies that values for this key are stand-ins for another value and may not be visually interesting. The token attribute is informational, primarily for UI elements to display the value in a more useful or visually pleasing format.

• transform

  This child element of key indicates that any values for a given meta key should be transformed and the resulting value persisted in another meta key. The transform element is only required on Decoders and Log Decoders and is informational if specified on any other Core services. The transform element has the following attributes and children:

Name	Type	Description	Optional or Required
destination	attribute	Specifies the key name where the transformed value will be persisted.	required
type	attribute	The transform algorithm to apply. If not specified, the value of /decoder/parsers/transforms/default.type is used.	optional
param	child-element	A name/value pair, where each param element has a required attribute name and the child text is the value. The only supported param is used to specify a meta key-specific salt value. If not specified, the following value is used: /decoder/parsers/transforms/default.salt	optional

Example 1

On a Decoder or Log Decoder, mark username as protected and hash all values into username.hash with the default algorithm and salt:

<key name="username" description="Username" format="Text" protected="true">
<transform destination="username.hash"/></key>

Example 2

On a Concentrator, mark username as protected and username.hash as token:

<?xml version="1.0" encoding="utf-8"?> <language level="IndexNone" defaultAction="Auto">
<key description="Username" format="Text" level="IndexValues" name="username" protected="true"/>
<key description="Username Hash" format="Binary" level="IndexValues" name="username.hash"
token="true"/> </language>

Example 3

On a Decoder or Log Decoder, mark username as protected and hash all values into username.bin with the specified algorithm and salt:

<key name="username" description="Username" format="Text" protected="true">
<transform destination="username.bin" type="sha-256">
param name="salt">0000</param> </transform></key>

Configure Metadata and Content Visibility Per User Role on Core Services

On individual Broker, Concentrator, Decoder, Log Decoder, and Archiver services being viewed in the Services Security view (Admin) > Services > Security), administrators can configure the visibility of metadata and content based on the user group or role assigned to a user. This is called the SDK meta roles capability, and it is enabled by default. Administrators who want to configure metadata and content visibility per user must not disable the sdk.content permission (in the Roles tab). If the sdk.content permission has been disabled in the Roles tab, packets and raw logs are not visible to system.roles node. The system.roles node handles the filtering using the method configured in the Settings tab.

With sdk.content capability enabled, the next step is to select the method of filtering metadata and content in the Settings tab. Selecting a blacklist or whitelist option makes additional permissions for specific meta keys available in the Roles tab. The result is that administrators can choose a user role, such as analyst, in the Roles tab and select specific meta keys (and content) to be blacklisted or whitelisted for that user group. The permissions apply to any user in the user group.

For the Events view to display events correctly, the sdk.meta.medium meta key must be enabled to display values in the Type column (one of the default columns in the Events list). In addition, any meta keys used in a Query Prefix under User Settings in the Users tab or a PreQuery configured for Investigate must be enabled (Admin) > Security > Users or (Admin) > Security > Roles). If the meta key is not enabled for the user group, the column will not be displayed in the Events list.

Note: In the Version 11.4 and later Events panel, events are listed even when the user does not have permission to view the event reconstruction. In addition, the download options may still be available when a user does not have permission to view the event reconstruction. In these cases the user is allowed to download the events, but the output is an empty file because restricted permissions are enforced during the download.

The following table lists the options for filtering in the Settings tab and the numeric values used to disable (0) and the types of filtering (1 through 6). There is no need to know the numeric value unless configuring metadata and content visibility manually in the system.roles node.

system.roles Node Value	Settings Tab Option	Event Metadata	Original Event
0	No Filtering. System roles that define permissions on a per meta key basis are disabled.	Visible	Visible
1	Whitelist meta and content. By default no meta keys and no packets are visible. Selecting individual SDK meta roles per user group allows users to see metadata and packets for that SDK meta role.	Not Visible-Select to Show	Not Visible-Select to Show
2	Whitelist only meta. By default packets are shown, but no metadata is visible. Selecting individual SDK meta roles per user group allows users to see metadata for that SDK meta role.	Not Visible-Select to Show	Visible
3	Whitelist only content. By default metadata is visible, but no packets are visible. Selecting individual SDK meta roles per user group allows users to see packets for that SDK meta role.	Visible	Not Visible-Select to Show
4	Blacklist meta and content. By default all metadata and all packets are visible. Selecting individual SDK meta roles per user group prevents users from seeing metadata and packets for that SDK meta role.	Visible-Select to Hide	Visible-Select to Hide
5	Blacklist only meta. By default all metadata and all packets are visible. Selecting individual SDK meta roles per user group prevents users from seeing metadata for that SDK meta role.	Visible-Select to Hide	Visible
6	Blacklist only content. By default all metadata and all packets are visible. Selecting individual SDK meta roles per user group prevents users from seeing packets for that SDK meta role.	Visible	Visible-Select to Hide

Three factors determine what a user sees:

• The SDK meta role setting (blacklist or whitelist).
• The restricted meta keys configured for the group to which the user belongs.
• The meta keys in the session being analyzed. These include meta keys specified in a query prefix for a user at the service level, and for roles and users at the system level.

Caution: Be aware that with blacklisting, implicit trust is granted for all except the configured metadata. For a Decoder to have RBAC enabled and use implicit trust, it must only use a blacklist system setting; a whitelist setting will result in some issues with meta keys that are not explicitly enabled and therefore not visible. It is impossible to grant implicit trust under whitelist rules because the universe of meta keys cannot be known. If you want to use whitelisting, a workaround is to turn RBAC off for the Decoder and disable any user accounts from connecting directly to the Decoder if they should use RBAC.

Here is an example of how the SDK meta role configuration meshes with a Group that has restricted meta keys.

Configuration:

• The SDK meta role setting is Blacklist meta and content. With this option implemented, all metadata and all content (packets and logs) are visible by default.
• The administrator has restricted meta keys configured for the Analysts group to prevent viewing of sensitive data (for example, username).
• The packets and logs for any session that includes the username meta key are not visible to an analyst.

Result: Now a user who is a member of the Analyst Group investigates a session. Depending on the content of the session, the results are different:

• Session 1 includes the following meta keys: ip, eth, host, and file The session does not include username so all packets and logs in the session are displayed.
• Session 2 includes the following meta keys, ip, time, size, file, and username, Because the session includes username , no packets or logs from the session are displayed for the analyst.

To configure metadata and content restrictions for a Decoder or Log Decoder:

1. Go to (Admin) > Services.

2. In the Services view, select a Broker, Concentrator, Decoder, Log Decoder, or Archiver service and select ​ > View > Security. Click the Roles tab, select a role, and verify that the sdk.content permission is enabled.

   

3. Click the Settings tab.

   

4. Select one of the filtering methods (blacklist or whitelist) and content types (meta and content, meta only, or content only), and click Apply.

5. Click the Roles tab and a role for which you want to allow content (whitelist) or block content (blacklist) as specified in the SDK Meta Role Permissions setting.

   The Role Permissions for the selected role are displayed, and the SDK Meta Role Permissions are available for selection, for example, sdk.meta.medium. If you selected one of the whitelist options in the SDK Role Permissions setting, you must assign each SDK meta role to make the selected content visible to users assigned that SDK meta role. The sdk.meta.medium permission must be enabled in order for the Type column to be displayed in the Investigate > Events view. If you selected one of the blacklist options in the SDK Role Permissions setting, selected content will be hidden from users assigned that SDK meta role.
   

6. Select the SDK meta role permissions for users assigned this role. Click Apply.

The settings become effective immediately and apply to new packets and logs processed by the Decoder or Log Decoder.

Configure Meta Keys Not Written to Disk Per Parser on a Decoder

On a Decoder and Log Decoder, a Data Privacy Officer can configure individual meta keys that are not written to disk. To do so, the DPO specifies the meta keys as transient in the index and the parser configuration.

Note: The same capability was previously available on Log Decoders, and was configured when setting up parsers by modifying the table-map.xml file. Now it is integrated in the Services Config view.

To configure selected meta keys on individual parsers that will not written to disk:

1. Go to (Admin) > Services.
2. In the Services view, select a Decoder or Log Decoder service and select ​ > View > Config.

3. In the Parsers Configuration section of the General tab, select a parser and then select Transient in the Config Value drop-down list. Access the list by clicking on the configuration value (Enabled, Disabled, or Transient).

   The configuration change is marked by a red triangle.

   

4. Click Apply.

   The change is effective immediately. The parser configured as Transient will no longer store meta keys to disk.

Previous Page	Next Page