Configure NetWitness Export Connector

Note: Make sure you open the firewall of the Decoder or Log Decoder to establish connection with the Logstash. For more information, see "Network Architecture and Ports" in Deployment Guide for RSA NetWitness.

You must configure the Logstash configuration file to process the NetWitness events. Create a Logstash configuration file and add the NetWitness Export Connector plugin parameter settings for event processing. Save the file as netwitness-<decoder-ip>-input.conf. After adding the NetWitness Export Connector plugin parameter settings, place the configuration file in /etc/logstash/conf.d/ location.

A Logstash configuration file can have three separate sections for each type of plugin that you want to add to the event processing pipeline. The first section is for Input plugin (NetWitness Export Connector), the second section is for Filter plugin (optional) and the third section is for Output plugin.

To configure the NetWitness Export Connector plugin, add the parameter settings in the first section the Logstash configuration file.

For multiple pipelines configuration, see Multiple Pipelines Configuration documentation.

The configuration of each NetWitness Export Connector plugin must consist of the plugin name followed by a block of parameter settings for that plugin. If the NetWitness Export Connector has multiple plugins with block of parameters, they are applied in the order of their appearance.

The following is an example of NetWitness Export Connector with one plugin instance with block of parameter settings which fetches data from a single decoder .

input {
netwitness {
host => "<host>" # Mandatory
username => “<username>" # Mandatory
password => "<password>" # Mandatory
decoder_type => "logdecoder" # Mandatory
}
}

The following is an example of NetWitness Export Connector with two plugin instances with block of parameter settings which fetches data from two different decoders. Each plugin in the configuration is applied in the order as shown.

input {
netwitness {
host => "<host>" # Mandatory
username => “<username>" # Mandatory
password => "<password>" # Mandatory
decoder_type => "logdecoder" # Mandatory
}
netwitness {
host => "<host>" # Mandatory
username => “<username>" # Mandatory
password => "<password>" # Mandatory
decoder_type => "logdecoder" # Mandatory
}
}

Note: When configuring the Logstash, you may need to specify sensitive settings such as passwords. You can use the Logstash keystore to securely store secret values instead of file system permissions for using it in configuration settings. For more information, see Logstash keystore Documentation.

Following are the parameters accepted by NetWitness Export Connector.

Parameter Description Parameter Type Default Value
host IP address or hostname of the Decoder or Log Decoder (mandatory)

String

 N/A
username Username used to access the Decoder or Log Decoder (mandatory) String N/A
password Password of the user (mandatory)

String

 N/A
decoder_type Accepts only 'decoder' or 'logdecoder' (mandatory) String N/A
ssl_enable Enable SSL connection between the Decoders and the NetWitness Export Connector. For more information, see Configure SSL

Boolean

 false
ssl_certificate_path Path of the CA certificate that is used for SSL and Trusted Connections /etc/pki/nw/trust/truststore.pem
ssl_certificate_password Password of the certificate in use. Mandatory if SSL is enabled

String

 N/A
ssl_client_certificate_path Client's SSL certificate /etc/pki/nw/node/node-cert.pem
ssl_version Version of the SSL connection String TLSv1.2
plugin_metrics_enable Enables metrics reporting to Elastic (New Health and Wellness), for more information see Health and Wellness

Boolean

 false
elastic_host IP address or hostname of the Elastic host. Mandatory if plugin metrics is enabled String N/A
elastic_port Port number of the Elastic host

String

 9200
elastic_username The username that is used to access the Elastic host. Mandatory if plugin metrics is enabled String N/A
elastic_password The password that is used to access the Elastic host. Mandatory if plugin metrics is enabled

String

 N/A
meta_include Aggregates only the meta keys that are added in this parameter setting. Accepts comma separated values (csv) format String nil
meta_exclude Excludes the meta keys that are added in this parameter setting from aggregation. Accepts comma separated values (csv) format

String

 nil
start_session Session from which the aggregation starts. Setting the value to 0 starts the aggregation from last.session.id in the Decoder Number 0
export_log Includes the raw log with the meta in the session (applicable only for Log Decoder aggregation)

Boolean

 false
aggregate_sessions Number of sessions ingested in a batch from the Decoders Number 1000
aggregation_interval Time interval (in milliseconds) between two event cycles

Number

 1000

prefetch_count

Controls the number of batches to be pulled into the buffer that is available for the plugin to collect

 Number

2
compression The number of bytes in each message before message is compressed. Setting the value to '0' does not allow compression. Ranges between 0 to 131071

Number

 0
compression_level The level of compression. Ranges between 0 to 9 where 1 is fastest and 9 has the better compression. A value of 0 selects the best balance between speed and compression Number 6
buffer_size Controls the number of records that the stream in the buffer before it is ingested

Number

 40000

position_tracking_path

Path where the last consumed session id is stored. Default storage location is /var/lib/logstash. For more information, see Position tracking and start session

 String

/var/lib/logstash

custom_meta_config_path Path to the custom multivalued meta configuration file. For more information, see Configuring Custom Multi-valued Meta

String

 nil
Query Takes any NetWitness Platform query as Input

Note: Only Indexed meta key must be the part of the query. For example, select * where device.type = 'rhlinux'

 String Select *

Position tracking and start session

Position tracking or bookmarking is used to track the sessions that are aggregated by logstash and sent to the consumer. Position tracking initiates automatically and updates the tracking file every 60 seconds in the path mentioned in position_tracking_path parameter in the configuration file. The file consists of two parts [sessionid,timestamp]. Default location is /var/lib/logstash, if it is not mentioned in the configuration file.

The start_session parameter accepts a number (long: primitive datatype) and indicates which is the first sessionid the plugin should request from its corresponding source. if the start_session parameter is not mentioned in the configuration file or if the value is mentioned as 0, the first session requested by the plugin will be the last.session.id + 1, last_session_id as in the decoder's REST API /database/stats/last.seesion.id.

if the position tracking file exists for a source and start_session is not configured or the start_session is set as value 0, the plugin with initiate aggregation from the sessionid indicated in the position tracking file.

if the position tracking file exists for a source and the start_session is set as non-zero value, the start_session value will take precedence over the position tracking file. The plugin will request from the session mentioned in the start_session parameter onwards.