Configure Logstash Output Plugin

You must configure the Logstash Output plugin to send the input events to a data warehouse destination. To configure the Output plugin, in the third section of the Logstash configuration file (netwitness-<decoder-ip>-input.conf), add the Output plugin parameter settings. You can use the existing Logstash standard output plugins for adding the parameter settings to the configuration file. For more information on existing Logstash standard output plugins, see Output Plugin Documentation.

The configuration of the plugin must consist of the plugin name followed by a block of parameter settings for that plugin. The following is an example of Logstash Output plugin configuration parameters.

output
{
kafka {
codec => json
topic_id => "logstash"
}
}

Performance tuning for Kafka and Kafka Output Plugin

Following are the output plugin parameters for Kafka and Kafka Broker.

Kafka Output Plugin:
Following parameters require change in value for each parameters to add additional throughput in the deployment.
batch_size, buffer_memory, compression_type, send_buffer_bytes
For more information, see Kafka output plugin documentation.

Kafka Broker:
Following parameters require change in value for each parameters to add additional throughput in the deployment.
num.network.threads, num.io.threads, socket.receive.buffer.bytes, socket.request.max.bytes, message.max.bytes, replica.fetch.max.bytes
For more information, see Kafka Broker Performance documentation.