Log Parser Customization

Note: The JSON Mapping information in this guide applies to NetWitness Version 11.5 and later.

You use the Log Parser Rules view (available from the netwitness_configureicon_24x21.png (Configure) view) to customize rules and to map meta for your log parsers.

The default log parser parses logs that do not match any installed log parsers. The information contained in such a log is processed against the default log parser's rules, and metadata is then extracted by those rules and is available for Enrichment, Investigation, Reporting, and Alerting. This provides immediate visibility into logs from custom or unsupported sources.

You can also add or extend a log parser. For example, you may need to parse certain fields differently than in the manner provided by the log parser for a particular event source. You can add rules that change the way meta information is extracted from the logs for the event source. And you can then map the extracted information to NetWitness meta keys.

Finally, you can view and test sample log messages and rules for your log parsers, including the default log parser.

To access this tab, go to netwitness_configureicon_24x21.png (Configure) > Log Parser Rules. For more details, see Parser Rules Tab.

Dynamic Rules

The Dynamic Rules entry for a log parser displays the following information:

  • The default log parser that parses logs that are not associated with a particular log parser
  • Native XML-defined device parsers that have been extended with dynamic log parser rules, and
  • User-created custom device parsers used to parse unsupported custom event sources

The dynamic parse rules are used to parse arbitrary values triggered by a literal token on the left hand side of the value in unstructured logs. Currently, this is used to parse name-value pairs.

Following are some examples (the token is in bold red):

src: 1.2.3.4
src = 1.2.3.4
src=1.2.3.4
Source address is 1.2.3.4

JSON Mapping

This allows you to parse JSON nodes from Logstash as well as Log Collection Plugins by mapping JSON nodes to an appropriate meta on which the value should be saved.

The JSON Mappings for a log parser displays the following information:

  • Sample JSON Message - This allows you paste the log messages.

  • The list of JSON Mappings: these are the names that represent the meta information.
  • Details of each mapping: for each mapping, the display name, path, NetWitness meta key, and a text description.

The JSON mapping functionality is for strictly paring structured JSON logs, and mapping values from the log to meta or fine parsing. The parsing is not applied to arbitrary logs; only logs where we know the exact structure of the data.

For example:

{ “event”: { “source”: { “address”: ”1.2.3.4”, “port”:8080 } } }

NetWitness knows the structure of logs when it knows the event source type, or when you add specific JSON mappings.