JSON Mappings

JSON mappings is applicable from version NetWitness Platform 11.5 and Later.

View JSON Mappings

  1. In the NetWitness UI, go to netwitness_configureicon_24x21.png (Configure) > Log Parser Rules.

  2. From the Log Parsers pane, select a parser, then click JSON Mappings.

    The JSON Mappings and Mapping Details are shown for the parser you selected.

    122_JSONMapping_1222.png

The Mapping Details pane displays the following information.

Field Details
display name

This name corresponds to the name displayed in the JSON Mappings panel.
path

The path to where the values for this portion of the log are stored.

description

Optionally, you can enter a text description for this mapping.
meta

Select a meta key to which this value from the log is mapped. Select a value from the drop-down menu.

Optional if you choose a Value Format.
value format

Choose a value format parser onto which to pass this JSON value. You can either select an existing meta or Custom Regex Type. If you select custom regex type, you must define the regex and capture to fine parse the value in the meta.

Optional if you choose a Meta.

custom regex type

Select Custom Regex Type from the Value Format drop-down, which allows you to add new custom regex type.
regex pattern Specify a regex to identify different pieces of data contained within a JSON node value.
first capture

Select a meta key that should be captured first based on the value defined in the Regex pattern.

add a capture

New capture field is added. By default, it is loaded with meta keys in the drop-down. You can add maximum of 20 captures and this option will be disabled once it reaches maximum.

Note: You need to select a meta or enter a Value Format, but you do not need to fill in values for both settings.

Add a JSON Mapping

After you add a parser, as described in Add a Log Parser, you can then add JSON mappings.

  1. Follow the procedure to add a parser.

  2. Select the JSON Mappings entry for the newly-added parser.

    The following screen shows an example where an Accurev is added:

    122_AccPar_1222.png

  3. Click Add New to begin adding a mapping.
  4. Enter values for display name, path, meta or value format (or both), and (optionally) a description.
  5. Click Save Parser to save your new mapping.

Auto Discover JSON Mappings

Beginning with NetWitness version 11.5.1, you can automatically create the mappings without the need to manually enter the name and path of the mapping.

As an example, we are using the following JSON log:

{"terminal":"WIN-OT2OAJHG9NN","@timestamp":"2020-05-21T05:45:31.787Z","host_name":"WIN-OT2OAJHG9NN","global_userid":null,"dbusername":"C##TET_USER","object_schema":null,"os_process":"7992:5208","audit_option":null,"role":null,"unified_audit_policies":"ORA_LOGON_FAILURES","action_name":"LOGON","entry_id":1,"audit_type":"Standard","authentication_type":"(TYPE=(DATABASE));(CLIENT ADDRESS=((PROTOCOL=beq)(HOST=10.31.204.34)));","dbproxy_username":null,"external_userid":null,"@version":"1","new_schema":null,"new_name":null,"statement_id":1,"proxy_sessionid":0,"os_username":"WIN-OT2OAJHG9NN\\Administrator","system_privilege":null,"sql_binds":null,"timestamp":"2020-05-21 10:22:12","client_program_name":"sqlplus.exe","sessionid":4125005309,"userhost":"WORKGROUP\\WIN-OT2OAJHG9NN","rman_device_type":null,"object_name":null,"event_timestamp_utc":"2020-05-20T23:22:12.452Z","system_privilege_used":null,"return_code":1017,"version":"19.0.0.0.0","instance_name":"orcl","sql_text":null,"target_user":null,"fga_policy_name":null,"rman_object_type":null,"dbid":1566661212,"rman_operation":null}

To auto-discover JSON mappings:

  1. Select the JSON Mappings entry for the appropriate parser.

  2. Paste JSON formatted log message in to the Sample JSON message text box, and click Render JSON.

    The screen looks like this:
    122_RenJSON_1222.png

    Rendering JSON in Editing Mode allows you to view and edit (if needed) the logs in a pretty format. Additionally, if your text is not valid JSON, the text field is displayed in red.

  3. Click Mapping Mode, to view the JSON in a collapsable tree format which also highlights the mapping.

Note: In Mapping mode, you cannot edit the sample Log.

  1. Click Auto-Discover Mappings to discover the JSON nodes and create mappings.

    The Meta Mappings panel is populated as shown here:
    122_AutoDis_1222.png

  2. After you auto-discover, note that all the mappings are invalid (preceded by this icon: netwitness_ic-invalidmapping.png). You cannot save your changes until all the mappings are valid (mapping is preceded by this icon: netwitness_unsaved.png) or removed.

    Note:
    The meta value is highlighted in blue if it matches the selected mapping including the regex (if used).
    The meta value is highlighted in green if it matches any other mappings that are not selected.

  3. To make a mapping valid, you need to select a Meta Key or Value Format for all the mappings that you want to parse and save.

Note: In the Value Format you can either select an existing meta or Custom Regex Type.

  1. If you want to fine parse the value in the meta in the Value Format drop-down, select Custom Regex Type and enter the regex in the Regex Pattern field. For example,\s*(\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b):?(\d*).

Note: The custom regex will be added to the database, when you Save Parser.

  1. For a value in the Regex, you can select a meta key and store the capture as below:
    • First capture: The first portion of the string, up to the period character is stored to the meta key. For example, ip.src
    • To add more capture, click Add Capture and select the meta key.
      For more information on Regex, see Regex Values.

  2. If there are mappings that you do not want to save, you can select the mapping and click Delete. Alternatively, after you complete all of the mappings that you want to keep, you can click Remove Unmapped to remove all mappings that you have not yet validated.

  3. After you have either completed or removed all of your mappings, click Save Parser to save your new mappings. Note that the icon preceding each mapping is removed.

Deploy JSON Parser

You need to deploy a JSON parser so that logs coming in to any decoder are parsed appropriately and meta is generated and stored correctly.

To deploy a parser, select it from the list and click Deploy. The parser, its dynamic rules, and its mappings are sent to all Log Decoders.

Note: A JSON parser must have at least one rule or mapping to enable deployment.