Manage a Stream

You can manage a stream using the following procedures:

  • Edit a Stream
  • Reload the Stream
  • Specify meta filters for a Stream
  • Define multi-valued metas

Edit a Stream

You can edit a stream to perform the following:

  • Add data sources to the stream.
  • Delete existing data sources from the stream.

To edit a stream:

  1. Go to netwitness_adminicon_25x22.png (Admin) > Services.

  2. In the Services view, select the added Warehouse Connector service and select netwitness_actions_icon.png > View > Config.

    The Services Config view of Warehouse Connector is displayed.

    121_WC_services_config_view_1122.png

  3. On the Streams tab, click netwitness_edit_icon.png.

  4. In the Edit Stream dialog, you can perform the following:

    • On the Available Sources tab, you can select the available data sources to add to the stream and click Save.

      netwitness_wc_edit_stream.png

    • On the Current Sources tab, you can delete an existing data source from the stream. Select the data source and click netwitness_delete_icon.png.

      netwitness_wc_edit_stream2.png

Reload the Stream

When you reload the stream, the Warehouse Connector updates the schema file for the stream. You must reload the stream when you add a new custom meta to the Log Decoder or Decoder.

To reload the stream:

  1. Go to netwitness_adminicon_25x22.png (Admin) > Services.

  2. In the Services view, select the added Warehouse Connector service and select netwitness_actions_icon.png > View > Config.

    The Services Config view of Warehouse Connector is displayed.

  3. On the Streams tab, select the stream that you want to reload.
  4. Click Reload.

Specify Meta Filters for a Stream

You need to specify the filter for each stream in the export.session.meta.fields parameter in the Explore view of the Warehouse Connector.

The following table lists the values that you can provide as a filter:

Values Description
* All the collected metas are written to SAW.
*, meta1, meta2 All the metas except the defined metas are written to SAW.
For example,
Filter:*,ip.src
All the metas except ip.src is written to SAW.
meta1, meta2, meta3 Only the defined metas are written to SAW.

Note: By default, the following metas are written to Warehouse even if you specify them in the filter:
- ng_source
​- unique_id
- time

To specify meta filters for a Stream:

  1. Go to netwitness_adminicon_25x22.png (Admin) > Services.

  2. In the Services view, select a Warehouse Connector services and select netwitness_actions_icon.png > View > Config.

    The Explore view of the Warehouse Connector service is displayed.

    121_WC_explore_view_1122.png

  3. In the options panel, select warehouseconnector > streams > <stream_name> > loader > config.​

  4. In the export.session.meta.fields parameter, enter the filter.

    121_WC_specify_meta_filter_1122.png

  5. Restart the stream.

Define Multi-valued Metas

You can also define an existing meta or a custom meta to be treated as multi-valued meta.

To define multi-valued metas:

Caution: Defining an existing meta to be treated as multi-valued may change the data type of the meta and cause the associated reports to fail.

  1. Create a new file with the filename multivalue-users.xml in the /etc/netwitness/ng directory.

  2. Add the following entries:

    <?xml version="1.0" encoding="utf-8"?>

    <NetWitness>
    <MultiValueMetas>
    <Meta>NEWMETANAME</Meta>
    </MultiValueMetas>
    </NetWitness>

    Where NEWMETANAME​ is the existing meta or a custom meta to be treated as multi-valued meta.

    Caution: Make sure that you do not add metas that are by default treated as non multi-value.

  3. Restart the stream.

View the current schema

You can view the current schema that is used by warehouse connector for writing in AVRO files.

To view the current schema:

  1. Go to netwitness_adminicon_25x22.png (Admin) > Services.

  2. In the Services view, select a Warehouse Connector services and select netwitness_actions_icon.png > View > Explore.

    The Explore view of the Warehouse Connector service is displayed.

  3. In the options panel, select warehouseconnector > streams > <stream_name> > config.
  4. Set the value for refresh.schemanode.enabled​ parameter to true. By default, this value is set to false.
    121_View_Schema_1122.png

  5. Reload the stream. For more information see, Reload the Stream.

  6. Restart the Warehouse Connector service.
  7. Go to netwitness_adminicon_25x22.png (Admin) > Services.

  8. In the Services view, select a Warehouse Connector services and select netwitness_actions_icon.png > View > Explore.
    The Explore view of the Warehouse Connector service is displayed.

  9. In the options panel, select warehouseconnector > streams > <stream_name> > schema, to view the current schema.

Note: You must to reload the stream every time, before you want to view the current schema or you can add a scheduler to reload the stream automatically at regular intervals as mentioned in step 10.

  1. (Optional) To reload the stream automatically at regular intervals, follow the below steps.
    1. Go to netwitness_adminicon_25x22.png (Admin) > Services.
    2. In the Services view, select a Warehouse Connector services and select netwitness_actions_icon.png > View > Explore.
      The Explore view of the Warehouse Connector service is displayed.
    3. In options panel, select warehouseconnector > sys > config > scheduler. Right click and select properties.
    4. In the property drop down select addInter and in the Parameters text box, add "hours=24 pathname=/warehouseconnector/streams/<stream name> msg=reload".
      121_rescheduler_1122.png

    Note: Standard time format used is hours. You can use seconds or minutes format in lieu of hours.

    1. Then click Send.