Configure Storage Using the REST API

In NetWitness Platform 11.3 and later releases, you use the REST API for all storage configuration operations. For information about how to use the REST API, see the RESTful API User Guide. Go to the NetWitness All Versions Documents page and find NetWitness Platform guides to troubleshoot issues.

REST API Storage Configuration Commands

Each of the commands listed below has built-in help that describes their function and usage. If you are using the REST interface, select the command from the drop-down menu to see the help text. For examples of REST API storage configuration commands, see Appendix D. Sample Storage Configuration Scenarios for 15-Drive DACs.

Commands for Direct-Attached RAID Volumes

  • raidList : List the RAID controllers and direct-attach enclosures that are present on this host.
  • raidNew : Allocate direct-attached enclosures to block devices.

Commands for Allocating Block Devices as Storage

  • devlist : List available block devices on the host.
  • partNew : Allocate partitions on a block device and create volume groups.
  • vgs : Summarize how block devices are organized into volume groups.

Commands for Allocating Storage to Services

  • srvList :List services on the host and their allocated storage paths.
  • srvAlloc : Allocate a volume group to a service.
  • srvFree : Remove a volume group from a service.
  • multipath-II : To verify if SAN devices are attached.

Command to Reconfigure Services to Detect and Use All of the New Storage

  • reconfig - After configuring new storage, detect and use new storage on the associated service and database.

Storage Configuration Tasks

Task 1 - Attach storage to the host and access the REST API storage configuration commands.

Task 2 - (Conditional) Configure RAID if necessary.

Task 3 - Allocate block devices to partitions, volume groups, and logical volumes.

Task 4 - Allocate volume groups to NetWitness services.

Task 5 - Reconfigure services and databases to detect and appropriately use new storage.

Task 1 - Attach Storage to the Host and Access the REST API Storage Commands

IMPORTANT: Task 1 is not applicable for NetWitness version 11.5.0.0 and 11.5.0.1.

Complete the following steps to attach an external storage device to a host and access the storage configuration commands available through the REST API.

  1. Attach the storage and make it available to this host.
  2. There are two ways that you can access the REST API storage commands: from a Browser, or from the Services > Explore view from the User Interface.
  3. Note: Once you have accessed the REST API, the steps that you perform are the same, no matter which method you used to access it.

    • From a Browser.
      1. Open a Browser and specify the ip-address of the host with port 50106.
        The following example is the Decoder, but you need to use port 50106 for any host hardware for which you are configuring storage using the REST API.
        https://<decoder-ip-address>:50106
      1. Log in with the admin account credentials.
        The following REST API menu is displayed.
        netwitness_restglobal1_181x282.png
      2. Click on the (*) next to appliance to access the REST command set.
        The Properties for /appliance dialog is displayed under the initial REST menu. The Output (or command manual help) section describes the commands that the REST API can send to the device, their usage, and their parameters.
        netwitness_unityrest2.png
    • From the User Interface.
      1. In the NetWitness menu, go to netwitness_adminicon_25x22.png (Admin) > SERVICES.
      2. Select the service (for example, a Concentrator).
      3. Under netwitness_actions_button.png (actions), select View > Explore.
      4. Navigate to deviceappliance/appliance, right click, and click Properties.
        netwitness_explore1_300x100.png

      Note: If you are on NetWitness version 11.5.0.0 or 11.5.0.1, you must navigate to System > Host Tasks >Task.

      You can now access the storage commands from the Properties dialog.

  4. Proceed to:
    • Task 2 if you need to configure RAID for PowerVault or DACs.
    • Task 3 if you do not need to configure RAID and already have a block device available.

Task 2 - (Conditional) RAID Configuration for PowerVault and DACs

IMPORTANT: Task 2 is mandatory if you are on NetWitness version 11.5.0.0 or 11.5.0.1.

The NetWitness Platform hardware uses direct-attached SAS drives for storage. These drives are housed in a SAS enclosure. SAS enclosures are shelves of drives attached to the NetWitness node by a cable connected to the SAS host bus adapter.

SAS enclosures are also known as other names, such as "DAC" (Direct-Attached Capacity), or "JBOD" (Jumbo Box of Disks), or "RSA PowerVault".

The NetWitness Platform utilizes RSA PERC SAS host bus adapters. NetWitness Platform devices typically include two SAS host bus adapters. One is used for controller drives that are internal to the NetWitness Node, and another is used for controlling drives attached to the SAS enclosures. The internal controller and drives are configured when the node is built, but the external SAS enclosures are not. You execute the raidList and raidNew commands to identify and configure the external SAS enclosures.

These commands work with the following SAS enclosure types:

  • EMC ESAS 15-drive enclosures
  • EMC ESAS 60-drive enclosures
  • RSA PowerVault 12-drive enclosures
  • RSA PowerVault 8-drive enclosures

Note: EMC 60-drive enclosures are logically organized as four separate 15-drive sub-enclosures. They behave as if there are four 15-drive enclosures, each of which can be configured independently.

The raidList and raidNew commands operate on entire enclosures. Execute raidList to identify the enclosues. execute raidNew to configure an enclosure to perform one of the pre-determined roles within a NetWitness Platform node.

After you attach storage to the host and access the REST API storage commands, complete the following steps to create RAID if required.

  1. Execute the raidList command to identify the controllers and enclosures that are attached to the system.
    In the following example, Controller 1 does not display any block devices. This indicates the array is not configured.
    netwitness_final-raidlist-1.png
  1. Select a RAID layout scheme for the Enclosure.

The following table lists the PV to Supported Hosts Mapping.

Type SKU Specification Supported Hosts
High Performance NW-PV-C RSA Storage MD 1400 6 x 12TB NL-SAS SED, 2 x 3.8TB SSD SED Concentrator
High Performance NW-PV-D RSA Storage MD 1400 9 x 12TB NL-SAS SED, 3 x 3.8TB SSD SED Concentrator
High Density NW-PV-A RSA Storage MD1400 12 x 12 TB NL-SAS SED Decoder, LogDecoder, Archiver, hybrid (log & packet)
High Density NW-PV-B RSA Storage MD 1400 8 x 12TB NL-SAS SED Decoder, LogDecoder, Archiver, hybrid (log & packet)

The following tables show you the supported allocation schemes.

Note: For RAID configuration, when the Decoder is configured for 10G capture, use the decoder scheme for the first two enclosures and the archiver scheme for subsequent enclosures.
When you are not configuring for 10G capture, use the decoder scheme for the first enclosure and the archiver scheme for subsequent enclosures.
These configurations will maximize storage capacity and performance.

Scheme Drives Required Allocation
decoder or logdecoder 8 or 12 or 15 HDDs

3x drives in RAID 5 for decodersmall or logdecodersmall, all remaining drives in RAID 5

archiver 8 or 12 or 15 HDDs All drives in RAID 6 for archiver or decoder database volume
networkhybrid 8 or 12 or 15 HDDs 3x drives in RAID 5 for meta expansion, all remaining drives in RAID 5 for packet expansion
loghybrid 8 or 12 or 15 HDDs

Half of the drives in RAID 5 for meta expansion, half the drives in RAID 5 for packet expansion

concentrator 2 or more SSDs, 4 or more HDDs All SSDs in RAID 1 or RAID 5 for index, all HDDs in RAID 6 for meta
  1. After the controller, enclosure, and scheme are identified, execute the raidNew command to create RAID Volumes. For example:
    send /appliance raidNew controller=1 enclosure=82 scheme=decoder preferSecure=false
    Add the commit=1 parameter to actually execute this operation. Execute the raidList command to list the created block devices.
  2. (Optional) Configure SEDs (Self-Encrypting Drives). If the raidNew command detects self-encrypting drives and a security key has been set on the controller, the raidNew command will attempt to create a secure array. To set a security key on the controller, execute the raidKey command. For example:
    send /appliance raidKey controller=1 key=myPasssphrase keyId=1
    • To create a secured (that is, encrypted) array on physical devices attached to a controller with a security key set, specify preferSecure=true when using raidNew
    • To create an unsecured (that is, unencrypted) array on physical devices attached to a controller with a security key set, specify preferSecure=false when using raidNew.
  3. Go to Task 3 - Allocate Block Devices to Partitions, Volume Groups, and Logical Volumes, after you create RAID volumes.

Task 3 - Allocate Block Devices to Partitions, Volume Groups, and Logical Volumes

The partNew command prepares a storage device to use in NetWitness Platform. It performs the following tasks.

  • Creates the partition table on the block device.
  • Creates the Linux Volume Manager physical device partition.
  • Creates a volume group containing the physical device.
  • Creates logical volumes in the volume group.
  • Creates XFS filesystems on each logical volume.
  • Creates /etc/fstab entries for each logical volume.
  • Mounts each logical volume.

Complete the following steps to allocate block devices to partitions, volume groups, and logical volumes.

  1. Run the devlist command to locate unused block devices. The following example shows the devlist command output.
    netwitness_final-devlist-1.png

    Also,you must provide a name for the service that will be used with the storage, for example, decoder for the Network Decoder service, or concentrator for the Concentrator service. You have the option of providing the volume type. The default volume type has the same name as the service.

  2. Note: Run the devlist command to see if the multipath user-friendly names are listed correctly.

  3. Run the multipath_II command to make sure that SAN devices are attached. The following is an example when SAN devices are attached.netwitness_multipathcommands.png
  4. Note: Block devices should be configured with a user-friendly name such as mpatha, mpathb etc.

  5. Execute the partNew command to allocate block devices to partitions, volume groups, and logical volumes.

    By default, the partNew command does not make changes. It displays the actions that will be taken if you commit the command string. To actually make the changes to the system, add the commit=true parameter to the command.
    For example, to assign devices sdd and sde to Decoder:
    send /appliance partNew name=sdc service=decoder volume=decodersmall commit=true
    send /appliance partNew name=sdd service=decoder volume=decoder commit=true

    Caution: For the decoder and concentrator services, you must create storage volumes in a specific order.
    - The decoder has the decodersmall and decoder volumes. Create the decodersmall volume before the decoder volume because decodersmall contains the small filesystem mounted at /var/netwitness/decoder.
    - The concentrator has the concentrator and index volumes. Create the concentrator volume before index volume or it will fail and you receive the following message.
    Failed to process message partNew for /appliance com.rsa.netwitness.carlos.transport.TransportException: Volumes for index require mount point /var/netwitness/concentrator to be created and mounted first.

  6. Execute the vgs command to validate that the partNew command created the correct Logical Volumes.
    The output of this command:
    • Enumerates all the volume groups on this host.
    • Displays the physical volumes that the volume group consists of, and the logical volumes within the volume group.
  7. Go to Task 4 - Allocate Volume Groups to NetWitness Services- srvAlloc.

Task 4 - Allocate Volume Groups to NetWitness Services - srvAlloc

The srvAlloc command configures services on a host to use storage in a volume group. You must provide the name of the service to configure and the volume group to assign to the service (the service you provide must be installed on the host). For information about NetWitness Platform service volumes, see "NetWitness Platform Service Volume Reference" in Storage Requirements.

Allocate services in the following order:

  • For the Decoder, allocate decodersmall first then the decoder
  • For a Concentrator, allocate concentrator first then index.

Note: By default, the srvAlloc command does not make changes. You must append the commit=1 parameter to the command string to actually make the changes to the system and restart the specified service after making changes.

  1. Execute the srvList command to see a list of services installed on this host.
    The srvList command communicates with the service through the SSL port. You install a Category on a host. A Category can be a single service, or multiple related services, located on the same host.
  1. Execute the srvAlloc command to configure a service on a host to use storage in a volume group. For example:
    service=concentrator volume=concentrator commit=1
    service=concentrator volume=index commit=1
  2. Go to Task 5 - Reconfigure Services and Databases to Detect and Appropriately Use New Storage.

Task 5 - (Optional) Reconfigure Storage Configuration for 10G Capture

You need to reconfigure the Decoder service and databases for 10G capture. Complete the following steps so that the Network Decoder service and its database detect and use new free space.

  1. In the NetWitness menu, go to netwitness_adminicon_25x22.png (Admin) > SERVICES.
    The SERVICES view is displayed.
  2. Select the decoder.
  3. Under netwitness_actions_button.png (actions), select View > Explore.
    The Explore tree for the service is displayed.
  4. Reconfigure space on the decoder service.
    1. Navigate to the decoder, right click, and click Properties.
      netwitness_explore3-decoder_300x220.png
      The Properties dialog is displayed.
    1. Execute the reconfig command by selecting it from the drop-down list, specifiy update=1 op=10g in Parameters, and click Send.
      netwitness_explore4-decod.png
  5. Reconfigure space on the database.
    1. Navigate to database in the service Explore tree, right click, and click Properties.
      netwitness_explore3-decoder-database_300x220.png
      The Properties dialog is displayed.
    1. Execute the reconfig command by selecting it from the drop-down list, specifiy update=1 op=10g in Parameters, and click Send.
      netwitness_explore4-decodatabase.png