Connecting to a Service

To connect and interact with a NetWitness Core service (Decoder, Concentrator, Broker, Archiver, and so on), you must first issue the login command. You must have an account on that service. You can type help login at any time for more information. Here is the syntax of the login command:

login <hostname>:<port>[:ssl] <username> [password]

For example: login 10.10.1.15:56005:ssl someuser

If you do not include the password, the NwConsole prompts you.

If you have set up proper trust between NwConsole and the endpoint, you can use the tlogin command and avoid having to enter a password. Setting up trust is beyond the scope of this documentation, but it involves adding NwConsole's SSL cert to the endpoint with the send /sys peerCert op=add --file-data=<pathname of cert> command. You must first use a normal login with the proper permissions before you can add a peer cert for subsequent trusted logins.

Once connected, you can interact with the endpoint service through a virtual file system. Instead of files, what you are looking at are the nodes of that service. Some nodes are folders and have child nodes, forming a hierarchical structure. Each node serves a purpose and all of them support a subset of commands like info and help. The help message returns information about the commands that each node supports. When you first log on, you are on the root node, which is the path /, just like a Linux or Mac system. To see a list of nodes under /, type the ls command.

All services have nodes like sys and logs. To interact with the /logs API, you can first send the help command to the /logs node. To do this, you must use the send message, which has this syntax:

Usage: send {node pathname} {message name} [name=value [name=value]]
            [--file-data=<pathname>] [--string-data=<text>] [--binary-data=<text>]
            [--file-format={binary,params,params-list,string,params-binary}]
            [--output-pathname=<pathname>] [--output-append-pathname=<pathname>]
            [--output-format={text,json,xml,html}]
Sends a command to a remote pathname.  For remote help, use "send <pathname>help" for details.
    pathname                 - The node pathname to retrieve information on
    message                  - The command (message) to send
    parameters               - Zero or more name=value parameters for the command
    --file-data              - Loads data from a file and send as either a BINARY
                               message or as a PARAMS_BINARY message if other
                               parameters exist. Providing the --file-format parameter will 
                               override automatic detection. 
   --file format             - Treat file input as the provided format
  --string-data             - Sends text as a STRING message type
   --binary-data             - Send text as either a BINARY message type or as a
                               PARAMS_BINARY message type if other parameters
                               exist
    --output-pathname        - Writes the response output to the given pathname,
                               overwriting any existing file
    --output-append-pathname - Writes the response output to the given pathname,
                               will append output to an existing file
    --output-format          - Writes the response in one of the given formats,
                               the default is text

Parameters are formatted in <name>=<value> pairs, with whitespace between each pair. If a <value> has whitespace, you should place each value in double quotes. If the value itself should have double quotes, you can escape them by preceding with a backslash. For example: send /sdk query query="select * where time=\"12-31-2019 14:30:00\"-u" size=100

You must also escape backslashes; a double backslash is treated as a single backslash. It is also possible to pass special characters by giving their hex value using \x##. For example, \x0a would pass a line feed in the parameter value.

To send a help message, you would send this:

send /logs help

And your response would look something like this:

  description: A container node for other node types
    security.roles: everyone,logs.manage
    message.list: The list of supported messages for this node
    ls:  [depth:<uint32>] [options:<string>] [exclude:<string>]
    mon:  [depth:<uint32>] [options:<uint32>]
    pull:  [id1:<uint64>] [id2:<uint64>] [count:<uint32>] [timeFormat:<string>]
    info:
    help:  [msg:<string>] [op:<string>] [format:<string>]
    count:
    stopMon:
    download:  [id1:<uint64>] [id2:<uint64>] [time1:<date-time>] [time2:<date-time>] 
      op:<string> [logTypes:<string>] [match:<string>] [regex:<string>] 
      [timeFormat:<string>] [batchSize:<uint32>]
    timeRoll:  [timeCalc:<string>] [minutes:<uint32>] [hours:<uint32>] [days:<uint32>] 
      [date:<string>]

To get more information about a specific message or command, you can specify the msg=<message name> on the help command as a parameter. For example, look at the pull message help:

send /logs help msg=pull 

   pull: Downloads N log entries
    security.roles: logs.manage
    parameters:
    id1 - <uint64, optional> The first log id number to retrieve, this is 
       mutually exclusive with id2
    id2 - <uint64, optional> The last log id number that will be sent, defaults to 
       most recent log message when id1 or id2 is not sent
    count - <uint32, optional, {range:1 to 10000}> The number of logs to pull
    timeFormat - <string, optional, {enum-one:posix|simple}> The time format used in each 
       log message, default is posix time (seconds since 1970)

The built-in message help says that this command grabs the last N log entries if you omit ID1 and ID2. To look at the last 10 log entries:

send /logs pull count=10 timeFormat=simple

Almost all of the commands on the service follow this simple format. The only commands that do not are the ones that require more complicated handshaking, like importing a PCAP to a Decoder. To import a PCAP, use the NwConsole import command, which takes care of the complicated communication channel handshaking.

Some parameters are specific to NwConsole's send command and are not actually sent to the service. You can use these parameters to change the output format of the response, write the response to a file, or read a file from the local machine and send it to the service. The local parameters to NwConsole's send command all start with two dashes --.

  • --output-format — This parameter changes the normal output of the command from plain text to one of these types: JSON, XML, or HTML. The format values is a text output that only writes values (query output, stat/config values, and so on) without any other decorations.
  • --output-pathname — Instead of writing the output to the terminal, the output is written to the specified pathname (truncates any existing file).
  • --output-append-pathname — This is the same as --output-pathname except that it appends the output to an existing file (or creates the file if it does not exist).
  • --file-data — Reads in a file and uses it as the command payload. This is useful for commands like /sys fileEdit. The following example shows how you can send an updated index-concentrator-custom.xml file using NwConsole:

send /sys fileEdit op=put filename=index-concentrator-custom.xml --file-data="/Users/user/Documents/index-concentrator-custom.xml"

  • --file-format — When reading an input file with --file-data, this parameter forces NwConsole to interpret the file as a specific type of input. The allowed enumerations are: binary, params, params-list, string and params-binary. As an example, to send a file of application rules (*.nwr) to a Decoder, you can use this command:

send /decoder/config/rules/application replace --file-data=/path/rules.nwr --file-format=params-list

  • --string-data — Sends the command payload as a string instead of a list of parameters.
  • --binary-data — Sends the command payload as binary instead of a list of parameters.

Example Streaming Query to JSON file (could be a large result set):

send /sdk query size=0 query="select * where service=80 && time='2018-03-05 13:00:00'-'2018-03-05 13:59:59'" --output-format=json --output-pathname=/tmp/query.json

One thing to note about the send command is the fact that, by default, there is a timeout of 30 seconds waiting for a response. Some commands (like the query above) may take longer to receive results. To avoid a premature client-side timeout, you can use the timeout [secs] command to increase the wait. For instance, timeout 600 would wait 10 minutes for a response before timing out. Once enacted, it takes effect for all subsequent commands.

To navigate around the virtual node hierarchy of the service, you can use the cd command like you would on any command shell. This covers the basics of connecting and interacting with a service. Once you are connected, the help command lists all the commands that you can use to interact with the endpoint. These commands do not display when you are not connected to an endpoint.