Decrypt Incoming Packets (TLS 1.2)

Administrators can configure a Network Decoder to decrypt incoming network packets. This capability is currently supported with encrypted communications to managed servers using TLS 1.2 or earlier and certain cipher suites, which are described in Supported Cipher Suites . The following figure illustrates the supported configuration of a Network Decoder deployed as a passive network device where it captures network traffic for managed servers. The Network Decoder is able to decrypt the traffic by using private keys uploaded to it from the managed servers .

netwitness_decryption-networkdecoder.png

As the network traffic is collected and passed through the system, the encrypted packets are stored as captured even if the system has been able to decrypt them. As the Decoder parses the encrypted traffic, it uses any private keys that have been uploaded, and determines if the network traffic can be decrypted. If the Decoder is able to decrypt the traffic, it generates an additional meta key, defined as tls.premaster, which contains the decryption key for that particular session. Whether the traffic is decrypted or not, the metadata service is tagged as 443 or SSL/TLS traffic, as the Decoder is port agnostic, even though it is marked with the default port number. The following figure shows the basic progression through the NetWitness platform as the packets are being decrypted and the session metadata is generated.

netwitness_decryption-sessionmeta.png

Follow the steps below to configure the Network Decoder to natively decrypt this network traffic.

Note: Currently, you cannot review the decypted payloads directly in the in the user interface. However, you can review them using the RESTful API interface and the NwConsole. However, the Decoders being able to decrypt traffic enables them to detect and generate potentially malicious communications that would otherwise not be detected.

See Troubleshooting for resolving any issues you might encounter while performing these steps.

Step 1: Validate That The Network Decoder Captures Encrypted Traffic

First, make sure the Decoder is actually capturing the SSL/TLS traffic from the managed servers that you want to use to gain visibility into the encrypted sessions. You can do this by searching in the Investigate views (Navigate or Events) using the IP addresses of the servers as the destination IP addresses, and the service as SSL/TLS.

For example, if you are searching for HTTPS traffic to 192.168.1.2 and 192.168.1.3, you can run the following query:

service = 443 && ip.dst = 192.168.1.2, 192.168.1.3

Conduct this search on the Concentrator that aggregates metadata from the Decoder that is expected to capture this network segment, or on a Broker that may have access to all collected metadata.

In case of simultaneous ingestion of the encrypted and decrypted traffic streams in the Decoder, ensure that multi-thread assembly is enabled. This is to prevent mixing of packets from different sessions by single thread assembly. For more information, see the following topics:

Step 2: Obtain Private Keys from Managed Servers

Private keys are the asymmetric keys generated by services that accept TLS traffic. They are normally stored in PEM files. These keys are used during the TLS handshake to encrypt the premaster symmetric key that is used for the rest of the payload encryption.

Note: The method to export private keys from web servers (from which to decrypt traffic) varies depending on the product. Refer to the product documentation for instructions for exporting private keys.

Extracting Certificates for the Decoder SSL DecryptionExtracting Certificates for the Decoder SSL Decryption

Extracting the proper key materials from popular servers is required to leverage the Decoder built-in decryption capability. The following are a few examples to extract private keys and certificates from popular servers like Apache, Microsoft IIS, and Exchange.

Note: These procedures can also be used as references to extract keys from other servers as well.

Extract private key from Apache

The private key file location would be referenced in the main Apache configuration file, which is httpd.conf or apache2.conf. The config directive SSLCertificateKeyFile will specify the path on your server where private keys are stored.

If Apache is using OpenSSL default certificates and keys, then locate private keys (which are usually saved to the folder /usr/local/ssl by default). You can run the command openssl version –a to find OPENSSLDIR where your server is saving the keys.

If the private key is password protected, then it can be extracted using the below command with the password.

openssl rsa -in yourcertificate.pem -out serverkey.pem

Validate the located private key using the following command.

openssl rsa -in serverkey.pem -check -noout

Extract Private Key and Certificate from Windows Servers

Windows servers provide the interface to extract certificates and private keys in PKCS12 format files (.pfx format). It is recommended to use password protection while exporting private keys.

After exporting keys to the pfx file, you can use the OpenSSL tool to extract private keys as PEM files and upload them to Decoder.

Step 2 (a): Extract PKCS12 file from Internet Information Services (IIS)

There are multiple ways to extract certificates from IIS and the following are few examples.

Using IIS Manager App

  • Open Internet Information Services Manager App.

  • Under IIS > Navigate to Select Certificates > Select Features view > Select the required web or application server certificate.

  • Right-click the certificate and select the Export option. In the export wizard, select the file path to save the pfx file (PKCS12 format)

  • Example: C:\ServerCerts\servercrt.pfx.

  • Enter the password and confirm the password in the export.

  • This would create password protected pfx file which contains a private key and associated certificate.

Using Microsoft Management Console (MMC)

  • Open Microsoft Management Console (MMC).

  • In the Console Root expand Certificates (Local Computer).

  • If the Certificates folder is not available then navigate to File > Add Snap In > Add Certificates.

  • Locate your server certificate, it is located in the Web Server or Personal sub-folders.

  • Right-click the certificate, identified by the Common Name, select Export, and follow the wizard. Use password to protect the pfx file.

  • This would create password protected pfx file which contains a private key and associated certificate.

Step 2 (b): Extract PKCS12 file from Exchange Servers

Refer the Export a Certificate from an Exchange Server document for information on how to extract the pfx file from the Exchange server.

Step 2 (c): Extract Private Key From a pfx File Using OpenSSL

OpenSSL tool provides API to export keys in PEM format.

  1. Extract private key in PEM format, at the openssl prompt use the password to decrypt the pfx file.
    openssl pkcs12 -in servercrt.pfx -nocerts -nodes -out serverkey.pem

  2. Validate the private key generated

    openssl rsa -in serverkey.pem -check -noout

  3. Extract cert in PEM format to validate compatibility with private key

    openssl pkcs12 -in servercrt.pfx -nokeys -out serverkey.crt

  4. Validate private key and certificate modulus. The modulus should be the same for both the private key and its certificate.

    openssl rsa -noout -modulus -in serverkey.pem

    openssl x509 -noout -modulus -in serverkey.crt

Recommendations

Once private keys are extracted from servers and imported to Decoder, it is recommended to secure or delete the extracted PEM file of the private key (serverkey.pem) generated by the above OpenSSL commands.

Step 3: Validate That The Private Key Cipher Suite is Supported

As a passive collection device, the Network Decoder can only decrypt ciphers that use the RSA key exchange. The list of supported keys, and whether the keys are FIPS compliant, are listed in Supported Cipher Suites and Unsupported Cipher Suites . If you are expecting to decrypt less-secure channels of communication, you might need to disable FIPS. However, this is a system change to the entire Decoder, not just the parsing of captured traffic, so take caution before disabling FIPS. If you require more-secure cipher suites for the managed server communications, search for "SSL decrypt" in our third party partner integrations section at NetWitness Community (https://community.netwitness.com/t5/netwitness-discussions/bd-p/netwitness-discussions).

Note: If FIPS is enabled, the list of ciphers for decryption is restricted to only those that are FIPS-compliant.

Supported Cipher Suites

The following table lists which cipher-suites are supported using private keys.

Cipher Suite Name (RFC) Name (OpenSSL) Cipher Suite TLS Version KeyExch. FIPS
TLS_RSA_WITH_AES_256_GCM_SHA384 AES256-GCM-SHA384 [0x9d] TLSv1.2 Kx=RSA Compliant
TLS_RSA_WITH_AES_256_CBC_SHA256 AES256-SHA256 [0x3d] TLSv1.2 Kx=RSA Compliant
TLS_RSA_WITH_AES_256_CBC_SHA AES256-SHA [0x35] SSLv3 Kx=RSA Compliant
TLS_RSA_WITH_CAMELLIA_256_CBC_SHA CAMELLIA256-SHA [0x84] SSLv3 Kx=RSA Non-Compliant
TLS_RSA_WITH_3DES_EDE_CBC_SHA DES-CBC3-SHA [0x0a] SSLv3 Kx=RSA Compliant
TLS_RSA_WITH_AES_128_GCM_SHA256 AES128-GCM-SHA256 [0x9c] TLSv1.2 Kx=RSA Compliant
TLS_RSA_WITH_AES_128_CBC_SHA256 AES128-SHA256 [0x3c] TLSv1.2 Kx=RSA Compliant
TLS_RSA_WITH_AES_128_CBC_SHA AES128-SHA [0x2f] SSLv3 Kx=RSA Compliant
TLS_RSA_WITH_SEED_CBC_SHA SEED-SHA [0x96] SSLv3 Kx=RSA Non-Compliant
TLS_RSA_WITH_CAMELLIA_128_CBC_SHA CAMELLIA128-SHA [0x41] SSLv3 Kx=RSA Non-Compliant
TLS_RSA_WITH_IDEA_CBC_SHA IDEA-CBC-SHA [0x07] SSLv3 Kx=RSA Non-Compliant
TLS_RSA_WITH_RC4_128_SHA RC4-SHA [0x05] SSLv3 Kx=RSA Non-Compliant
TLS_RSA_WITH_DES_CBC_SHA DES-CBC-SHA [0x09] SSLv3 Kx=RSA Non-Compliant
TLS_RSA_EXPORT_WITH_DES40_CBC_SHA EXP-DES-CBC-SHA [0x08] SSLv3 Kx=DES Non-Compliant

Unsupported Cipher Suites

The following table lists which cipher-suites are not supported using private keys.

Cipher Suite Name (RFC) Name (OpenSSL) Cipher Suite TLS Version KeyExch. FIPS
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 ECDHE-RSA-AES256-GCM-SHA384 [0xc030] TLSv1.2 Kx=ECDH Compliant
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 ECDHE-ECDSA-AES256-GCM-SHA384 [0xc02c] TLSv1.2 Kx=ECDH Compliant
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 ECDHE-RSA-AES256-SHA384 [0xc028] TLSv1.2 Kx=ECDH Compliant
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 ECDHE-ECDSA-AES256-SHA384 [0xc024] TLSv1.2 Kx=ECDH Compliant
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA ECDHE-RSA-AES256-SHA [0xc014] SSLv3 Kx=ECDH Compliant
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA ECDHE-ECDSA-AES256-SHA [0xc00a] SSLv3 Kx=ECDH Compliant
TLS_DHE_DSS_WITH_AES_256_GCM_SHA384 DHE-DSS-AES256-GCM-SHA384 [0xa3] TLSv1.2 Kx=DH Compliant
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 DHE-RSA-AES256-GCM-SHA384 [0x9f] TLSv1.2 Kx=DH Compliant
TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 DHE-RSA-AES256-SHA256 [0x6b] TLSv1.2 Kx=DH Compliant
TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 DHE-DSS-AES256-SHA256 [0x6a] TLSv1.2 Kx=DH Compliant
TLS_DHE_RSA_WITH_AES_256_CBC_SHA DHE-RSA-AES256-SHA [0x39] SSLv3 Kx=DH Compliant
TLS_DHE_DSS_WITH_AES_256_CBC_SHA DHE-DSS-AES256-SHA [0x38] SSLv3 Kx=DH Compliant
TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA DHE-RSA-CAMELLIA256-SHA [0x88] SSLv3 Kx=DH Non-Compliant
TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA DHE-DSS-CAMELLIA256-SHA [0x87] SSLv3 Kx=DH Non-Compliant
TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 ECDH-RSA-AES256-GCM-SHA384 [0xc032] SSLv3 Kx=ECDH/RSA Compliant
TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 ECDH-ECDSA-AES256-GCM-SHA384 [0xc02e] TLSv1.2 Kx=ECDH/ECDSA Compliant
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 ECDH-RSA-AES256-SHA384 [0xc02a] TLSv1.2 Kx=ECDH/RSA Compliant
TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 ECDH-ECDSA-AES256-SHA384 [0xc026] TLSv1.2 Kx=ECDH/ECDSA Compliant
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA ECDH-RSA-AES256-SHA [0xc00f] SSLv3 Kx=ECDH/RSA Compliant
TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA ECDH-ECDSA-AES256-SHA [0xc005] SSLv3 Kx=ECDH/ECDSA Compliant
TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA ECDHE-RSA-DES-CBC3-SHA [0xc012] SSLv3 Kx=ECDH Compliant
TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA ECDHE-ECDSA-DES-CBC3-SHA [0xc008] SSLv3 Kx=ECDH Compliant
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA EDH-RSA-DES-CBC3-SHA [0x16] SSLv3 Kx=DH Compliant
TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA EDH-DSS-DES-CBC3-SHA [0x13] SSLv3 Kx=DH Compliant
TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA ECDH-RSA-DES-CBC3-SHA [0xc00d] SSLv3 Kx=ECDH/RSA Compliant
TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA ECDH-ECDSA-DES-CBC3-SHA [0xc003] SSLv3 Kx=ECDH/ECDSA Compliant
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 ECDHE-RSA-AES128-GCM-SHA256 [0xc02f] TLSv1.2 Kx=ECDH Compliant
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 ECDHE-ECDSA-AES128-GCM-SHA256 [0xc02b] TLSv1.2 Kx=ECDH Compliant
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 ECDHE-RSA-AES128-SHA256 [0xc027] TLSv1.2 Kx=ECDH Compliant
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 ECDHE-ECDSA-AES128-SHA256 [0xc023] TLSv1.2 Kx=ECDH Compliant
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA ECDHE-RSA-AES128-SHA [0xc013] SSLv3 Kx=ECDH Compliant
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA ECDHE-ECDSA-AES128-SHA [0xc009] SSLv3 Kx=ECDH Compliant
TLS_DHE_DSS_WITH_AES_128_GCM_SHA256 DHE-DSS-AES128-GCM-SHA256 [0xa2] TLSv1.2 Kx=DH Compliant
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 DHE-RSA-AES128-GCM-SHA256 [0x9e] TLSv1.2 Kx=DH Compliant
TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 DHE-RSA-AES128-SHA256 [0x67] TLSv1.2 Kx=DH Compliant
TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 DHE-DSS-AES128-SHA256 [0x40] TLSv1.2 Kx=DH Compliant
TLS_DHE_RSA_WITH_AES_128_CBC_SHA DHE-RSA-AES128-SHA [0x33] SSLv3 Kx=DH Compliant
TLS_DHE_DSS_WITH_AES_128_CBC_SHA DHE-DSS-AES128-SHA [0x32] SSLv3 Kx=DH Compliant
TLS_DHE_RSA_WITH_SEED_CBC_SHA DHE-RSA-SEED-SHA [0x9a] SSLv3 Kx=DH Non-Compliant
TLS_DHE_DSS_WITH_SEED_CBC_SHA DHE-DSS-SEED-SHA [0x99] SSLv3 Kx=DH Non-Compliant
TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA DHE-RSA-CAMELLIA128-SHA [0x45] SSLv3 Kx=DH Non-Compliant
TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA DHE-DSS-CAMELLIA128-SHA [0x44] SSLv3 Kx=DH Non-Compliant
TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 ECDH-RSA-AES128-GCM-SHA256 [0xc031] TLSv1.2 Kx=ECDH/RSA Compliant
TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 ECDH-ECDSA-AES128-GCM-SHA256 [0xc02d] TLSv1.2 Kx=ECDH/ECDSA Compliant
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 ECDH-RSA-AES128-SHA256 [0xc029] TLSv1.2 Kx=ECDH/RSA Compliant
TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 ECDH-ECDSA-AES128-SHA256 [0xc025] TLSv1.2 Kx=ECDH/ECDSA Compliant
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA ECDH-RSA-AES128-SHA [0xc00e] SSLv3 Kx=ECDH/RSA Compliant
TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA ECDH-ECDSA-AES128-SHA [0xc004] SSLv3 Kx=ECDH/ECDSA Compliant
TLS_ECDHE_RSA_WITH_RC4_128_SHA ECDHE-RSA-RC4-SHA [0xc011] SSLv3 Kx=ECDH Non-Compliant
TLS_ECDHE_ECDSA_WITH_RC4_128_SHA ECDHE-ECDSA-RC4-SHA [0xc007] SSLv3 Kx=ECDH Non-Compliant
TLS_ECDH_RSA_WITH_RC4_128_SHA ECDH-RSA-RC4-SHA [0xc00c] SSLv3 Kx=ECDH/RSA Non-Compliant
TLS_ECDH_ECDSA_WITH_RC4_128_SHA ECDH-ECDSA-RC4-SHA [0xc002] SSLv3 Kx=ECDH/ECDSA Non-Compliant
TLS_DHE_RSA_WITH_DES_CBC_SHA EDH-RSA-DES-CBC-SHA [0x15] SSLv3 Kx=RSA Non-Compliant
TLS_DHE_DSS_WITH_DES_CBC_SHA EDH-DSS-DES-CBC-SHA [0x12] SSLv3 Kx=DSS Non-Compliant
TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA EXP-EDH-RSA-DES-CBC-SHA [0x14] SSLv3 Kx=DSS Non-Compliant
TLS_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA EXP-EDH-DSS-DES-CBC-SHA [0x11] SSLv3 Kx=DSS Non-Compliant

Step 4: Confirm HTTPS Parser is Enabled on Decoders

The decryption process requires the native HTTPS parser to be enabled to examine the TLS handshake and to compare the uploaded encryption keys to determine if they decrypt the traffic. This can be validated in netwitness_adminicon_25x22.png (Admin) > Services > Decoder > Config > Parsers Configuration panel as shown in the following figure.

netwitness_https-parser-enabled.png

Step 5: Upload the Supported Private Keys to Decoders

The sslKeys command provides a way to upload premaster or private keys to the Decoder, so that captured encrypted packets that match the keys can be decrypted before parsing. Administrators configure the Decoder by entering the sslKeys command using the NwConsole command line interface or the Decoder RESTful interface. After upload, you must issue a parser reload command so that the newly installed key becomes visible to the HTTPS parser. Now, all TLS handshakes that use that private key will be able to be decrypted by the Decoder.

For information about how to use the NwConsole command line interface, see the NwConsole User Guide for NetWitness Platform. For information about how to use the RESTful interface, see the RESTful API User Guide for NetWitness Platform. Go to the NetWitness All Versions Documents page and find NetWitness Platform guides to troubleshoot issues.

netwitness_sslkeysrest.png

The RESTful interface form at the path /decoder/sslkeys allows uploading a single PEM-encoded private key, a single file containing multiple private keys concatenated together, or a single file of multiple premaster keys.

netwitness_sslrestready.png

Upload Multiple Premaster and Private Keys

Note: Uploading premaster keys is not supported for real-time capture, but can be used to validate that decryption is functioning properly on the Decoder.

You can use the RESTful interface form to facilitate uploading of multiple keys, both premaster and private at the same time.

  1. Open the RESTful API in your browser and navigate to this path on the Decoder that you want to configure: /decoder/sslkeys.

    netwitness_restformsslkeys.png

  2. Next to Upload File 1, click Choose File and locate the premaster key file or PEM file that you want to upload on your local file system.

  3. (Optional) Repeat for Upload File 2 and Upload File 3.

    netwitness_sslrestready.png

  4. Click Upload.

    The files are uploaded to the Decoder and results are displayed in the form.

    netwitness_sslrestfrmsucces.png

Parameters for Managing Keys

The sslKeys command has several parameters for managing premaster and private keys. This is the full list of parameters:

Parameter Description
clear Removes all premaster keys from memory. Does not delete any PEM files installed on the system.
maxKeys Changes the maximum number of premaster keys that are stored in memory.
listPems Returns a list of all installed private key PEM files.
deletePem Deletes the named PEM file from the file system. You can pass this parameter more than once to remove multiple files.
random The random hash used to identify the premaster key.
premaster The premaster key that will be installed for the previous random parameter. They must show up in pairs and random must be first.

Return Values

Most sslKeys commands return name/value pairs of statistics about the premaster keys in memory. The statistics are listed in the following table.

Name Description
added The number of premaster keys just added during this command.
total The total number of premaster keys loaded in memory.
agedOut The total number of premaster keys that were removed during this command; this is not a lifetime stat.
maxKeys The current maximum allowed premaster keys

Encryption Keys

The sslKeys command accepts two types of encryption keys:

  • Premaster key: the symmetric key used in the TLS payload stream for encryption and decryption.
  • Private key: the asymmetric private key used during the TLS handshake that encrypts the premaster.

Premaster Key

The premaster key is generated randomly and is ephemeral for the life of one specific TLS session. Normally, there is not a good way to get premaster keys to a Decoder in time for the parsing step. However, both Chrome and Firefox can write the premaster keys they generate to a file. This is useful for testing purposes. To configure your browser to do this, create an environment variable called SSLKEYLOGFILE and assign it the pathname of a file to which the keys will be written. The Decoder will accept the file exactly as written and will use all the decryption keys in the file for any encrypted traffic it captures.

This is a sample NwConsole script that uploads the file to a Decoder:

login <decoder>:50004 <username> <password>

send /decoder sslKeys --file-data=SSLKeys.txt

This is an example using a curl command (with the RESTful port) to upload the file to a Decoder:

curl -u "<username>:<password>" -H "Content-Type: application/octet-stream" --data-binary @"/path/SSLKeys.txt" -X POST "http://<hostname>:50104/decoder?msg=sslKeys"

After the symmetric keys are uploaded, they will immediately be used for any necessary decryption. Symmetric keys are stored in memory and there is a limit to how many can be stored at any point in time. As more keys are added, the earliest keys will be aged out. You can also add premaster keys by just passing the random and premaster parameters to sslKeys.

Private Keys or PEM Files

Private keys are normally stored in PEM files and are the asymmetric keys generated by services that accept TLS traffic. These keys are used during the TLS handshake to encrypt the premaster symmetric key that will be used for the rest of the payload encryption.

For example, if you have a web server whose traffic you want visibility into, you need to upload the private key it uses to encrypt traffic. You only need to do this once, as it is stored permanently (or until removed by a delete command). Private keys are automatically encrypted before storing to protect them. After upload, you must issue a parser reload command so that the newly installed key becomes visible to the HTTPS parser. Now, all TLS handshakes that use that private key will be able to be decrypted by the Decoder.

Note: Not all ciphers suites use a "known" private key (for example, Ephemeral Diffie Hellman). Encrypted traffic with those ciphers cannot be decrypted unless the premaster key is uploaded to the Decoder before the session is parsed.

These are some sample commands that upload a PEM file to be used for decryption.

Using NwConsole:

send /decoder sslKeys pemFilename=MyKey.pem --file-data=/path/MyKey.pem

Using the RESTful interface (you must provide the pemFilename parameter in the URL):

curl -u "<username>:<password>" -H "Content-Type: application/octet-stream" --data-binary @"/path/MyKey.pem" -X POST "http://<hostname>:50104/decoder?msg=sslKeys&pemFilename=MyKey.pem"

Step 6: Reload HTTPS Parser on Decoders

After uploading the private keys, you must issue a parser reload command so that the newly-installed key becomes visible to the HTTPS parser. All TLS handshakes that use that private key are now able to be decrypted by the Decoder.

To issue a parser reload command:

  1. Go to netwitness_adminicon_25x22.png (Admin) > Services and select the Decoder.
  2. Click netwitness_ic-actns.png > View > Explore > decoder, right-click parsers and select Properties.
  3. In the Properties for Decoder (DECODER) /decoder/parsers pane, select reload from the drop-down menu and click Send.
    netwitness_reload-parsers.png

Step 7: View Decrypted Sessions

All storage of the packets is still encrypted, so when you search for any decrypted traffic, your query should still focus on service=443 or HTTPS (HTTP over SSL/TLS). If any packets were able to be decrypted, tls.premaster metadata is associated with the session that has the premaster key required for decrypting that session. The session will also have crypto metadata denoting the cipher suite, with crypto formated as tls.premaster.

Viewing Unencrypted Traffic

If packets are decrypted during the parse stage, encrypted packets are written to disk, and the matching premaster key used for decrypting is written to the tls.premaster meta key, analysts can view the unencrypted packets using the tls.premaster meta key.

One Decoder API that you can use to see the unencrypted packets is the /sdk/content RESTful service. You need to know the Session ID of the encrypted packets and the flags parameter masked to the value 128 (or 0x80 in hex). Point your browser to the Decoder RESTful interface and type in the following command, substituting the actual Session ID for <id>:

http://<decoder>:50104/sdk/content?session=<id>&flags=128&render=text

The Decoder returns a simple web page showing the packets after they are decrypted.

If you want to see what the packets look like encrypted, type in one of the following commands, substituting the Session ID for <id>:

http://<decoder>:50104/sdk/content&session=<id>&render=text

http://<decoder>:50104/sdk/content&session=<id>&flags&render=text

For more information on the /sdk/content service, see the manual page for /sdk content.

Alternatively, you can use the NetWitness Investigator thick client to decrypt and display the network sessions, as it can communicate to a Concentrator or have PCAPs and private keys loaded locally.

If you are not able to view the encrypted sessions, check Troubleshooting. However, if you are able to see the tls.premaster metadata, that confirms that the Decoder is able to see inside the encrypted session, which enables parsers to see the unencrypted packet payload and create metadata accordingly. For these decrypted sessions, look for additional metadata around usernames, passwords, or HTTP body content that would be obfuscated when encrypted, which would result in metadata not being generated.

Troubleshooting

If you are still not able to decrypt or view the decrypted payloads after attempting the steps above, try these steps.

  1. Validate that your private key is using an acceptable cipher-suite.
  2. Make sure your Network Decoder is capturing the traffic matching the server for which you have the private key. To check, you can upload the private key and PCAP into Wireshark and see if it is able to decrypt it.

Decryption of Secure SMTP

NetWitness supports opportunistic SSL/TLS decryption, which addresses RFC 3207 (https://tools.ietf.org/html/rfc3207). You can add an HTTPS parser option that provides a .csv list of destination ports of the session where the STARTTLS command will be searched, with at least one encryption key that has been uploaded. This enables the STARTTLS functionality.

Note: The .csv list must include all the destination ports of a session. If a destination port is not in the list, the search for STARTTLS will not be started.

The STARTTLS search is limited to the first 1024 bytes of the session's payload, to prevent performance degradation. If STARTTLS is found, the parser uses the next client packet with payload as the start of a TLS negotiation.

To add an HTTPS parser option for secure SMTP decryption:

  1. In the NetWitness User Interface, go to netwitness_adminicon_25x22.png (Admin) > Services.
  2. Select a Log Decoder service and click netwitness_ic-actns2.png > View > Explore.
  3. In the left panel, select decoder > parsers > config.

  4. In parsers.options, enter an HTTPS parser option in the following format that provides a .csv list of the destination ports of the session where the STARTTLS command is searched:

    HTTPS="smtpsPorts=<port#1,port#2,port#n>"

    The following image shows an example of an HTTPS parser option for secure SMTP decryption.

    netwitness_12.1_smtp-decrypt-https-option_1122.png

Performance Considerations

Decrypting packets in real time requires extra work in the parsing stage. Before implementing this feature, plan carefully to ensure the incoming traffic bandwidth does not overwhelm the available compute power. You may need more Decoders to decrypt traffic than you would need if not decrypting.

Packets captured on a Decoder normally have a timeout of ~60 seconds in the assembly stage before they are sent to the parsing step. If the Decoder is under memory pressure due to very high bandwidth, the lifetime of the packets in Assembler may be shortened. To alleviate this situation, you can configure a longer timeout value and increase the amount of memory available to hold packets in Assembly. Also, in order to perform decryption of the packets, the Decoder must receive the decryption key before the parsing stage.

Note: Currently, only TLS 1.2 and earlier protocols can be decrypted

With no feeds loaded, the following parsers enabled, and 50% of the sessions being decrypted, a Decoder can process traffic at 3 Gbps .

Parser Name Description
SYSTEM Session Details
NETWORK Network Layer
ALERTS Alerts
GeoIP Geographic data based on ip.src and ip.dst
GeoIP2 Geographic data by default based on IPv4 (ip.src, ip.dst) and IPv6 (ipv6.src, ipv6.dst) meta keys
HTTP Hyper Text Transport Protocol (HTTP)
HTTP_Lua Hyper Text Transport Protocol (HTTP) Lua
FTP File Transfer Protocol (FTP)
TELNET TELNET Protocol

SMTP

Simple Mail Transport Protocol (SMTP)
POP3 Post Office Protocol (POP3)

NNTP

Network News Transport Protocol (NNTP)
DNS Domain Name Service (DNS)

HTTPS

Secure Socket Layer (SSL) Protocol
MAIL Standard E-Mail Format (RFC822)

VCARD

Extracts VCARD fullname and email information
PGP Identifies PGP blocks within network traffic

SMIME

Identifies SMIME blocks within network traffic
SSH Secure Shell (SSH)

TFTP

Trivial File Transfer Protocol (TFTP)
DHCP Dynamic Host Configuration Protocol (DHCP and BOOTP)

NETBIOS

Extracts NETBIOS computer name information.
SNMP Simple Network Management Protocol (SNMP)

NFS

Network File System (NFS) protocol
RIP Routing Information Protocol (RIP).
TDS MSSQL and Sybase database protocol (TDS)

TNS

Oracle database protocol (TNS)
IRC Internet Relay Chat (IRC) protocol

RTP

Real Time Protocol (RTP) for audio/video
SIP Session Initiation Protocol (SIP)

H323

H.323 Teleconferencing protocol
SCCP Cisco Skinny Client Control Protocol

GTalk

Google Talk (GTalk)
VlanGre Vlan ID and GRE/EtherIP tunnel addresses

BITTORRENT

BitTorrent File Sharing Protocol
FIX Financial Information eXchange Protocol

GNUTELLA

Gnutella file sharing protocol
IMAP Internet Message Access Protocol

MSRPC

Microsoft Remote Procedure Call protocol
RDP Remote Desktop Protocol

SHELL

Command Shell Identification
TLSv1 TLSv1

SearchEngines

A parser that extracts search terms
FeedParser External Feed Parser

TLS Certificate Hashing

The Network Decoder can produce hashes of certificates that are seen in the packet stream. These hashes are the SHA-1 value of any DER-encoded certificate encountered during a TLS handshake. The hashes produced can be used to compare network traffic with hashes from public SSL blacklists, such as the one from sslbl.abuse.ch.

The TLS Certificate hashing feature is disabled by default. To enable TLS Certificate hashing:

  1. Go to netwitness_adminicon_25x22.png (Admin) > Services, select a Network Decoder service and netwitness_ic-actns.png​ > View > Explore.
  2. Select decoder > parsers > config.
  3. In the values column next to parsers.options, type HTTPS="cert.sha1=true".
    When this option is enabled, the SHA-1 is stored as a text value in the cert.thumbprint meta key.
  4. The change takes effect on parser reload. In the left panel, right-click parsers and click Properties. In the drop-down menu, select reload and then click Send

The SHA-256 hash of the certificate can be enabled by adding the parser option:

HTTPS="cert.sha256=true"

to a Network Decoder's /decoder/parsers/config/parsers.options configuration.

When this option is enabled, SHA-256 is stored as a text value in the meta keys:
cert.sha256
cert.thumbprint

JA3 and JA3S TLS Fingerprints

The Network Decoder can produce the JA3 value of TLS clients and the JA3S value of TLS servers that are observed in a Network session. The values that are produced conform to the values generated by the open source JA3 tools (https://github.com/salesforce/ja3). The JA3 features are disabled by default.

To enable JA3 features:

  1. In the NetWitness Platform User Interface, go to netwitness_adminicon_25x22.png (Admin) > Services.
  2. Select a Decoder service and click netwitness_ic-actns.png​ > View > Explore.
  3. In the left panel, select decoder > parsers > config.

  4. In parsers.options, add HTTPS="ja3=true ja3s=true".

  5. In the left panel, right click on parsers and select properties.

  6. In the right panel, in the lower pane, set the value in the first drop down to Reload.

  7. Click Send.

JA3 and JA3S can be enabled independently using these options. When enabled, the JA3 is stored as a text value in the meta key ja3. The JA3S is stored as a text value in the meta key ja3s.

Additional meta keys are activated using a mechanism similar to the JA3S and JA3S meta items. For example, ja3.orig is enabled by adding ja3.orig=true to the HTTPS parser options. This is also true for ja3s.orig, tls.extensionlen, and tls.cipherlen.

ja3.orig, tls.extensionlen, and tls.cipherlen are only created if JA3 is enabled.

ja3s.orig is only created if JA3S is enabled.

JA4 TLS Fingerprints

The Network Decoder can produce the JA4 value of TLS clients that are observed in a Network session. The values that are produced conform to the values generated by the open source JA4 tools (https://github.com/FoxIO-LLC/ja4). The JA4 features are disabled by default.

To enable JA4 features:

  1. In the NetWitness Platform User Interface, go to AdminIcon.png (Admin) > Services.
  2. Select a Decoder service and click ic-actns.png​  > View > Explore.
  3. In the left panel, select decoder > parsers > config.

  4. In parsers.options, add HTTPS="ja4=true".

  5. In the left panel, right click on parsers and select properties.

  6. In the right panel, in the lower pane, set the value in the first drop down to Reload.

  7. Click Send.

JA4 can be enabled independently using these options. When enabled, the JA4 is stored as a text value in the meta key ja4.

Additional meta keys are activated using a mechanism similar to the JA4 meta items. For example, ja4.orig is enabled by adding ja4.orig=true to the HTTPS parser options.

The ja4.orig is only created if JA4 is enabled.

Perform Simultaneous Ingestion of the Encrypted and Decrypted Traffic Streams to Decoder

In previous versions, ingesting encrypted and decrypted traffic streams of the same session to a Decoder causes packets from both the sessions to mix on the Decoder as reassembly is performed by the same thread. It cause session parsing and content extraction inaccuracies.

Using multi-adapter capture and multi-thread assembler features, you can configure the Decoder to receive encrypted and decrypted streams on separate adapters. The multi-thread assembler feature allows decoder to assemble packets from its corresponding capture work thread. It keeps the packets from encrypted and decrypted sessions separate during assembly and avoid inaccuracies in session parsing and content extraction.

To configure the Decoder for the simultaneous ingestion of the encrypted and decrypted traffic streams:

  1. Configure Decoder to capture from multiple adapters simultaneously, see (Optional) Multiple Adapter Packet Capture.

  2. Set /decoder/config/assembler.threading.enabled=on to enable multi-thread assembler.

  3. Restart the Decoder service.

Community ID

The Network Decoder generates Community ID flow hash values that are compatible with the Community ID specification defined by https://github.com/corelight/community-id-spec.

The Community ID is written to a text-formatted meta key named community.

The Community ID is generated on sessions containing TCP over IPv4, TCP over IPv6, UDP over IPV4, and UDP over IPV6.