Decrypt Incoming Packets TLS 1.3

Administrators can configure a nonfips Network Decoder to decrypt incoming TLS 1.3 network packets. This capability is currently supported with encrypted communications to managed servers using TLS 1.3 or earlier.

 

For TLS 1.2 and earlier protocols refer Decrypt Incoming Packets (TLS 1.2)

netwitness_tls1.3_arch.png

As the network traffic is collected and passed through the system, the encrypted packets are stored as captured even if the system can decrypt them. As the Decoder parses the encrypted traffic, it uses TLS 1.3 keys that have been pushed or uploaded, and determines if the network traffic can be decrypted. If the Decoder is able to decrypt the traffic, it generates an additional meta keys, defined as tls.client.hdshk, tls.server.hdshk, tls.client.app, tls.server.app, tls.client.early which contains the decryption keys for that particular session. Whether the traffic is decrypted or not, the metadata service is tagged as 443 or SSL/TLS traffic, as the Decoder is port agnostic, even though it is marked with the default port number.

Non-FIPS BuildsNon-FIPS Builds

Network Decoder TLS 1.3 decryption is supported ONLY through special nonfips rpm builds which run in NON FIPS mode. The nonfips builds use crypto library which has pending FIPS certification in NIST, hence they are labeled as nonfips. If you are restricted to run only FIPS in your environment then TLS 1.3 decryption feature is not supported.

The following nonfips builds for components Decoder, Appliance , Concentrator and Broker are required to be installed, as they provide dependencies and support to TLS 1.3 decryption in session parsing and during content reconstruction.

 

Decoder (rsa-nw-decoder-nonfips rpm) Decrypt on Session Parsing and Content Reconstruction
Concentrator (rsa-nw-concentrator-nonfips rpm) Decrypt on Content Reconstruction
Broker (rsa-nw-broker-nonfips rpm) Decrypt on Content Reconstruction
Appliance (rsa-nw-appliance-nonfips rpm) Installs dependencies and scripts for TLS 1.3 Decryption

Install or update Non-FIPS buildsInstall or update Non-FIPS builds

The rsa-nw-appliance rpm now includes a script nw-nonfips-override-generator.sh to generate nonfips override descriptors. It will be installed at the path: /usr/local/sbin/nw-nonfips-override-generator.sh

The script automates the descriptor override generation steps.

(Option 1) Instructions to install nonfips rpms during an upgrade(Option 1) Instructions to install nonfips rpms during an upgrade

Step 1: Upgrade Node-Zero

  • Follow upgrade instructions for node-zero from UI or command line, this would install the latest repo under /var/netwitness/common/repo/

Step 2: Before upgrading other Node-X hosts with core services, run the script on Node-Zero.

  • /usr/local/sbin/nw-nonfips-override-generator.sh

    • This would invoke script in default mode which uses latest installed version ex: 12.0.0.0 and repo /var/netwitness/common/repo/12.0.0.0/RSA .

    • The script generates and validates nonfips descriptor override files.

Step 3: Refresh Node-Zero to apply applicable nonfips rpms (broker, appliance) using the command:

  • nw-manage --refresh-host --host-key <node-zero-ip-address>

  • Reboot Node-Zero

Step 4: Upgrade each Node-X

  • (Option 1) : Upgrade Node-X via Command line

    • Start upgrade host Node-X with the following command from Node-Zero.

      • Example: upgrade-cli-client –-upgrade --host-key <Host UUID or IP address> --version 12.0.0.0

    • Reboot the Node-X

  • (Option 2): Upgrade Node-X from UI:

    • Continue upgrading from UI and follow instructions to upgrade the remaining hosts.

  • If applicable, all the hosts would have the nonfips rpms (decoder, concentrator, broker, appliance) installed.

(Option 2) Instructions to install nonfips rpms after upgrade(Option 2) Instructions to install nonfips rpms after upgrade

After upgrading Node-Zero and all other Node-X hosts, and if you wish to install nonfips rpms at later then following are instructions:

  • Step 1: Run the script on Node-Zero

    • /usr/local/sbin/nw-nonfips-override-generator.sh

      • This would invoke script in default mode which uses latest installed version ex: 12.0.0.0 and repo /var/netwitness/common/repo/12.0.0.0/RSA .

      • The script generates and validates nonfips descriptor override files.

  • Step 2: Command line update:

    • Refresh all the hosts (including Node-Zero) to apply applicable nonfips rpms (decoder, concentrator, broker, appliance).

      • nw-manage --refresh-host --host-all

      • Reboot all the hosts.

Note: The /usr/local/sbin/nw-nonfips-override-generator.sh script would generate logs while running the commands, and also update /var/log/messages using the logger command.

TLS 1.3 Keys and Cipher suitesTLS 1.3 Keys and Cipher suites

According to RFC 8446 the TLS 1.3 protocol doesn’t support Private key based decryption and it only support a set of ephemeral symmetric keys also called as key secrets for the life of one specific TLS 1.3 session. The TLS 1.3 session also encrypts handshake, all the TLS records after Server Hello are encrypted , and at each stage a corresponding key is used for the encryption ex: client handshake, server handshake, client application and server application etc.. . So in order for decrypting a TLS 1.3 session all the required session keys tls.client.hdshk, tls.server.hdshk, tls.client.app, tls.server.app, tls.client.early(optional) are needed to be available.

Cipher SuitesCipher Suites

TLS 1.3 protocol recommends to support only strong and advanced Cipher suites. The Network Decoder supports all the advertised cipher suites from RFC-8446, however on integration with Thirdparty key forwarder the supported cipher suites might be limited to list supported by Thirdparty server. Example only GCM ciphers are supported etc..

The following is list of cipher suited supported in Network Decoder

  • TLS_AES_128_GCM_SHA256

  • TLS_AES_256_GCM_SHA384

  • TLS_CHACHA20_POLY1305_SHA256

  • TLS_AES_128_CCM_SHA256

  • TLS_AES_128_CCM_8_SHA256

Steps to configure Network Decoder to decrypt TLS 1.3 trafficSteps to configure Network Decoder to decrypt TLS 1.3 traffic

Step 1: Validate that the Network Decoder Captures Encrypted TLS 1.3 Traffic

First, make sure the Decoder is actually capturing the SSL/TLS traffic from the managed servers that you want to use to gain visibility into the encrypted sessions. You can do this by searching in the Investigate views (Navigate or Events) using the IP addresses of the servers as the destination IP addresses, and the service as SSL/TLS.

For example, if you are searching for HTTPS traffic to 192.168.1.2 and 192.168.1.3, you can run the following query:

service = 443 && ip.dst = 192.168.1.2, 192.168.1.3

Conduct this search on the Concentrator that aggregates metadata from the Decoder that is expected to capture this network segment, or on a Broker that may have access to all collected metadata.

(Version 11.6 or later) In case of simultaneous ingestion of the encrypted and decrypted traffic streams in the Decoder, ensure that multi-thread assembly is enabled. This is to prevent mixing of packets from different sessions by single thread assembly. For more information, see the following topics:

Step 2: Confirm HTTPS Parser is Enabled on Decoders

The decryption process requires the native HTTPS parser to be enabled to examine the TLS handshake and to compare the uploaded encryption keys to determine if they decrypt the traffic. This can be validated in Admin > Services > Decoder > Config > Parsers Configuration panel as shown in the following figure.

netwitness_tls1.3_config.png

If TLS_Lua parser is enabled then meta version: TLS 1.3 is registered for TLS 1.3 encrypted traffic.

Step 3: Integrate Network Decoder to receive keys from Thirdparty TLS 1.3 key forwarder

TLS 1.3 protocol only support ephemeral keys which are unique per session and Network Decoder need to receive those keys in time for parsing step.

Network Decoder provides sslKeys API which can receive the keys forwarded or injected to Decoder on RESTful interface.

A Thirdparty TLS 1.3 key forwarder which acts as Man-In-The-Middle (MITM) gateway for TLS 1.3 may intercept SSL handshake and forward the TLS 1.3 keys to Network Decoder.

Example: F5 BIG-IP Local Traffic Manager.

Option 1: Configure F5 BIG-IP Local Traffic Manager (LTM)Option 1: Configure F5 BIG-IP Local Traffic Manager (LTM)

If you are using BIG-IP LTM in your environment for application traffic management, load balancing etc.. and if you would like to decrypt and analyze TLS 1.3 traffic handled by BIG-IP LTM, then refer the following section to Configure F5 BIG-IP LTM and forward keys to Network Decoder.

Refer F5 BIG IP - NetWitness Perfect Forward Secrecy Inspection Visibility for more information.

Option 2: Using sslKeys APIOption 2: Using sslKeys API

If you are using other Man-In-The-Middle (MITM) gateway which can intercept TLS 1.3 handshake and forward the keys, then you can refer the following information and parameters to send keys to sslKeys API on Network Decoder.

The sslKeys API is used to push SSL crypto information to enable SSL decryption of a session's packets prior to parsing.

Parameters for Managing KeysParameters for Managing Keys

The sslKeys command has several parameters for managing premaster and private keys. This is the full list of parameters:

Parameter

Description

clear

Removes all premaster and TLS 1.3 keys from memory. Will not delete any PEM files installed on the system.

maxKeys

Changes the maximum number of premaster and TLS 1.3 keys that will be stored in memory.

counts

Returns key counts for sslKeys. Cannot be used with any other parameters.

listPems

Returns a list of all installed private key PEM files.

deletePem

Deletes the named PEM file from the file system. You can pass this parameter more than once to remove multiple files.

random

The random hash used to identify the premaster key or TLS 1.3 keys.

premaster

The premaster key that will be installed for the previous random parameter. They must show up in pairs and random must be first.

CETS

The TLS 1.3 client early traffic secret key that will be installed for the previous random parameter. It must show up with random parameter.

CHTS

The TLS 1.3 client handshake traffic secret key that will be installed for the previous random parameter. It must show up with random parameter.

SHTS

The TLS 1.3 server handshake traffic secret key that will be installed for the previous random parameter. It must show up with random parameter.

CTS0

The TLS 1.3 client traffic secret 0 key (application) that will be installed for the previous random parameter. It must show up with random parameter.

STS0

The TLS 1.3 server traffic secret 0 key (application) that will be installed for the previous random parameter. It must show up with random parameter.

Usage:Usage:

The Client random and TLS 1.3 keys can be passed as parameters to sslKeys, ex:

  • Using Native port 56004:

    • /decoder sslKeys random=<value> CETS=<value> CHTS=<value> SHTS=<value> CTS0=<value> STS0=<value>

  • Using RESTful port 50104:

    • /decoder?msg=sslKeys&random=<value>&CETS=<value>&CHTS=<value>&SHTS=<value>&CTS0=<value>&STS0=<value>

  • Replace <value> with base64 encoded TLS 1.3 key value in the above cases.

Note: For most of the sessions the CETS (CLIENT_EARLY_TRAFFIC_SECRET) key may not be generated or used by TLS clients and it is normal. However, if the other keys (handshake and application keys) are not generated and not available then TLS 1.3 decryption will NOT be successful.

Return ValuesReturn Values

Most commands return name/value pairs of statistics about the premaster and TLS 1.3 keys in memory. The statistics are listed in the following table.

Parameter

Description

added

The number of premaster and TLS 1.3 keys just added during this command.

total

The total number of premaster and TLS 1.3 keys loaded in memory.

agedOut

The total number of premaster and TLS 1.3 keys that were removed during this command (this is not a lifetime stat).

maxKeys

The current maximum allowed premaster and TLS 1.3 keys.

premaster

The total number of premaster keys loaded in memory.

tls13

The total number of TLS 1.3 keys loaded in memory.

 

Step 4: (Optional) Upload TLS 1.3 keys to validate Decryption

Note: Uploading TLS 1.3 keys is not supported for real-time capture, but can be used to validate that decryption is functioning properly on the Decoder.

The sslKeys command provides a way to upload TLS 1.3 keys to the Decoder, so that captured encrypted packets that match the keys can be decrypted before parsing. Administrators configure the Decoder by entering the sslKeys command using the NwConsole command line interface or the Decoder RESTful interface. Now, all TLS handshakes that use the TLS 1.3 keys and will be able to be decrypted by the Decoder.

For information about how to use the NwConsole command line interface, see the NwConsole User Guide for NetWitness Platform. For information about how to use the RESTful interface, see the RESTful API User Guide for NetWitness Platform. Go to the Master Table of Contents to find all NetWitness Platform 11.x documents.

You can use the RESTful interface form to facilitate uploading of multiple keys, both premaster and private at the same time.

  1. Open the RESTful API in your browser and navigate to this path on the Decoder that you want to configure:

    /decoder/sslkeys.

  2. Click Choose File Next to Upload File 1, and locate the TLS 1.3 keys file that you want to upload on your local file system.

  3. Click Upload. The files are uploaded to the Decoder and results are displayed in the form.

TLS 1.3 KeysTLS 1.3 Keys

Similar to premaster key, capturing TLS 1.3 keys and uploading to Decoder in time for parsing is not an easy way. However, well known browsers like Chrome , Firefox and IE can write the TLS 1.3 keys they generate to a file and this is useful for testing purposes. To configure your browser to do this, create an environment variable called SSLKEYLOGFILE and set with a pathname to text file to write the keys. Decoder will accept the file exactly as it is written and will use all the decryption keys in the file for any encrypted traffic it captures.

The following is a sample NwConsole script that uploads the file to a Decoder:

login <decoder>:50004 <username> <password>

send /decoder sslKeys --file-data=SSLKeys.txt

Optionally the parameters can be passed in as list in the upload file, ex:

send /decoder sslKeys --file-data=SSLKeys.txt --file-format=params-list

or you could use the following curl command (with the RESTful port):

curl -u "<username>:<password>" -H "Content-Type: application/octet-stream" --data-binary @"/path/SSLKeys.txt" -X POST "http://<hostname>:50104/decoder?msg=sslKeys"

Once the symmetric keys are uploaded, they will immediately be used for any necessary decryption. Symmetric keys are stored in memory and there is a limit to how many can be stored at any point in time. As more are added, the earliest keys will be aged out. You can also add available TLS 1.3 keys by just passing the random, CETS , CHTS , SHTS , CTS0 and STS0 parameters to sslKeys.

Note: For most of the sessions the CLIENT_EARLY_TRAFFIC_SECRET key may not be generated or used by TLS clients and it is normal. However, if handshake and application keys are not generated and not available then TLS 1.3 decryption wouldn't be successful.

Step 5: Validate TLS 1.3 keys are received on Decoder

It is advised to validate if Decoder is receiving keys to Decrypt TLS 1.3 sessions. As Decryption wouldn’t be successful if the keys are not available.

The following are few ways:

1. Decoder generates the following TLS 1.3 meta per session

tls.client.hdshk, tls.server.hdshk, tls.client.app, tls.server.app, and tls.client.early

2. The sslKeys API provides an option to display current key stats using counts=true .

Example: /decoder sslKeys counts=true

premasterKeys: 478

tls13Keys: 341

total: 819

maxKeys: 1000000

Step 6: View Decrypted Sessions

All storage of the packets is still encrypted, so when you search for any decrypted traffic, your query should still focus on service=443 or HTTPS (HTTP over SSL/TLS). If any packets were able to be decrypted, the tls.client.hdshk, tls.server.hdshk, tls.client.app, tls.server.app, and tls.client.early metadata is associated with the session that has the TLS 1.3 keys required for decrypting that session. The session will also have crypto metadata denoting the cipher suite.

Viewing Unencrypted TrafficViewing Unencrypted Traffic

If packets are decrypted during the parse stage, encrypted packets are written to disk, and the matching TLS 1.3 keys used for decrypting is written to the TLS 1.3 meta, analysts can view the unencrypted packets using the TLS 1.3 meta keys.

Option 1 : Using Event Analysis UIOption 1 : Using Event Analysis UI

Starting 12.0 the Event Analysis View support Displaying Decrypted and Encrypted Payloads of TLS sessions when TLS keys meta are registered.

Option 2 : Using RESTful API on Decoder serviceOption 2 : Using RESTful API on Decoder service

The Decoder API that you can use to see the unencrypted packets is the /sdk/content RESTful service. You need to know the Session ID of the encrypted packets and the flags parameter masked to the value 128 (or 0x80 in hex). Point your browser to the Decoder RESTful interface and type in the following command, substituting the actual Session ID for <id>:

http://<decoder>:50104/sdk/content?session=<id>&flags=128&render=text

The Decoder returns a simple web page showing the packets after they are decrypted.

If you want to see what the packets look like encrypted, type in one of the following commands, substituting the Session ID for <id>:

http://<decoder>:50104/sdk/content&session=<id>&render=text

http://<decoder>:50104/sdk/content&session=<id>&flags&render=text

For more information on the /sdk/content service, see the manual page for /sdk content.

If you are not able to view the encrypted sessions, check Troubleshooting. However, if you are able to see the TLS 1.3 metadata, that confirms that the Decoder is able to see inside the encrypted session, which enables parsers to see the unencrypted packet payload and create metadata accordingly. For these decrypted sessions, look for additional metadata around usernames, passwords, or HTTP body content that would be obfuscated when encrypted, which would result in metadata not being generated.

TroubleshootingTroubleshooting

If you are still not able to decrypt or view the decrypted payloads after attempting the steps above, try these steps.

  1. Validate if Decoder is receiving TLS 1.3 keys. Refer #Step 5 listed above.

  2. Validate that your TLS 1.3 session key is using an acceptable cipher-suite.

  3. Make sure your Network Decoder is capturing the traffic matching the server for which you have the TLS 1.3 keys. To check, you can upload the TLS 1.3 keys and PCAP into Wireshark and see if it is able to decrypt it.

Additional FeaturesAdditional Features

The following features which are supported in TLS 1.2 are also supported in TLS 1.3 decryption.

  1. TLS Certificate Hashing

  2. JA3 and JA3S TLS Fingerprints

  3. Performance Considerations

For more details refer Decrypt Incoming Packets (TLS 1.2)