Upgrade Tasks for 12.3

[#1]

Upgrade the systems in your environment in the following order:

1. NW Server hosts
2. Analyst UI hosts
3. ESA Primary hosts
4. ESA Secondary hosts

5. Standalone Broker hosts

6. Concentrator hosts

7. Archiver hosts

8. Packet Decoder hosts

9. Log Decoder hosts

10. Log Collector / VLC hosts

11. The rest of your component hosts

Note: NW Server, Analyst UI, and ESA Primary and Secondary hosts must all be upgraded on the same day. The rest of your component hosts can be upgraded on the same day or later.

For information about all the host types in NetWitness, see the Host and Services Getting Started Guide for NetWitness Platform. Go to the NetWitness All Versions Documents page and find NetWitness Platform guides to troubleshoot issues.

Note: Make sure that you plan the upgrade process so that Correlation servers are upgraded immediately after the Admin Server is done. For more information, see "Task 3. Preparing ESA Deployments for Migration to the 12.3 Version" in topic Upgrade Preparation Tasks.

Important Notes

This section lists few important notes you must read before proceeding with the upgrade tasks.

Synchronize Time on Component Hosts with NW Server Host

Before upgrading your hosts, make sure that the time on each host is synchronized with the time on the NetWitness Server.
To synchronize the time, do one of the following:

• Configure the NTP Server. For more information, see"Configure NTP Servers" in the System Configuration Guide.
• Perform the following steps on each host:
  
  1. SSH to the Admin Server host.

  2. Run the following commands.

     salt \* service.stop ntpd

     salt \* cmd.run 'ntpdate nw-node-zero'

     salt \* service.start ntpd

Mixed Mode Unsupported for ESA Hosts

Mixed mode is not supported for ESA hosts in NetWitness Platform version. The NetWitness server, ESA primary host, and ESA secondary host must all be on the same NetWitness Platform version.

Respond Server Service Not Enabled Until NW Server and Primary ESA Host Upgraded to 12.3.0.0

After upgrading the primary NW Server (including the Respond Server service), the Respond Server service is not automatically re-enabled until after the Primary ESA host is also upgraded to 12.3.0.0 The Respond post-upgrade tasks only apply after the Respond Server service is upgraded and is in the enabled state.

Deploy_Admin Password Guidelines

In NetWitness Platform version 11.6 or Later, deployment account password (only on node-zero) must contain at least one number, one upper and lower case letter, and one special characters (!@#%^,+ . ) along with the existing policy. The same password policy applies while updating deploy_admin password using nw-manage script.

If deploy_admin password is changed on Primary NW Server, it must be changed in the Warm Standby Server if it exists

Removal of DNSMasq in 12.1 and 12.2 Versions

After upgrading to 12.1 and 12.2 versions, the package, service, and the configuration files associated with DNSMasq are removed. However, this will not impact the system.

Additional Post Upgrade Steps for 12.3.0.0 Version with Legacy Windows Log Collector

For 12.3.0.0 version with Legacy Windows Log Collector, you should perform few additional post upgrade tasks. Refer to Legacy Windows Log Collection section in Post Upgrade Tasks for these additional post upgrade tasks.

Upgrade Options

You can choose one of the following upgrade methods based on your Internet connectivity. They are listed in the order recommended by NetWitness.

• Option 1: Upgrade NetWitness Platform XDR
• Option 2: Upgrade NetWitness Platform XDR Offline
• Option 3: Upgrade NetWitness Platform XDR using CLI (Offline)
• Option 4 (Optional): Pre-Stage Upgrade Repository by Downloading Packages

The following rules apply when you are upgrading hosts for all of these upgrade methods:

• You must upgrade the NW Server host first.
• You can only apply a version that is compatible with the existing host version.
• The NW Server, ESA primary, ESA secondary, and Analyst UI hosts must all be on the same NetWitness Platform version.

• Add Warm Standby (if exists) to the list of hosts and it must be on same version as NetWitness Platform.

Option 1: Upgrade NetWitness Platform XDR

You can use this method if the NW Server host is connected to Live Services and if you are able to obtain the package.

Prerequisites

Make sure that:

1. The Automatically download information about new upgrades every day option is selected and is applied in  (Admin) > System > Updates.
2. Updates are available. Go to  (Admin) > Hosts > Update > Check for Updates to check for updates. The Host view displays the Update Available status.
3. 12.3.0.0 is available in the Update Version column.

Procedure

To upgrade from 11.7.0.x, 11.7.1.0, 11.7.1.1, 11.7.1.2, 11.7.2.0, 11.7.3.0, 12.0.0.0, 12.1.0.0, 12.1.0.1, 12.1.1.0, 12.2.0.0, and 12.2.0.1 to 12.3.0.0, follow the steps below:

1. Go to  (Admin) > Hosts.
2. Select the NW Server (nw-server) host.

3. Check for the latest updates.

   

4. Update Available is displayed in the Status column if you have a version update in your Local Update Repository for the selected host.

5. Select 12.3.0.0 from the Update Version column. If you:

   • Want to view a dialog with the major features in the upgrade and information on the updates, click the information icon ( ) to the right of the upgrade version number.

   • Cannot find the version you want, select Update > Check for Updates to check the repository for any available updates. If an update is available, the message "New updates are available" is displayed and the Status column updates automatically to show Update Available. By default, only supported updates for the selected host are displayed.

6. Click Update > Update Host from the toolbar.
7. Click Begin Update.
8. Click Reboot Host.
9. Repeat steps 6 to 8 for other hosts.

Note: You can select multiple hosts to upgrade at the same time only after updating and rebooting the NW Server host. All ESA, Endpoint, and Malware Analysis hosts should be upgraded to the same version as that of the NW Server host.

Note: In 11.7.1.0 or later versions, you can pre-stage the upgrade repository using the Pre Stage Host feature. Refer the following screenshot. For more information, see Option 4 (Optional): Pre-Stage Upgrade Repository by Downloading Packages.

Option 2: Upgrade NetWitness Platform XDR Offline

Task 1. Populate Staging Folder (/var/netwitness/common/update-stage/) with Version Upgrade Files

1. Download the upgrade package netwitness-12.3.0.0.zip from NetWitness Community (https://community.netwitness.com/) > Downloads > NetWitness Platform > Version 12.3 to a local directory:
2. Upload netwitness-12.3.0.0.zip to /var/netwitness/common/update-stage/ on the NW Server Host.
   For example:

mv /var/netwitness/tmp/netwitness-12.3.0.0.zip /var/netwitness/common/update-stage/

Note: NetWitness Platform unzips the file automatically.

Task 2. Apply Upgrades from the Staging Area to Each Host

Caution: You must upgrade the NW Server host before upgrading any non-NW Server host.

1. Log in to NetWitness.

2. Go to (Admin) > Hosts.

   Note: If you are already on the (Admin) > Hosts page and the Check for Updates option (Update > Check for Updates) is grayed out, refresh the page from the browser to check for the updates.

3. Check for updates and wait for the upgrade packages to be copied, validated, and ready to be initialized.

   "Ready to initialize packages" is displayed if:

   • NetWitness Platform can access the upgrade package.
   • The package is complete and has no errors.

   Refer to Troubleshooting Version Installations and Updates for instructions on how to troubleshoot errors (for example, "Error deploying version <version-number>" and "Missing the following update package(s)," are displayed in the Initiate Update Package for RSA NetWitness Platform dialog.)

4. Click Initialize Update.

   It takes some time to initialize the packages because the files are large and need to be unzipped. The time varies depending on how the host is configured.
   After the initialization is successful, the Status column displays Update Available and you complete the rest of the steps in this procedure to finish the upgrade of the host.

5. Click Update > Update Hosts from the toolbar.

   

6. Click Begin Update from the Update Available dialog.
   After the host is upgraded, it prompts you to reboot the host.
7. Click Reboot Host from the toolbar.

Option 3: Upgrade NetWitness Platform XDR using CLI (Offline)

You can use this method if the NW Server host is not connected to Live Services.

Prerequisites

Make sure that you have downloaded the netwitness-12.3.0.0.zip file from NetWitness Community (https://community.netwitness.com/) > Products > NetWitness Platform > Downloads > Version 12.3 > Full Product Downloads to a local directory:

Procedure

You must perform the upgrade steps for NW Server hosts and for component servers.

Note: If you copy and paste the commands from PDF to Linux SSH terminal, the characters do not work. However, you can copy the commands from the HTML page https://community.netwitness.com/t5/netwitness-platform-online/upgrade-tasks-for-12-1-1/ta-p/695018#Option3 and paste them to Linux SSH terminal.

1. Stage the 12.3.0.0 files to prepare them for the upgrade.

   • Log into the NW Server as root and create the following directory:
     /var/netwitness/tmp/upgrade/12.3.0.0
     and then copy the package zip files to the /var/netwitness/tmp/ directory of the NW Server and extract the package files from /var/netwitness/tmp/ to the appropriate directories using the following command:
     unzip netwitness-12.3.0.0.zip -d /var/netwitness/tmp/upgrade/12.3.0.0/

     Make sure you remove the update zip file from the staging directory after it is extracted.

2. Initialize the upgrade, using the following command:
   upgrade-cli-client --init --version 12.3.0.0 --stage-dir /var/netwitness/tmp/upgrade

3. Upgrade the NW Server host, using the following command:
   upgrade-cli-client --upgrade --version 12.3.0.0 --host-key <ID / display name / (hostname/ IP address)>
4. When the NW Server host upgrade is successful, reboot the host from NetWitness Platform user interface in the Hosts view.
5. Repeat steps 3 and 4 for other component hosts.

Note: You can check versions of all the hosts, using the command upgrade-cli-client --list on the NW Server host. If you want to view the help content of upgrade-cli-client, use the command upgrade-cli-client --help.

Note: If the following error is displayed during the upgrade process:
2017-11-02 20:13:26.580 ERROR 7994 — [ 127.0.0.1:5671] o.s.a.r.c.CachingConnectionFactory : Channel shutdown: connection error; protocol method: #method<connection.close>(reply-code=320, reply-text=CONNECTION_FORCED - broker forced connection closure with reason 'shutdown', class-id=0, method-id=0)
the service pack will install correctly. No action is required. If you encounter additional errors when updating a host to a new version, contact Customer Support for assistance.

External Repo Instructions for CLI upgrade

1. Stage the 12.3.0.0 files to prepare them for the upgrade.

   • Log into the NW Server as root and create the following directory:
     /var/netwitness/tmp/upgrade/12.3.0.0
     and then copy the package zip files to the /var/netwitness/tmp/ directory of the NW Server and extract the package files from /var/netwitness/tmp/ to the appropriate directories using the following command:
     unzip netwitness-12.3.0.0.zip -d /var/netwitness/tmp/upgrade/12.3.0.0/

     Make sure you remove the update zip file from the staging directory after it is extracted.

2. Initialize the upgrade, using the following command:
   upgrade-cli-client --init --version 12.3.0.0 --stage-dir /var/netwitness/tmp/upgrade

3. Upgrade the NW Server host, using the following command:
   upgrade-cli-client --upgrade --version 12.3.0.0 --host-key <ID / display name / (hostname/ IP address)>
4. When the NW Server host upgrade is successful, reboot the host from NetWitness Platform user interface in the Hosts view.
5. Repeat steps 3 and 4 for other component hosts.

Note: You can check versions of all the hosts, using the command upgrade-cli-client --list on the NW Server host. If you want to view the help content of upgrade-cli-client, use the command upgrade-cli-client --help.

Note: If the following error displays during the upgrade process:
2017-11-02 20:13:26.580 ERROR 7994 — [ 127.0.0.1:5671] o.s.a.r.c.CachingConnectionFactory : Channel shutdown: connection error; protocol method: #method<connection.close>(reply-code=320, reply-text=CONNECTION_FORCED - broker forced connection closure with reason 'shutdown', class-id=0, method-id=0)
the service pack will install correctly. No action is required. If you encounter additional errors when updating a host to a new version, contact Customer Support for assistance.

Option 4 (Optional): Pre-Stage Upgrade Repository by Downloading Packages

You can pre-stage the upgrade repository by downloading the required packages (.zip) without affecting the system. This minimizes the upgrade downtime and ensures the upgrade is completed within the planned time.

Note: Pre Stage Host feature is supported from version 11.7.1.0 or later.

Procedure

1. Go to  (Admin) > Hosts.

2. Click Update > Check for Updates from the toolbar.

   All possible update versions will be displayed in the Versions drop-down list.

3. Click Update > Pre Stage Host and select the version in the update version column.

   A confirmation message for downloading the files is displayed.

   

   

4. Click Yes to download the upgrade packages to the repo.

5. Verify the status of the download in the notifications tray as shown below.

   The Pre Stage Host and Upgrade Host will be disabled until pre stage is completed.

   

   Note: The current version and the update version in the UI will be the same during the pre stage as it is not the actual update. This is because only the repo files are downloaded and no actual upgrade is done. The version will change only after upgrade.

6. If the download is successful, Check for Updates again to start the initialization.

7. Click Initialize Update.

   The initialization of the package will take some time as the files are large and will need to be unzipped.

   

   IMPORTANT: Pre Stage Repo preparation steps from 1 to 4 can be performed at any time. However, from steps 5 to 8 the upgrade process begins and you must NOT reboot the host or restart the jetty server during this time as it will corrupt the .ZIP files.

8. Check the status of initialization in the notifications tray.

9. After the initialization is completed successfully, click Update > Update Host.

   After the host is updated, you will be prompted to reboot the host.

10. Set up the host and reboot the host.