Perform Upgrade Tasks

IMPORTANT: NetWitness strongly recommends that you run the pre-upgrade checks before you upgrade to NetWitness Platform 12.5. For more information on how to run the pre-upgrade checks, see Run Pre-Upgrade Checks

Upgrade the systems in your environment in the following order:

  1. NW Server hosts
  2. Analyst UI hosts
  3. ESA Primary hosts
  4. ESA Secondary hosts

  5. Standalone Broker hosts

  6. Concentrator hosts

  7. Archiver hosts

  8. Packet Decoder hosts

  9. Log Decoder hosts

  10. Log Collector / VLC hosts

  11. The rest of your component hosts

IMPORTANT: NW Server, Analyst UI, and ESA Primary and Secondary hosts must all be upgraded on the same day. The rest of your component hosts can be upgraded on the same day or later. Make sure that you plan the upgrade process so that Correlation servers are upgraded immediately after the Admin Server is done. For more information, see "Task 3. Prepare ESA Deployments for Migration to 12.5" in the topic Prepare to Upgrade NetWitness Platform. Mixed mode is not supported for ESA hosts in NetWitness Platform. The NetWitness server, ESA primary host, and ESA secondary host must all be on the same NetWitness Platform version.

For information about all the host types in NetWitness, see the NetWitness Hosts and Services Getting Started Guide. Go to the NetWitness All Versions Documents page and find NetWitness Platform guides to troubleshoot issues.

IMPORTANT: After upgrading the primary NW Server (including the Respond Server service), the Respond Server service is not automatically re-enabled until after the Primary ESA host is also upgraded to same version. The Respond post-upgrade tasks only apply after the Respond Server service is upgraded and is in the enabled state.

Note: For 12.5.0.0 version with Legacy Windows Log Collector, you should perform few additional post upgrade tasks. Refer to Legacy Windows Log Collection section in Perform Post Upgrade Tasks for these additional post upgrade tasks.

Select Upgrade Options

You can select one of the following upgrade options based on your Internet connectivity. They are listed in the order recommended by NetWitness Platform.

The following rules apply when you are upgrading hosts using any of the 4 upgrade methods:

  • You must upgrade the NW Server host first.
  • You can only apply a version that is compatible with the existing host version.
  • The NW Server, ESA primary, ESA secondary, and Analyst UI hosts must all be on the same NetWitness Platform version.

Option 1: Upgrade NetWitness Platform using Live Services

You can use this method if the NW Server host is connected to Live Services.

Caution: You must review your network policy before downloading the upgrade package which is around 11.7 GB. If you have set up any policy that disallows file download beyond 10 GB, the upgrade package download fails.

Note: You can pre-stage the upgrade repository using the Pre Stage Host feature. Refer the following figure. For more information, see Option 4 (Optional): Pre-Stage Upgrade Repository by Downloading Packages.

pre-stage_upgrade_guide_12.0.PNG

Prerequisites

  1. The Automatically download information about new upgrades every day option is selected and is applied in AdminIcon.png (Admin) > System > Updates.
  2. Updates are available. Go to AdminIcon.png (Admin) > Hosts > Update > Check for Updates to check for updates. The Host view displays the Update Available status.
  3. 12.5.0.0 is available in the Update Version column.

To upgrade from 12.2.0.0, 12.2.0.1, 12.3.0.0, 12.3.1.0, 12.4.0.0, 12.4.1.0, and 12.4.2.0 to 12.5.0.0:

  1. Go to AdminIcon.png (Admin) > Hosts.
  2. Select the NW Server (nw-server) host.

  3. Check for the latest updates.
    Chk4Upds.PNG

    Update Available is displayed in the Status column if you have a version update in your Local Update Repository for the selected host.

  4. Select 12.5.0.0 from the Update Version column.

    Note:
    - If you want to view a dialog with the major features in the upgrade and information on the updates, click the information icon (ic-inline_help.png) to the right of the upgrade version number.

    - If you cannot find the version you want, select Update > Check for Updates to check the repository for any available updates. If an update is available, the message "New updates are available" is displayed and the Status column updates automatically to show Update Available. By default, only supported updates for the selected host are displayed.

  5. Click Update > Update Host from the toolbar.
  6. Click Begin Update.
  7. Click Reboot Host.
  8. Repeat steps 5 to 7 for other hosts.

Note: You can select multiple hosts to upgrade at the same time only after updating and rebooting the NW Server host. All ESA, Endpoint, and Malware Analysis hosts should be upgraded to the same version as that of the NW Server host.

Option 2: Upgrade NetWitness Platform Offline

You can manually upgrade NetWitness Platform by performing the following tasks.

Task 1. Populate Staging Folder (/var/netwitness/common/update-stage/) with Version Upgrade Files. Do the following.

  1. Download the upgrade package netwitness-12.5.0.0.zip from NetWitness Community (https://community.netwitness.com/) > Downloads > NetWitness Platform > Version 12.5.0.0 to a local directory:

    • If you are upgrading from 12.2.0.0, 12.2.0.1, 12.3.0.0, 12.3.1.0, 12.4.0.0, 12.4.1.0, and 12.4.2.0 download netwitness-12.5.0.0.zip.

  2. SSH to the NW Server host.

  3. Upload netwitness-12.5.0.0.zip to /var/netwitness/common/update-stage/ on the NW Server Host.
    For example:

    mv /var/netwitness/tmp/netwitness-12.5.0.0.zip /var/netwitness/common/update-stage/

Note: NetWitness Platform unzips the file automatically.

Task 2. Apply Upgrades from the Staging Area to Each Host. Do the following.

Caution: You must upgrade the NW Server host before upgrading any non-NW Server host.

  1. Log in to NetWitness.

  2. Go to AdminIcon.png (Admin) > Hosts.

    Note: If you are already on the AdminIcon.png (Admin) > Hosts page and the Check for Updates option (Update > Check for Updates) is grayed out, refresh the page from the browser to check for the updates.

  3. Check for updates and wait for the upgrade packages to be copied, validated, and ready to be initialized.

    "Ready to initialize packages" is displayed if:

    • NetWitness Platform can access the upgrade package.
    • The package is complete and has no errors.

    Refer to Troubleshooting Version Installations and Updates for instructions on how to troubleshoot errors (for example, "Error deploying version <version-number>" and "Missing the following update package(s)," are displayed in the Initiate Update Package for NetWitness Platform dialog.)

  4. Click Initialize Update.

    It takes some time to initialize the packages because the files are large and need to be unzipped. The time varies depending on how the host is configured.
    After the initialization is successful, the Status column displays Update Available.

  5. Click Update > Update Hosts from the toolbar.

    update_host_12.0_upgrade_guide.PNG

  6. Click Begin Update from the Update Available dialog.
    After the host is upgraded, it prompts you to reboot the host.
  7. Click Reboot Host from the toolbar.

Option 3: Upgrade NetWitness Platform using CLI (Offline)

You can use this option if the NW Server host is not connected to Live Services.

Before you begin 

Make sure that you have downloaded the following file from NetWitness Community (https://community.netwitness.com/) > Products > NetWitness Platform > Downloads to a local directory:

  • If you are upgrading from 12.2.0.0, 12.2.0.1, 12.3.0.0, 12.3.1.0, 12.4.0.0, 12.4.1.0, and 12.4.2.0, download:

    netwitness-12.5.0.0.zip

  • If you are using external repository, you can update the external repository with the latest upgrade content. For more information, see External Repo Instructions for CLI upgrade.

To upgrade NW Server Hosts and component servers:

Note: If you copy and paste the commands from PDF to Linux SSH terminal, the characters do not work. However, you can copy the commands from the HTML page https://community.netwitness.com/t5/netwitness-platform-online/upgrade-tasks-for-12-5/ta-p/717024#Option3 and paste them to Linux SSH terminal.

  1. Stage the 12.5.0.0 files to prepare them for the upgrade. Consider the following scenarios. 

    • Option 1 (Manual) : Log into the NetWitness Server and create the following directory:

      /var/netwitness/tmp/upgrade/12.5.0.0/
      Then copy the package zip file to the /var/netwitness/tmp/ directory of the NW Server and extract the package files from /var/netwitness/tmp/ to the appropriate directory using the following command:

      unzip netwitness-12.5.0.0.zip -d /var/netwitness/tmp/upgrade/12.5.0.0
      Make sure you remove the update zip file from the staging directory after it is extracted.

    • Option 2 (Automated) : Log into the NetWitness Server and create the following directory:
      /var/netwitness/tmp/upgrade/
      Then copy the NetWitness 12.5.0.0 package zip files to the /var/netwitness/tmp/ directory on the NetWitness Server.
      After this, run the below command to extract, validate, and initialize the 12.5.0.0 zip files:
      [root@SA ~]# upgrade-cli-client --init --stage-dir /var/netwitness
      /tmp/upgrade --download-path /var/netwitness/tmp/ --version 12.5.0.0
      Once the message (INFO) Download and extraction of all the necessary NetWitness zips are completed is displayed in the console of the admin server, only then the initialization process will begin.

    Note: If you do not receive the message (INFO) Download and extraction of all the necessary NetWitness zips are completed, run the previous command again.

    IMPORTANT: After staging 12.5.0.0 (using the Option 2), if the initialization fails, run the command upgrade-cli-client --init --version 12.5.0.0 --stage-dir /var/netwitness/tmp/upgrade. If the initialization succeeds, ignore the step 2 Initialize the upgrade below and proceed with the further steps 3-6.

  2. Initialize the upgrade using the following command:
    upgrade-cli-client --init --version 12.5.0.0 --stage-dir /var/netwitness/tmp/upgrade

  3. Upgrade the NW Server host, using the following command:
    upgrade-cli-client --upgrade --version 12.5.0.0 --host-key <ID / display name / (hostname/ IP address)>

    Note: Once the upgrade is triggered, NW Server will reboot automatically ~10 mins into the upgrade process. It will boot into the new kernel (4.18 for Alma Linux 8.10).

    Caution: Users are advised to wait until the UI is up and running, which may take up to an hour to complete. After 20 to 30 minutes of the migration, you can SSH and check if the OS is migrated. Once the OS migration is complete, it may take at least 30 minutes for the UI to appear as the NW Upgrade runs in the background.

    The above upgrade process can be tracked through a virtual console for VMs or remote console for servers with iDRACs.

    For more information on how to connect to the virtual console, see https://www.dell.com/support/kbdoc/en-in/000179797/dell-poweredge-idrac-virtual-console.

    Once the OS is migrated and able to SSH to Admin Node, run the following command on the host to confirm successful OS migration:

    • cat /etc/redhat-release

    • AlmaLinux release 8.10 (Cerulean Leopard)

    Caution: After the OS migration, reinstall any third-party RPMs you have previously installed.

  4. Once the orchestration-server is up, it will automatically trigger the NW Upgrade through chef to the desired NW Version. To check the progress of this, please SSH to the Admin Server and run the following command:

    • orchestration-cli-client --check-admin-upgrade-status

    Note: Run the above command only for NW Admin Server.

  5. When the NW Server host upgrade is successful, reboot the host from NetWitness Platform user interface in the Hosts view.
  6. (Conditional) If Warm Standby Server is deployed, repeat steps 1 to 5 on the Warm Standby Server host.
  7. Repeat steps 3 and 5 for each component host, changing the IP address to the component host which is being upgraded.

Note: You can check versions of all the hosts, using the command upgrade-cli-client --list on the NW Server host. If you want to view the help content of upgrade-cli-client, use the command upgrade-cli-client --help.

External Repo Instructions for CLI upgrade

For information about setting up an external repository, see Appendix A. Set Up External Repo in the 12.5.0.0 Upgrade Guide for NetWitness Platform. The following instructions assume that you already have an external repository set up. Go to the NetWitness All Versions Documents page and find NetWitness Platform guides to troubleshoot issues.

  1. Stage the 12.5.0.0 files to prepare them for the upgrade. Consider the following scenarios.
    • If you are upgrading from 12.2.0.0, 12.2.0.1, 12.3.0.0, 12.3.1.0, 12.4.0.0, 12.4.1.0 and 12.4.2.0, you only need to stage 12.5.0.0.

      • Option 1 (Manual) : Log into the NetWitness Server and create the following directory:

        /var/netwitness/tmp/upgrade/12.5.0.0/
        Then copy the package zip file to the /var/netwitness/tmp/ directory of the NW Server and extract the package files from /var/netwitness/tmp/ to the appropriate directory using the following command:

        unzip netwitness-12.5.0.0.zip -d /var/netwitness/tmp/upgrade/12.5.0.0
        Make sure you remove the update zip file from the staging directory after it is extracted.

      • Option 2 (Automated) : Log into the NetWitness Server and create the following directory:
        /var/netwitness/tmp/upgrade/
        Then copy the NetWitness 12.5.0.0 package zip files to the /var/netwitness/tmp/ directory on the NetWitness Server.
        After this, run the below command to extract, validate, and initialize the 12.5.0.0 zip files:
        [root@SA ~]# upgrade-cli-client --init --stage-dir /var/netwitness
        /tmp/upgrade --download-path /var/netwitness/tmp/ --version 12.5.0.0
        Once the message (INFO) Download and extraction of all the necessary NetWitness zips are completed is displayed in the console of the admin server, only then the initialization process will begin.

      Note: If you do not receive the message (INFO) Download and extraction of all the necessary NetWitness zips are completed, run the previous command again.

      IMPORTANT: After staging 12.5.0.0 (using the Option 2), if the initialization fails, run the command upgrade-cli-client --init --version 12.5.0.0 --stage-dir /var/netwitness/tmp/upgrade. If the initialization succeeds, ignore the step 2 Initialize the upgrade below and proceed with the further steps 3-6.

    1. Initialize the upgrade using the following command:
      upgrade-cli-client --init --version 12.5.0.0 --stage-dir /var/netwitness/tmp/upgrade

    1. Upgrade the NW Server host, using the following command:
      upgrade-cli-client --upgrade --version 12.5.0.0 --host-key <ID / display name / (hostname/ IP address)>

      Note: Once the upgrade is triggered, NW Server will reboot automatically ~10 mins into the upgrade process. It will boot into the new kernel (4.18 for Alma Linux 8.10).

      Caution: Users are advised to wait until the UI is up and running, which may take up to an hour to complete. After 20 to 30 minutes of the migration, you can SSH and check if the OS is migrated. Once the OS migration is complete, it may take at least 30 minutes for the UI to appear as the NW Upgrade runs in the background.

      The above upgrade process can be tracked through a virtual console for VMs or remote console for servers with iDRACs.

      For more information on how to connect to the virtual console, see https://www.dell.com/support/kbdoc/en-in/000179797/dell-poweredge-idrac-virtual-console.

      Once the OS is migrated and able to SSH to Admin Node, run the following command on the host to confirm successful OS migration:

      • cat /etc/redhat-release

      • AlmaLinux release 8.10 (Cerulean Leopard)

      Caution: After the OS migration, reinstall any third-party RPMs you have previously installed.

4. Once the orchestration-server is up, it will automatically trigger the NW Upgrade through chef to the desired NW Version. To check the progress of this, please SSH to the Admin Server and run the following command:

      1. orchestration-cli-client --check-admin-upgrade-status

Note: Run the above command only for NW Admin Server.

5. When the NW Server host upgrade is successful, reboot the host from NetWitness Platform user interface in the Hosts view.

6. (Conditional) If Warm Standby Server is deployed, repeat steps 1 to 5 on the Warm Standby Server host.

7. Repeat steps 3 and 5 for each component host, changing the IP address to the component host which is being upgraded.

Note: You can check versions of all the hosts, using the command upgrade-cli-client --list on the NW Server host. If you want to view the help content of upgrade-cli-client, use the command upgrade-cli-client --help.

Option 4 (Optional): Pre-Stage Upgrade Repository by Downloading Packages

You can pre-stage the upgrade repository by downloading the required packages (.zip) without affecting the system. This minimizes the upgrade downtime and ensures the upgrade is completed within the planned time.

To pre-stage the upgrade repository and update the hosts:

    1. Go to AdminIcon.png (Admin) > Hosts.

    2. Click Update > Check for Updates from the toolbar.

      All possible update versions will be displayed in the Versions drop-down list.

    3. Click Update > Pre Stage Host and select the version in the update version column.

      125_Upgrade_Available.png

      A confirmation message for downloading the files is displayed.

      125_Upgrade_Download_Files.png

    4. Click Yes to download the upgrade packages to the repo.

    5. Verify the status of the download in the notifications tray as shown below.

      The Pre Stage Host and Upgrade Host will be disabled until pre stage is completed.

      125_Upgrade_Notification.png

      Note: The current version and the update version in the UI will be the same during the pre stage as it is not the actual update. This is because only the repo files are downloaded and no actual upgrade is done. The version will change only after upgrade.

    6. If the download is successful, Check for Updates again to start the initialization.

    7. Click Initialize Update.

      The initialization of the package will take some time as the files are large and will need to be unzipped.

      125_Upgrade_Ready.png

      IMPORTANT: Pre Stage Repo preparation steps from 1 to 4 can be performed at any time. However, from steps 5 to 8 the upgrade process begins and you must NOT reboot the host or restart the jetty server during this time as it will corrupt the .ZIP files.

    8. Check the status of initialization in the notifications tray.

    9. After the initialization is completed successfully, click Update > Update Host.

      After the host is updated, you will be prompted to reboot the host.

    10. Set up the host and reboot the host.