Reattaching DAC storage to an RSA Security Analytics Archiver appliance after upgrading to 10.4

[#1]

Article Number

000029949

Applies To

RSA Product Set: Security Analytics
RSA Product Set: Archiver
RSA Version/Condition: 10.3.x, 10.4.x
Platform: CentOS

Issue

If upgrading a Security Analytics Archiver appliance from 10.3 to 10.4 during a complete reinstall ("build stick"), the locations where the Archiver finds its storage have been changed, due to the new Hot, Warm, and Cold storage options that were added in the 10.4 release series.

To re-mount and use any attached DAC hardware in this scenario, please follow these steps.

Resolution

Once the appliance is booted into the upgraded Security Analytics release, begin by opening an ssh console session to the appliance, then stop the Archiver service:

# stop nwarchiver

Ensure that all mount points for the DACs exist. These are located in /var/netwitness/archiver, and start with the database0 directory. If they do not exist for any reason, create them, starting from database0. Create one directory for each DAC to be reconnected. In the example below, three DACs need to be reconnected:

# cd /var/netwitness/archiver/
# ls
cache  database0  manifest  rest  statdb
# mkdir database1 database2

In case a reboot is required at any point, comment out all DAC mount points in /etc/fstab, using a text editor. Prepend each line with a hash (#) symbol:

EXAMPLE:
# /dev/mapper/archiver-database1 /var/netwitness/database1 xfs defaults,noatime,nosuid 1 2

Check to see if any DACs are already mounted. If so, unmount them:

# df -h | grep -B 1 archiver
/dev/mapper/archiver-database1
                       28T   26T  1.4T  95% /var/netwitness/archiver/database1
/dev/mapper/archiver-database2
                       28T   26T  1.4T  95% /var/netwitness/archiver/database2
# umount /var/netwitness/archiver/database1
# umount /var/netwitness/archiver/database2

Start the Archiver service again:

# start nwarchiver

At this point, open the Security Analytics UI, select the Archiver service, and open the Config page. Select the Archiver Storage tab. Follow the instructions in the Security Analytics Administration Guide for adding additional storage locations. Once this is done, stop the Archiver service again:

# stop nwarchiver

Starting the Archiver service will have created some folders in the first Archiver storage location as well as any additional databaseX directories you added in the previous step in /var/netwitness/archiver/ . Remove them:

EXAMPLE
# cd /var/netwitness/archiver/database0
# rm -fr *

If the DAC mount points already existed under /var/netwitness/archiver, un-comment the relevant lines in /etc/fstab by removing the leading hash symbols as shown near the top of the instructions.

If the DAC mount points did not already exist, in addition to creating them, you may need to create new entries for each databaseX directory created in /etc/fstab:

EXAMPLE
/dev/mapper/archiver-database1 /var/netwitness/database1 xfs defaults,noatime,nosuid 1 2

Note that this example does not have leading hash (#) symbols.

Once the entries for the DAC mount points exist and are not commented out check your work by typing the following command:

# mount -a

You can also ensure the DACs are mounted by typing in the following command and ensuring the archiver-databases are printed to the screen:

# df -h | grep -B 1 archiver
/dev/mapper/archiver-database1
                       28T   26T  1.4T  95% /var/netwitness/archiver/database1
/dev/mapper/archiver-database2
                       28T   26T  1.4T  95% /var/netwitness/archiver/database2

Start the Archiver service one more time:

# start nwarchiver

With the data on the DACs added back, Archiver startup may take some time to complete. You may wish to monitor its progress with the tail command:

# tail -f /var/log/messages