Article Number
000039822
Applies To
RSA Product Set: RSA NetWitness Platform
RSA Product/Service Type: Security Analytics Server
RSA Version/Condition: 11.6.0.1
Platform: CentOS
O/S Version: 7
Issue
Running
upgrade-cli-client command failed within few seconds without any specific error while upgrading a host from 11.6.0.0 to 11.6.0.1.
No event is logged in
/var/log/netwitness/config-management/chef-solo.log on the target host.
/var/log/messages showed the salt-minion service failing to connect to the Salt Master.
Aug 13 01:17:02 NWVLC salt-minon: [ERROR ] Minion unable to successfully connect to a Salt Master.
curl -v nw-node-zero:4505 and
curl -v nw-node-zero:4506 confirmed no connectivity issue to node zero.
Re-provisioning the host by following KB
36443 failed as the enabling process for the discovered host appeared to be stuck./var/log/salt/master.log showed messages like below.
2021-08-13 02:18:17,010 [salt.transport.mixins.auth:138 ][ERROR ][121471] AES key not found
...
2021-08-13 02:52:16,859 [salt.master :1365][WARNING ][121453] Salt minion claiming to be 39a63bb1-1523-45bb-8bc1-a38d6a884b12 attempted to communicate with master, but key could not be read and verification was denied.
...
2021-08-13 03:19:10,590 [salt.transport.mixins.auth:388 ][INFO ][121471] Authentication failed from host 39a63bb1-1523-45bb-8bc1-a38d6a884b12, the key is in pending and needs to be accepted with salt-key -a 39a63bb1-1523-45bb-8bc1-a38d6a884b12
Cause
The issue can occur when /etc/hosts of the target host contain an incorrect/not current hostname for the localhost.
For example, having the old host name,
SAVLC, in
/etc/hosts can cause the Salt Master service to deny the authentication request from the Salt Minion service.
127.0.0.1 SAVLC localhost localhost.localdomain localhost4 localhost4.localdomain4 39a63bb1-1523-45bb-8bc1-a38d6a884b12
::1 SAVLC localhost localhost.localdomain localhost6 localhost6.localdomain6
...
10.10.7.15 39a63bb1-1523-45bb-8bc1-a38d6a884b12 39a63bb1-1523-45bb-8bc1-a38d6a884b12.netwitness NWVLC
Resolution
In order to resolve the issue, please ensure
/etc/hosts,
/etc/hosts.netwitness and
/etc/hosts.user contain the current hostname.
After the files are being updated, run
nwsetup-tui and discover/enable the host.
If the issue continues, try the steps below and discover again or follow KB
36443 to completely remove the host and start fresh.