Article Number
000001849
Applies To
RSA Product Set: NetWitness Platform
RSA Product/Service Type: Admin Server
RSA Version/Condition: 11.4.x
Platform: CentOS 7
Issue
The Admin Server service shows that it is in an unhealthy state for an unspecified amount of time but will recover for a short time before showing unhealthy again.
Cause
The cause for an unhealthy Admin Server service can be caused by permissions being incorrect on its configuration file that is found at /var/netwitness/admin-server/admin-server.conf. With incorrect permissions, it can cause the Java virtual machine to start up with the incorrect amount of memory that is allocated to it. In this case too much memory allocated, starving the server of needed RAM.
Resolution
To determine if incorrect permissions are causing the unhealthy Admin Server service issue, perform the following steps.
- SSH to the NW Admin Server
- Go to the /etc/netwitness/admin-server directory.
# cd /etc/netwitness/admin-server
- Perform a long listing of the files within the directory, looking specifically at the admin-server.conf file permissions.
# ls -l
-r--------. 1 root root 576 Jan 6 2020 admin-server.conf
-rw-r--r--. 1 netwitness netwitness 12550 Jun 24 2019 keystore.p12
-rw-r--r--. 1 netwitness netwitness 986 Jun 24 2019 lockbox.ss
- In the example from the previous step the admin-server.conf is owned and grouped by root. With these permissions, the Admin Server service will not be able to read the configuration file since the permissions on the file are very restrictive. The user and group permissions must be set to netwitness (UID/GID: 2000)
# chmod netwitness netwitness admin-server.conf
- Recheck the file permissions to confirm that it was changed correctly.
# ls -l
-r--------. 1 netwitness netwitness 576 Jan 6 2020 admin-server.conf
-rw-r--r--. 1 netwitness netwitness 12550 Jun 24 2019 keystore.p12
-rw-r--r--. 1 netwitness netwitness 986 Jun 24 2019 lockbox.ss
- Restart the Admin Server service to read in the configuration file.
# systemctl restart rsa-nw-admin-server
- Once the Admin Server is restarted, check the running processes and confirm that the correct memory configuration parameters are seen. It may be necessary to compare the output with the JAVA_OPTS= parameter within the admin-server.conf file.
# ps -ef | grep admin-server
netwitn+ 5243 1 0 May22 ? 00:00:00 /bin/bash /usr/sbin/admin-server.jar
netwitn+ 5338 5243 0 May22 ? 05:03:06 /usr/bin/java -Dsun.misc.URLClassPath.disableJarChecking=true -XX:+UseG1GC -Djava.security.egd=file:/dev/./urandom -Xmx2G -jar /usr/sbin/admin-server.jar --rsa.security.pki.ciphers=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256
- If this did not address the issue being seen, open a case with RSA Netwitness Support.