RSA NetWitness Logs & Network: Host Unreachable alarm appeared in Health and Wellness due to Federation Issue between collectd and rabbitmq

[#1]

Article Number

000002030

Applies To

RSA Product Set: NetWitness Logs & Network
RSA Product/Service Type: NetWitness Logs & Network
 

Issue

Host Unreachable alarm appears in Health and Wellness due to Federation Issue between collectd and rabbitmq.
You are not able to see carlos.sms.collectd queue to SA Server as shown below while all other services like puppet/collectd/rabbitmq-server are no problem.

# rabbitmqctl list_queues -p /rsa/system
Listing queues ...
federation: carlos.audit -> sa@SA-Server 0
federation: carlos.esm.eventsource.changes -> sa@SA-Server 0
federation: carlos.sms.collectd -> sa@SA-Server 0

Cause

There are two things which work together to get stats

1. collectd
2. rabbitmq

For both interactions, they require federation, but in this case, data is coming down to puppet node but federation was broken.

Resolution

You need to follow below steps to fix this issue.

1. ssh into SA Server
2. re-run federateNode twice
   • The first time: /etc/puppet/scripts/federateNode.sh <uuid of problematic host> <ip address of problematic host>
   • Second time: /etc/puppet/scripts/federateNode.sh <uuid of problematic host> <ip address of problematic host> /rsa/sa
3. ssh into problematic host
4. run "puppet agent -t" command

After then, you are now able to see that carlos.sms.collectd queue is successfully created via "rabbitmqctl list_queues -p /rsa/system" command in problematic host.