Appliance enable failed

[RSAAdmin]

I am facing an issue. I upgraded my SA with 10.5 version. But after upgrade, broker is showing message "enable failed, please retry"

I followed this process to resolve this:

In order to resolve the issue, the puppet certificates will need to be reissued on the remote appliance by following the steps below. 

1. 1. Remove the failing appliance from the Security Analytics UI by clicking the Minus ( - ) button and selecting the Remove and Repurpose Appliance option.
2. 2. Connect to both the failed appliance and the Security Analytics server via SSH.
3. 3. On the appliance that is being added, issue the following command, then take note of the Node ID:   cat /var/lib/puppet/node_id
4. 4. On the Security Analytics server, issue the puppet cert list --all command to list all of the certificates known by puppet.
5. 5. Using the Node ID from Step 3, issue the command puppet cert clean <node_id> to remove the certificate from the SA server.  Perform this step regardless of whether the Node ID is listed from step 3.
6. 6. Issue the command vi /var/lib/puppet/ssl/ca/inventory.txt and remove the Node ID from Step 3 if it is listed.
7. 7. On the appliance to be added, issue the following command to remove any previously issued certificates:  rm -rf /var/lib/puppet/ssl
8. 8. Remove the service-specific certificates depending on what services are running on the appliance by issuing the commands below:
   NOTE: Replace <service> below with the service name, i.e. appliance, broker, concentrator, etc.
   • rm -f /etc/netwitness/ng/<service>/storedproc/*
   • rm -f /etc/netwitness/ng/<service>/trustpeers/*
   • rm -f /etc/netwitness/ng/<service>/truststore/*
9. 9. On the same appliance, regenerate the certificates by issuing the following command:  puppet agent -t --waitforcert 30

 
After performing the steps above, move to the Security Analytics UI and click on the Discover button on the Administration -> Appliances screen.
At this point, the UI should be able to recognize the new appliance as expected.

-------------------------------------------------------------------------------------------------------------------------------------------

I was able to discover my appliance but again same problem is coming.: enable failed, please retry"

Any Solution?

Thanks and Regards,

Priya Malik

[Anonymous]

Hi Priya,

When you run the command puppet agent -t on the affected appliance, do you see any error messages in red?  Also, do you see any errors on the Security Analytics Server appliance (assuming it's not the same appliance as the broker that is having issues) when running the tail -f /var/log/messages | grep puppet command at the same time?

Thanks,

Jeff

[RSAAdmin]

Hi Jeff,

Command output :

[root@BSABROV01 ~]# puppet agent -t

Info: Retrieving pluginfacts

Info: Retrieving plugin

Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb

Info: Loading facts in /var/lib/puppet/lib/facter/management_interface.rb

Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb

Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb

Info: Loading facts in /var/lib/puppet/lib/facter/node_id.rb

Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb

Info: Loading facts in /var/lib/puppet/lib/facter/java_version.rb

Info: Loading facts in /var/lib/puppet/lib/facter/ca_cert_hash.rb

Info: Loading facts in /var/lib/puppet/lib/facter/java_major_version.rb

Info: Loading facts in /var/lib/puppet/lib/facter/iptables_persistent_version.rb

Info: Loading facts in /var/lib/puppet/lib/facter/java_patch_level.rb

Info: Loading facts in /var/lib/puppet/lib/facter/iptables_version.rb

Info: Loading facts in /var/lib/puppet/lib/facter/node_cert_hash.rb

Info: Loading facts in /var/lib/puppet/lib/facter/ip6tables_version.rb

Info: Caching catalog for 2ccb9a8e-2afb-4429-a289-40fd238a4139

Info: Applying configuration version '1439369081'

Notice: /Stage[main]/Yumconfig/Exec[disable-Centos-Repos]/returns: executed successfully

Notice: Finished catalog run in 24.36 seconds

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------

[root@BSASRVV01 ~]# tail -f /var/log/messages | grep puppet

Aug 13 04:14:17 BSASRVV01 puppet-master[2076]: Compiled catalog for 6ce79c58-c079-4559-a1bf-321070600e34 in environment production in 0.62 seconds

Aug 13 04:14:59 BSASRVV01 puppet-agent[11292]: (/Stage[main]/Yumconfig/Exec[disable-Centos-Repos]/returns) executed successfully

Aug 13 04:15:02 BSASRVV01 puppet-agent[11292]: (/Stage[main]/Yumconfig/Package[rsa-sa-gpg-pubkeys]/ensure) created

Aug 13 04:15:11 BSASRVV01 puppet-agent[11292]: Finished catalog run in 51.00 seconds

Aug 13 04:20:03 BSASRVV01 puppet-master[2076]: Compiled catalog for 2ccb9a8e-2afb-4429-a289-40fd238a4139 in environment production in 0.42 seconds

Aug 13 04:40:04 BSASRVV01 puppet-master[2076]: Compiled catalog for 2ccb9a8e-2afb-4429-a289-40fd238a4139 in environment production in 0.42 seconds

---------------------------------------------------------------------------------------------------------------------------------------------------------------------

But When I clicked on enable button at administrator-->appliance and checked

[root@BSASRVV01 ~]# tail -f /var/log/messages

Aug 13 04:44:17 BSASRVV01 python: Adding 2ccb9a8e-2afb-4429-a289-40fd238a4139 user to /rsa/sa

Aug 13 04:44:17 BSASRVV01 puppet-master[2076]: Compiled catalog for 6ce79c58-c079-4559-a1bf-321070600e34 in environment production in 0.52 seconds

Aug 13 04:44:18 BSASRVV01 add-rabbitmq-user: Cleared user 2ccb9a8e-2afb-4429-a289-40fd238a4139 password

Aug 13 04:44:19 BSASRVV01 add-rabbitmq-user: Set configure, read, and write permissions for 2ccb9a8e-2afb-4429-a289-40fd238a4139 in vhost /rsa/sa

Aug 13 04:44:19 BSASRVV01 python: User 2ccb9a8e-2afb-4429-a289-40fd238a4139 added to /rsa/sa

Aug 13 04:44:19 BSASRVV01 python: Federating upstream node 2ccb9a8e-2afb-4429-a289-40fd238a4139, on /rsa/system at 10.162.4.148

Aug 13 04:44:20 BSASRVV01 python: Federating upstream node 2ccb9a8e-2afb-4429-a289-40fd238a4139, on /rsa/sa at 10.162.4.148

Aug 13 04:44:20 BSASRVV01 python: Signing Puppet Cert

Aug 13 04:44:21 BSASRVV01 python: #033[1;31mError: Could not find certificate request for 2ccb9a8e-2afb-4429-a289-40fd238a4139#033[0m

Aug 13 04:44:21 BSASRVV01 python: Pinging host 2ccb9a8e-2afb-4429-a289-40fd238a4139 with a 30 second timeout.

Aug 13 04:44:52 BSASRVV01 python: Pinging host 2ccb9a8e-2afb-4429-a289-40fd238a4139 with a 30 second timeout.

Aug 13 04:44:52 BSASRVV01 collectd[2464]: ESMAggregator: Persisted 0 of 0 stats in /var/lib/netwitness/collectd/ESMAggregator

Aug 13 04:44:55 BSASRVV01 puppet-agent[7854]: (/Stage[main]/Yumconfig/Exec[disable-Centos-Repos]/returns) executed successfully

Aug 13 04:44:58 BSASRVV01 puppet-agent[7854]: (/Stage[main]/Yumconfig/Package[rsa-sa-gpg-pubkeys]/ensure) created

Aug 13 04:45:06 BSASRVV01 puppet-agent[7854]: Finished catalog run in 45.77 seconds

Aug 13 04:45:22 BSASRVV01 python: Pinging host 2ccb9a8e-2afb-4429-a289-40fd238a4139 with a 30 second timeout.

Aug 13 04:45:53 BSASRVV01 python: Pinging host 2ccb9a8e-2afb-4429-a289-40fd238a4139 with a 30 second timeout.

---------------------------------------------------------------------------------------------------------------------------------------------------------------

I have done remove and re-purpose too but it is also not working.

Thanks and Regards,

Priya Malik

[Anonymous]

Hi Priya,

Sorry for the delay.  At this point I would recommend engaging the support team the monitor the puppet exchange so that they can determine why the CSR can't be found.  Also, when you look in the /var/lib/puppet/ssl/certificate_requests directory on the SA Server during the puppet catalog run, do you see a file with the name 2ccb9a8e-2afb-4429-a289-40fd238a4139?

Thanks,

Jeff