Reissue Certificates Errors

[yazantaleb01]

Hello,

I received a critical notification indicating that  some internal certificates used be NetWitness are going to be expired after a few days.

To renew those certificates I followed the official documentation article (https://community.netwitness.com/t5/netwitness-platform-online/reissue-certificates/ta-p/669797)

After running the mention script on the Admin server, multiple errors appeared, and the whole platform went offline.

I'm able to login to the platform but whenever I try to open the any page I get this error:

After some investigation, I fount out that something caused  the script to break and it didn't complete the full job, and the old certificates used by the services were deleted and didn't be replaced, so I tried to manually provide the certificates for each service but this process is very complicated and will take a lot of time.

So I restored the old certificates hoping that I can at least operate the platform and try to reissue the certificates again.

But I'm still facing certificates related error messages.

Here are some of the error messages:

/var/log/messages:

NgNativeReader_NwBroker-FastUpdate: nwsdk failure: NwOpen returned 0; code 0; error: Could not create trusted session: server could not validate and trust our certificate; thread 2366

Admin Server logs:

https-jsse-nio-7009-exec-5] WARN Security|Certificate for CN=netwitness-backend,OU=NetWitness,O=RSA,L=Reston,ST=VA,C=US issued by C=US, ST=VA, L=Reston, O=RSA, OU=NetWitness, CN=NetWitness Intermediate CA is not trusted

Also I cannot run "cert-reissue" because of the following error:

ERROR 98308 --- [ main] c.r.n.i.o.c.OrchestrationApplication : Application startup failed

Is there anything can be done to restore the platform again?

Thanks in Advance.

[Omar_Imam]

Hello Yazan,

.

Kindly note that the process automatically backs up the certificates in /etc/pki/nw_installer.backup. You can use the contents of that directory to replace the contents in /etc/pki/nw/ if the old certificates have not expired yet. 

It is preferable you open a ticket with the support, they will be able to join and help you reissue those certificates.

Thanks

[yazantaleb01]

Hello Omar,

Actually, this is exactly what I've done, I've restored the contents of (nw_installer.backup) as you mentioned and I can see the old certificates

But major errors still appear which prevent the platform from being started.

Regarding the support, there are some points that will take a long time before we can open a support ticket, and we need to solve this problem urgently.

[Omar_Imam]

Can you share the errors (or the whole log file) found in /var/log/netwitness/config-management/chef-solo.log? This should show us the errors that caused the re-issue process to fail.

[yazantaleb01]

Hello,

Here are the errors which appear when trying to  execute 'cert-reissue' along with 'chef-solo.log' contents:

******************************************************************************************************************************************** 

Thanks.

[yazantaleb01]

Here is the errors while executing 'chef-client' that I expect it's being called by the 'cert-reissue' command:

Regards.

[Omar_Imam]

Can you try testing the deploy_admin password on all 4 services and make sure the account is not locked/expired?

You can use the below commands:

1) security-cli-client -p -u deploy_admin -k '<depoly-admin password>'

2) mongo admin -u deploy_admin -p <deploy_admin pw>

3) rabbitmqctl authenticate_user deploy_admin <deploy_admin pw>

4) security-cli-client --get-config-prop --prop-hierarchy nw.security-client --prop-name platform.deployment.password

Note: if the certificates are expired, some of those commands will timeout

 You can also check the below article to unlock the user:

https://community.netwitness.com/t5/netwitness-knowledge-base/how-to-unlock-deploy-admin-password-in-netwitness-platform/ta-p/677241

This issue seems complex enough that I recommend you reach out to support for a quicker/better resolution:

https://community.netwitness.com/t5/support-information/how-to-contact-netwitness-support/ta-p/563897

[h3ll]

Has your issue been solved? I encountered the same problem.