Troubleshooting Cert-Reissue Command

You must contact Customer Support (https://community.netwitness.com/t5/support-information/how-to-contact-netwitness-support/ta-p/563897) to troubleshoot problems. You know there is a problem if any <host-id> does not return a SuccessStatus. Success indicates that certificates were reissued for a host.

Argument Options Used for Troubleshooting

You use the following argument options with cert-reissue --host-all to troubleshoot problems.

You can run cert-reissue --host-all<arguments> multiple times without an adverse effect.

Note: Use the following Argument Options with caution. They force the cert-reissue command to execute for all the hosts.

Argument Option Description
--skip-health-checks

Reissues certificates for all hosts at one time without applying system health checks (force Reissue). This means that the command does not:

  • verify that all hosts are online line.
  • verify that all services are running.

Use case: You have numerous hosts and you know that a small minority of them will fail. This updates all the hosts that conform to the checking rules and you can reissue certificates for the others subsequently with the help of Customer Support.

--skip-version-checks

Do not verify that hosts are running version 11.4.0.0 or later.

Use Case: You have numerous hosts and your know that some of them are not updated to 11.4 or later. This reissues certificates for all the hosts that are at 11.4 or later and you can reissue certificates for the others subsequently with the help of Customer Support.

--ignore-trigger-errors

Ignore any errors that trigger failures. This option forces the cert reissue process to continue disregarding the errors instead of aborting or failing the cert reissue command quickly.

When a cert reissue for a host succeeds, the reissued certificates on that host are not provisioned to other dependent hosts (referred to as trusts). In this case, the:

  • host with reissued certificates is reported as “Partial.”
  • the hosts with trusts that failed to update are listed separately in the summary table to tell you that these hosts may require a refresh using the new --refresh-trusts-only option.
--refresh-trusts-only Refreshes trusts exclusively for host identified by <id> (does not reissue certificates for that host).

Problems and How to Troubleshoot Them

This section describes solutions to problems that you may encounter when running the cert-reissue command to reissue certificates with suggested causes and solutions.

Status Failed!
Error Message

...

2019-02-06 13:34:39.646 INFO 8540 --- [ main] c.r.n.i.o.client.OrchestrationClient : Checking host connections...

...

2019-02-06 13:34:57.861 ERROR 8540 --- [ main] c.r.n.i.o.client.HostValidator : Host '192.168.200.99' (nw-platform-esa-primary) verification failed!

...

2019-02-06 13:34:57.862 INFO 8540 --- [ main] c.r.n.i.o.client.OrchestrationClient : Checking status of services...

2019-02-06 13:35:57.931 ERROR 8540 --- [ main] c.r.n.i.o.client.HostValidator : Service 'nw-platform-node-zero - Investigate Server' not available!

...

netwitness_reissuecert-ts1.png

Cause cert-reissue --host-all failed because one or more hosts are offline or one or more run time services are unreachable. You can force this command to run in spite of this error by specifying the --skip-health-checks option, that is:
cert-reissue --host-all--skip-health-checks
Solution
  1. Bring appropriate hosts back online or make sure the NW Server hosts run time services are running.
  2. Run cert-reissue for the hosts affected.

Status Failed!
Error Message

...

2019-02-06 13:34:39.643 ERROR 8540 --- [ main] c.r.n.i.o.client.HostValidator : Host '192.168.200.102' (nw-platform-decoder) version '11.2.0.0' not supported, minimum required version: 11.3.0.0

2019-02-06 13:34:39.644 ERROR 8540 --- [ main] c.r.n.i.o.client.HostValidator : Host '192.168.200.101' (nw-platform-concentrator) version '11.2.0.0' not supported, minimum required version: 11.3.0.0

...

netwitness_reissuecert-ts2.png

Cause

cert-reissue -host-all command string failed because one or more hosts are running a version earlier than 11.4.0.0

Note: You can force the reissue of certificates for the remaining hosts using the -skip-version-checks argument.

Solution

Update the host to 11.4 or later and run cert-reissue for that host again.

Status Partial
Error Message

...

2019-02-06 02:27:09.078 ERROR 20647 --- [ main] c.r.n.i.o.client.OrchestrationClient : Trigger failed for host '<IP-address>' (nw-platform-decoder)

2019-02-06 02:27:09.079 ERROR 20647 --- [ main] c.r.n.i.o.client.OrchestrationClient : Trigger failed for host '<IP-address>' (nw-platform-concentrator)

...

2019-02-06 02:27:09.118 WARN 20647 --- [ main] c.r.n.i.o.client.OrchestrationClient : One or more host(s) may require manual refresh due to failed triggers:

netwitness_reissuecert-ts3.png

Cause cert-reissue command completed on NW Server host however one or more triggers failed. This aborted the cert-reissue command for other hosts.
Solution

Address all the errors and run the cert-reissue --host-all<arguments> command string again.

Status Partial
Error Message

...

2019-02-06 14:18:03.208 ERROR 17800 --- [ main] c.r.n.i.o.client.OrchestrationClient : Trigger failed for host '192.168.200.82' (nw-platform-node-x)

...

...

2019-02-06 14:29:05.200 WARN 17800 --- [ main] c.r.n.i.o.client.OrchestrationClient : One or more host(s) may require manual refresh due to failed triggers:

netwitness_reissuecert-ts4.png

Cause One or more hosts did not pass system health checks. In addition, one or more of the unhealthy hosts are running core services, which will result in the NW Server host cert-reissue to fail (because of failed triggers explained above). By disabling health checks and trigger errors, you can continue the process and reissue certificates for the remaining hosts. The NW Server host Status is reported as Partial because the cert-reissue command completed for the NW Server but downstream triggers failed for other hosts.
Solution

Manually refresh the failed core hosts (to synchronize trust peers).

Submit the following command string to reissue certificates for healthy hosts.
cert-reissue --host-all --skip-health-checks --ignore-trigger-errors