The content you are looking for has been archived. View related content below.
018-01-30 19:59:02,629 [ ServiceInstallJobHandler] WARN ConfigurationManagement|Failed to check isJobReady org.springframework.http.converter.HttpMessageNotReadableException: Could not read document: Can not deserialize instance of java.util.LinkedHashMap out of VALUE_FALSE token at [Source: java.io.PushbackInputStream@702de5ea; line: 1, column: 54] (through reference chain: com.rsa.asoc.orchestration.salt.client.domain.LocalResponse["return"]-> java.util.ArrayList[0]->java.util.LinkedHashMap["992dcb26-39c2-4c29-b9c9-7f5e98f3c542"]); nested exception is com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.util.LinkedHashMap out of VALUE_FALSE token at [Source: java.io.PushbackInputStream@702de5ea; line: 1, column: 54] (through reference chain: com.rsa.asoc.orchestration.salt.client.domain.LocalResponse["return"]-> java.util.ArrayList[0]->java.util.LinkedHashMap["992dcb26-39c2-4c29-b9c9-7f5e98f3c542"]) at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:240) at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:225) at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:95) at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:917) at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:901) at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:655) at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:628) at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:602) at com.rsa.asoc.orchestration.salt.client.DefaultSaltClient.run(DefaultSaltClient.java:118) at com.rsa.asoc.orchestration.task.AsynchronousTaskMonitor.isJobReady(AsynchronousTaskMonitor.java:115) at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174) at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at com.rsa.asoc.orchestration.task.AsynchronousTaskMonitor.pollTasks(AsynchronousTaskMonitor.java:88) at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.util.LinkedHashMap out of VALUE_FALSE token at [Source: java.io.PushbackInputStream@702de5ea; line: 1, column: 54] (through reference chain: com.rsa.asoc.orchestration.salt.client.domain.LocalResponse["return"]-> java.util.ArrayList[0]->java.util.LinkedHashMap["992dcb26-39c2-4c29-b9c9-7f5e98f3c542"]) at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:270) at com.fasterxml.jackson.databind.DeserializationContext.reportMappingException(DeserializationContext.java:1234) at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1122) at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1075) at com.fasterxml.jackson.databind.deser.std.StdDeserializer._deserializeFromEmpty(StdDeserializer.java:892) at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:358) at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:27) at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:517) at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:362) at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:27) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:287) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:259) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:26) at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:499) at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:101) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:357) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:148) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3798) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2922) at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:237) ... 23 common frames omittedThis could indicate that there is a problem with the salt service on the host being upgraded that prevents the orchestration server from communicating with it properly. As part of the chef run, we mask the salt-minion service to prevent users from interacting with it until the chef process is done. In the case of a previous failure, we may never get to the step that we unmask the salt service, thus the above error is displayed.
2018-12-17 23:44:10,004 [ cfg-mgmt-0] ERROR TaskManagement|Execution of '5c1834c991c71f054873335b' failed org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://localhost:8000/run": Connect to localhost:8000 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused); nested exception is org.apache.http.conn.HttpHostConnectException: Connect to localhost:8000 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused) at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:674) at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:636) at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:610) at com.rsa.asoc.orchestration.salt.client.DefaultSaltClient.run(DefaultSaltClient.java:118) at com.rsa.asoc.orchestration.salt.handler.HostVerifyTaskHandler.execute(HostVerifyTaskHandler.java:52) at com.rsa.asoc.orchestration.task.TaskExecutionService.execute(TaskExecutionService.java:163) at com.rsa.asoc.orchestration.task.TaskExecutionService.lambda$null$3(TaskExecutionService.java:147) at java.util.ArrayList.forEach(ArrayList.java:1257) at com.rsa.asoc.orchestration.task.TaskExecutionService.lambda$submit$4(TaskExecutionService.java:147) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8000 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:89) at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48) at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53) at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:660) ... 13 common frames omittedThis is also related to another salt issue but you are more likely to see this just on the Admin Server itself rather than another host.
systemctl unmask salt-minion systemctl restart salt-minionIf you are finding yourself fitting into Example 2, then do the above AND also the following on the Admin Server:
systemctl unmask salt-master && systemctl unmask salt-api systemctl restart salt-master && systemctl restart salt-api
tailf /var/log/netwitness/config-management/chef-solo.log