RSA Security Analytics UI is not available after upgrade from 10.4.x to 10.5.0.0 due to mongoDBMigration failure

[#1]

Article Number

000031638

Applies To

RSA Product Set: Security Analytics
RSA Product/Service Type: Security Analytics UI, Security Analytics Server
RSA Version/Condition: 10.5.0.0
 

Issue

When SA server is upgraded from 10.4.x to 10.5.0.0, the UI does not come online and shows the error below in the browser.
 

Error Description:

Security Analytics server is unable to start. Please make sure all services are running and restart the Security Analytics server.

View Error
Stack Backtrace

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mongoDBMigration': Invocation of init method failed; nested exception is java.lang.NullPointerException

at InitDestroyAnnotationBeanPostProcessor postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:133)
at AbstractAutowireCapableBeanFactory applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:396)
at AbstractAutowireCapableBeanFactory initializeBean(AbstractAutowireCapableBeanFactory.java:1,507)
at AbstractAutowireCapableBeanFactory doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
at AbstractAutowireCapableBeanFactory createBean(AbstractAutowireCapableBeanFactory.java:458)
at AbstractBeanFactory$1 getObject(AbstractBeanFactory.java:293)
at DefaultSingletonBeanRegistry getSingleton(DefaultSingletonBeanRegistry.java:223)
at AbstractBeanFactory doGetBean(AbstractBeanFactory.java:290)
at AbstractBeanFactory getBean(AbstractBeanFactory.java:191)
at DefaultListableBeanFactory preInstantiateSingletons(DefaultListableBeanFactory.java:636)
at AbstractApplicationContext finishBeanFactoryInitialization(AbstractApplicationContext.java:934)
at AbstractApplicationContext refresh(AbstractApplicationContext.java:479)
at ContextLoader configureAndRefreshWebApplicationContext(ContextLoader.java:410)
at ContextLoader initWebApplicationContext(ContextLoader.java:306)
at ContextLoaderListener contextInitialized(ContextLoaderListener.java:112)
at SafeModeContextListener contextInitialized(SafeModeContextListener.java:37)
at ContextHandler callContextInitialized(ContextHandler.java:788)
at ServletContextHandler callContextInitialized(ServletContextHandler.java:434)
at ContextHandler startContext(ContextHandler.java:780)
at ServletContextHandler startContext(ServletContextHandler.java:284)
at WebAppContext startContext(WebAppContext.java:1,322)
at ContextHandler doStart(ContextHandler.java:732)
at WebAppContext doStart(WebAppContext.java:490)
at AbstractLifeCycle start(AbstractLifeCycle.java:69)
at ContainerLifeCycle start(ContainerLifeCycle.java:118)
at ContainerLifeCycle addBean(ContainerLifeCycle.java:282)
at ContainerLifeCycle addBean(ContainerLifeCycle.java:214)
at ContainerLifeCycle updateBeans(ContainerLifeCycle.java:764)
at HandlerCollection setHandlers(HandlerCollection.java:89)
at ContextHandlerCollection setHandlers(ContextHandlerCollection.java:145)
at HandlerCollection addHandler(HandlerCollection.java:155)
at StandardDeployer processBinding(StandardDeployer.java:41)
at AppLifeCycle runBindings(AppLifeCycle.java:186)
at DeploymentManager requestAppGoal(DeploymentManager.java:495)
at DeploymentManager addApp(DeploymentManager.java:146)
at ScanningAppProvider fileAdded(ScanningAppProvider.java:175)
at ScanningAppProvider$1 fileAdded(ScanningAppProvider.java:64)
at Scanner reportAddition(Scanner.java:605)
at Scanner reportDifferences(Scanner.java:528)
at Scanner scan(Scanner.java:391)
at Scanner doStart(Scanner.java:313)
at AbstractLifeCycle start(AbstractLifeCycle.java:69)
at ScanningAppProvider doStart(ScanningAppProvider.java:145)
at AbstractLifeCycle start(AbstractLifeCycle.java:69)
at DeploymentManager startAppProvider(DeploymentManager.java:557)
at DeploymentManager doStart(DeploymentManager.java:232)
at AbstractLifeCycle start(AbstractLifeCycle.java:69)
at ContainerLifeCycle start(ContainerLifeCycle.java:118)
at Server start(Server.java:342)
at ContainerLifeCycle doStart(ContainerLifeCycle.java:100)
at AbstractHandler doStart(AbstractHandler.java:60)
at Server doStart(Server.java:290)
at AbstractLifeCycle start(AbstractLifeCycle.java:69)
at XmlConfiguration$1 run(XmlConfiguration.java:1,250)
at AccessController doPrivileged(AccessController.java:-2)
at XmlConfiguration main(XmlConfiguration.java:1,174)
at NativeMethodAccessorImpl invoke0(NativeMethodAccessorImpl.java:-2)
at NativeMethodAccessorImpl invoke(NativeMethodAccessorImpl.java:62)
at DelegatingMethodAccessorImpl invoke(DelegatingMethodAccessorImpl.java:43)
at Method invoke(Method.java:483)
at Main invokeMain(Main.java:509)
at Main start(Main.java:651)
at Main main(Main.java:99

Cause

This issue is due to inconsistent rules in ESA. If ESA rule deleted from rule library without first deleting in ESA synchronization,
Would cause the orphan reference to the ESA rule which fails MongoDB migration during upgrade. The failure of MongoDB migration would cause the GUI offline.

Resolution

The ESA rule inconsistency can fixed by using javascript file: FixEsaRecords.js which is available in this article.

1. Upload FixEsaRecords.js file into SA server using WinSCP tool.
2. Login to SA server using ssh with admin credentials.
3. Change the permissions to the file using below command.

chmod +x FixEsaRecords.js

3. Run below command remove orphan rules from MongoDB.

mongo localhost:27017/sa<FixEsaRecords.js

Sample reference which can show the removal of rule.

Image description

4.Reboot the SA server.
5. Verify GUI access by launching with browser.

Notes

This workaround would not remove any Existing ESA rules. Those rules which have been removed from rule library and still stayed in Synchronization would be deleted.