RSA SA Health & Wellness indicates /var/netwitness/logdecoder/sessiondb partition almost 100% utilization

[pranavsankar1]

Hi All ,

Health and Wellness indicates the /var/netwitness/logdecoder/sessiondb partition is almost 100% utilization.

When i check with df -h

/dev/mapper/logdecodersmall-sessiondb

                      600G  570G   30G  96% /var/netwitness/logdecoder/sessiondb

So im in a dilemma that if it is fully utilized then event capturing will be effect and when check for files in the decoder i could see large files listed by command du -sh * .

Is it possible to move online files to my temp backup server or how in SA we make online data to offline date ?

Thanks in Advance

Regards

Pranav Sankar

[KhaledGamal]

Hi Pranav,

The alarm in the health and wellness by default "if not changed" triggers when one or more database mount point gets over 97% full. When it goes down below 97 % it gets cleared but it could take some time till it gets cleared. Of course it will only get below 97 % if the rollover threshold is setup. As from the df -h output you sent, it seems that it is below 97 % so I guess by now it should already be cleared. If the alarm is not cleared, please send me a screenshot of the logdecoder -> explore -> database -> config page so I can check and confirm if the threshold parameter is configured.

It is not recommended to move online "database" files as it could corrupt the database.

Best regards

Khaled

[pranavsankar1]

Hi Khaled ,

Still df -h output is giving me the same result. Please find the screenshot of log decoder->explore->config page.

Awaiting for your valuable revert .

Thanks  !

Regards,

Pranav Sankar

[KhaledGamal · Accepted solution]

Hi Pranav,

From the explore view, could you get me a screenshot of the session.dir value as it is not shown above. If there is a value there, this means that the rollover threshold is set and the database is rolling over. This means that the alarm will just take some time and will clear "if policy is for 97%".

You could also open the Health and wellness policy to confirm the threshold that the alarm is triggered. From the screenshot below, it shows that the threshold is 95 % which means that is why it is triggered. Health and wellness-> Policies -> Host Policy

My recommendations:

1- Raise the threshold to 97 % and raise the recovery threshold to 97 % as well. This will prevent false positives in the future.

2- Confirm that the session.dir has a value for rollover to prevent the session DB to get full.

After doing the 2 steps above, monitor for around an hour and you will find the alarm cleared.

Hope this helps!

Best regards,

Khaled

[pranavsankar1]

Hi Khaled ,

Sorry for not including value for session.dir while snatching the screenshot i missed it .Please find details.

Thanks for the information looks good to me , But Khaled even after raising the threshold value to 97% is it applicable ?

In case if the threshold crosses 97% it will again triggers the alarm ?

Thanks !

Regards ,

Pranav Sankar

[KhaledGamal]

Hi Pranav,

yes you are correct, if the disk usage is over 97 % it will trigger an alarm, but in normal circumstances and as the threshold of the session DB is configured, the database will not pass the 97% threshold. It will only pass that threshold if there is an issue and for example core files are created. In normal circumstances, 97 % for the policy is good and will eliminate false positives.

Note that the alarm in your case is triggered because sometimes the threshold to rollover is a bit over 95 % and it will start rolling over after the whole file is written to disk and hence sometimes passing 95 or 96 %.

97 % is a safe threshold and it should eliminate most false positives.

Best regards

Khaled

[pranavsankar1]

Hi Khaled ,

Sounds good to me ill go for raising the threshold value to 97% and let you know if any alarms triggered.

Once again much appreciated for your valuable reverts.

Thanks !

Regards

Pranav Sankar

[KhaledGamal]

You are always welcome .