Help Us Help You: Full Filesystem on RSA NetWitness Appliance.
Should you encounter a full filesystem on RSA Netwitness appliance it is helpful to tell us which filesystem(s) are full when you open the case. Different filesystems can fill up for different reasons and depending on which has filled up the solution can often be provided in a single email, assuming we know what is full.
Common Filesystems That Fill Up.
The following filesystems commonly fill up.
| Filesystem | What is "Full" | What to Look For | Resolution |
|---|
| /var/log | 100% constitutes "full" but you may want to take action at 85% or greater. | Most often the logrotate plan is not optimized for smaller /var/log filesystems and a huge messages or security log fills up the filesystem as it is getting zipped for rotation. | Move the huge messages or security file to another partition, zip it manually and then move it back to the /var/log folder. Contact support if you need to modify your log rotation plan. |
| /root | 100% constitutes "full" but you may want to take action at 85% or greater. | We tend to leave a lot of files in /root by default. | Remove any old packages or hot fixes, nwtech output, or scratch files you may have placed in the /root folder. |
/var/netwitness/
<service>/metadb | Greater than 95% is "full." | The most common offending file in the metadb folder is core dump files. They can push the filesystem above 95% and cause a service to stop. | Move any core dump files to another filesystem or to another host. Huge files > 10GB are not easy to send and you may decide to remove them if you simply need to get the system up and running quickly. Numerous core dumps over a few days indicate another problem and you should contact support. |
When opening a case please remember to tell us which filesystems are full.
Also, let us know if the host is a virtual machine or a physical appliance.
Run a df -h command and screen-scrape that output and include with the case notes when you open a new case.
| Sample Output [root@Concentrator-100 /]# df -h from a Series 5 Concentrator with a Single DAC |
|---|
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/netwitness_vg00-root 30G 2.5G 27G 9% /
devtmpfs 63G 0 63G 0% /dev
tmpfs 63G 12K 63G 1% /dev/shm
tmpfs 63G 1.7G 62G 3% /run
tmpfs 63G 0 63G 0% /sys/fs/cgroup
/dev/sda3 516M 186M 331M 36% /boot
/dev/mapper/netwitness_vg00-varlog 10G 2.2G 7.9G 22% /var/log
/dev/mapper/netwitness_vg00-usrhome 10G 33M 10G 1% /home
/dev/mapper/netwitness_vg00-nwhome 203G 3.0G 200G 2% /var/netwitness
/dev/mapper/concentrator-root 30G 3.0G 28G 10% /var/netwitness/concentrator
/dev/mapper/index-index 929G 48G 881G 6% /var/netwitness/concentrator/index
/dev/mapper/concentrator-metadb 23T 1.2T 22T 6% /var/netwitness/concentrator/metadb
/dev/mapper/concentrator-sessiondb 2.6T 40G 2.6T 2% /var/netwitness/concentrator/sessiondb
tmpfs 13G 0 13G 0% /run/user/0 |
Screen-captures from Health & Wellness aren't always as useful in allowing us to diagnose a problem and respond quickly.
If you have any questions please contact Support and reference this article number.
