NwLogDecoder Failed to start capture

[Anonymous]

Our LogDecoder refuse to work today. When we  "Start Capture" an Initialization Error occurs.

Failed to start capture: Failed to process message start for /decoder com.rsa.netwitness.carlos.transport.TransportException: terminated

[RSAAdmin]

Hi guys, I had thasame problem and I fixed it the same way, running a data reset, but my folder /var/netwitness/logdecoder/metadb is increasing very fast, about 25GB a day.

I am collecting data from 80 Windows Server via winrm.

Is normal this folder to increase this way?

Regards.

[Anonymous]

That seems to be normal at first glance. Depending on the load on the Windows machines.

What about the size of /var/netwitness/logdecoder/packetdb - nearly the same?

How many entries in table-map.xml and table-map-custom.xml with  flags="None".

Have you used the Windows Event Source "Channel selection feature" for filtering unnecessary events like System^(101|201), Security(4672), Application^(211|300) or do you system collect all the windows channels?

Think about rising of  /Database/Config -> meta.compression.level ...

regards

[RSAAdmin]

Thanks for your reply Davme.

The size of /var/netwitness/logdecoder/packetdb

is not as big as /var/netwitness/logdecoder/metadb, I will check which is the size exactly.

I am not using "Channel selection feature". Is there a recommendation of Event IDs to exclude?

After run a data reset I still can see the logs in Investigation tab, so what kind of data is deleted using this command? Would be the raw log?

Regards.

[Anonymous]

Can you really see the raw logs in investigation after a decoder data reset,or did you only see the indexed metadata at the concentrator/broker level? Security Analytics is a distributed system, you have to clear data in several places decoder/concentrator/broker.

There is no recommendation of Event IDs to exclude, it depends on your experience. Useful Links for it:

https://www.owasp.org/index.php/Logging_Cheat_Sheet

https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/Default.aspx

http://sniperforensicstoolkit.squarespace.com/storage/logging/Windows%20Logging%20Cheat%20Sheet%20v1.1.pdf

[RSAAdmin]

Yes, I see just indexed metada at Concentrator.

My question is, in LogDecoder we have huge logs because LogDecoder stores the raw logs, right?

Is possible to schedule a data reset to run once a week for example?

Regards.

[JonathanSaxon]

Most of our requests are to store more data rather than purging existing data more quickly.   I am not sure why you are interested in manually purging the raw logs from the system. 

Your Log Decoder should be rolling out all data (session, meta, and packet) when the allocated filesystems reach 95% capacity.  If it is not, then something may not be configured properly in the appliance.

There are several things you can do to purge data earlier than the default retention period but I'd like to make sure I understand "what" and "why" before I start making recommendations.  But it is possible to script a "time-roll" or a "size-roll" or even data reset and execute that via a cron job. These are just kind of unusual requests.

A sample "size-roll" operation is shown in KB article 17193 found at https://rsaportal.force.com/customer/articles/Break_Fix/a64807-Log-Decoder-partitions-in-RSA-Security-Analytics-hybrid-o…

Not sure if it would help, but it is also possible to enable metadb compression on Log Decoders and Concentrators.  See KB 27795 at https://rsaportal.force.com/customer/articles/How_To/a65372-Q-A-on-Enabling-Meta-Compression-in-RSA-Security-Analytics-1…

[RSAAdmin]

Thanks JSaxon.

So, I think that LogDecoder is not rolling out the logs automatically but I will check again.

If it works I wont need to use a data reset.

Regards.

[JonathanSaxon]

If you run the latest technical support script (nwtech.sh) and submit the results for evaluation we can let you know if we find any configuration problems that might prevent data from automatically rolling out.

The latest script may be found at https://rsaportal.force.com/customer/articles/How_To/a59741-RSA-NetWitness-and-Security-Analytics-Tech-Support-Data-Gath…. 

Hope that helps.

[RSAAdmin]

Thanks JSaxon.

I am going to wait a week and check if it the logs are being automatically rolling out, if not I will sent you this technical support script.

Regards.