Remove session, meta and raw log data from Archivers

[KEVINDIENST]

I am trying to find a method where I can remove unneeded data from our archivers. 

Idea:

1. Run nwconsole (sdk content command) on all Archivers and output sessionid list that match where clause device.type = 'unknown'
2. Feed that sessionid list to wipe SDK command on archivers to clear meta, session, raw for those sessionids on all attached DAC storage. 

Has anyone done anything similar to this in the past and can provide me a way in NwConsole to export sessionids?

[NaushadKasu1]

Kevin,

You can use the nwget-logs.py script which uses sessionids to pull raw logs, you can modify that script to output a text file with the sessionids. I provided the script some time ago -- let me know if you need it again and I'll email it to you. However, I have not been able to get the 'wipe' REST call to work for meta or raw (packets/logs) and I have an open engineering case on the functionality. I'll let you know once I make progress on that case. If you try to send a valid sessionid to the Log Decoder or Concentrator 'database' -> 'wipe' -> using meta (m) or raw (p), it doesn't seem to remove that session.

See script attached. Here's a sample run:

./nwget-logs.py -u admin -p netwitness -d 192.168.1.123 -t "2019-Sep-30 00:00:00" -e "2019-Sep-30 01:00:00" -P 50105 -l 2 -f "did exists"

[KEVINDIENST]

OK, I'll try to use the nwget-logs.py script instead to pull those sessionids, but yeah, if the wipe command doesn't actually wipe that session, well...

Thank you

[MatthewTharp1]

Hey Kevin,

I have done similar things, but nothing exactly the same. Try this though:

NwConsole -c login {ip:port} {username} {password} -c send sdk/ values "size={however many results you want} fieldName=sessionid where="device.type = 'unknown'" --output-format=values --output-pathname={wherever}"

You may not want the values format, but that should give you the sessionids that match your where query.

NOTE: This queries all time, so you may want to add some time parameters.

[LeeKirkpatrick]

Hey Kevin,

Just to set expectations, the wipe command will not remove data, but instead, overwrite it with a pattern; this will therefore not free up space on the DAC if that is the intention.

Secondly, you will also want to delete the cached data after running the wipe command to ensure none of the wiped data is retrieved from the cache during investigations.

Cheers,

Lee

[NaushadKasu1]

Kevin,

I've been told this was patched/fixed in a future (11.3) build. However, as Lee mentioned -- this functionality will only wipe the data (zero it out) and will not free up disk space.