Different methods for exporting logs and pcaps in RSA NetWitness

[#1]

Article Number

000001294

Applies To

RSA Product Set: NetWitness Logs & Network
RSA Product/Service Type: Core Appliance
RSA Version/Condition: 10.6.x, 11.x

Issue

NetWitness imposes a hard coded limit of  <100,000 in the RSA NetWitness GUI.  

Resolution

The following 3 methods may be used to Extract PCAP/LOGS:
 
Method 1: using REST API, restricts export to 1GB or less

http://<Logdecoder_IP>:50102/sdk/packets
http://<Packetdecoder_IP>:50104/sdk/packets

Method 2: Using Curl command line, restricts export to 1GB or less

# curl -u admin "http://<LOG_DECODER_IP>:50102/sdk/packets?render=logs%time1=<START_TIME>&time2=<END_TIME>"
# curl -u admin "http://<PACKET_DECODER_IP>:50104/sdk/packets?render=pcap%time1=<START_TIME>&time2=<END_TIME>"

Method 3: Using SDK no published limitations 

Once connected to the log concentrator, log in to the NwConsole, and connect to the concentrator service.
(Note: ssl is only used if ssl has been enabled on the appliance service. nws = ssl, nw = non-ssl)

#NwConsole
> login localhost:50005:[ssl] admin [password]
> sdk open nw[s]://admin:[password]@[hostname]:50005   
(Note: ssl is only used if ssl has been enabled on the appliance service. nws = ssl, nw = non-ssl.
Also if your password has a @ character the hostame will likely not connect, try creating a new service account user and password that does not have a @)

Once connected, issue the following command to begin the log extraction process:

> sdk content sessions=1-now render=logs dir="/root/logs" where="(time='2014-03-12 18:00:00'-'2014-03-12 18:30:00' && device.class = 'windows hosts' && user.dst = 'envisionrsa')" fileExt=.log append=arc_log_extract


Command Breakdown:
sessions – the first session until now
render – generate the file as a "logs" or "pcap" for pckets
dir  - the location where the log file will be saved
where – the where query from step 3
fileExt – the extension that will be placed on the created log file
append – the name of the  log file that will be created


Sample output:

Sessions 1 to 9620098716 have meta range 1 to 190837549810
Found 10000+ new session(s) between meta range 1 to 190837549810
Activating thread for processing
Submitting request to stream logs for 10000 sessions
553 logs written, 5% complete
1050 logs written, 10% complete