Question relating to Batch Scripting PCAPS into a Decoder

[RSAAdmin]

First I thank those that posted the hints on how to batch upload, and after a few tweaks to get my version of Windows to behave they worked perfectly except!

The way Netwitness is used here is by pcap ingestion and multiple analyses is carried out on the same Decoder/Concentrator.  These individual  analyses are identified by the identification of the folder containing the pcaps (a 6 digit number).  If the ingestion is carried out via the Administrator tool (and the analyst has remembered to tick the box!) then the FQPN of the pcap being ingested is included in the 'sourcefile' metadata field.  However if the build update mechanisms using curl are used only the actual file name is included.  This could be a potential problem with our form of use as two or more paps can have the same file name, (they are auto-generated,) but are distinguished by their holding folder's 6 digit name.

Does anybody have any suggestions on how the batch upload, or something like it, can be persuaded to store the FQPN instead of just the file name.  The reason for the question is that it is proposed to introduce an automated system that could undertake the ingest on a continuous basis as new pcaps become available rather than individual analysts fighting to gain control of the Decoder/Concentrator to upload their latest pcaps. (Only one analyst can upload at any time.)

This may have already been addressed as currently we are only on a version 9 and are awaiting the funding to undertake the upgrade to the Security Analytics 10,,, If this is only available on 10 and above then we stay with the current method and moan at the finance providers to be quicker, but if there is a 9 solution then it would allow some pre-development to take place.

[Anonymous]

You can try NwConsole with something like

NwConsole -c login localhost:56004:ssl user password -c cd sdk -c import /mnt/Users/fullpath.pcap -c logout

As long as you pass it the full path it should store the full path in sourcefile, you can easily wrap this around a script or one liner to monitor a specific location, upload the files and move them.

Hope this helps! It should work on 9.x and 10.x the example above is from 10.5

Cheers,

Rui

[RSAAdmin]

Dear Sir

Thank you indeed for your helpful response.  I am replying to you immediately to show how grateful I am for your assistance.

Unfortunately due to certain circumstances I will not be able to try this for a brief period, e.g. the next reload of the database but I am extremely grateful for your time and assistance.  I thought I had cracked it with the RESTapi batch script but this simply took the final file name and ignored the rest of the FQPN, ouch.

When I get a chance to try I will drop you a quick email to say yes it worked or no it didn't.

Have a very pleasant weekend. Regards

Brian R

[RSAAdmin]

In respect of the solution provided, as I said I would respond when I was able to try this out.  Unfortunately, and nothing to do with NetWitness, the rebuilding the database came a little quicker than expected, however this did give me the opportunity to try the solution out and with  a few minor tweaks it works perfectly.  The minor tweaks were that the decoder  refused access via port 56004 and if 50004 was used then it was unhappy with the ssl.  However replacing :56004:ssl with just 50004 and it worked perfectly providing the FQPN for the pcap file ingested.  This need to adjust may be due the the way the Decoder / Concentrator have been configured, that is for somebody else to think about.  From my point of view IT WORKS so Thank You for all the assistance.

[Anonymous · Accepted solution]

Glad to hear it worked, apologies for the confusion with ports/SSL as mentioned my example was for 10.5 which is what I have available to test on.

For 9.8 and 10.x up to 10.4 the correct would be indeed 50004 and the ":ssl" only necessary if SSL was enabled on the services by default they are off

So for completeness and for what was described as you need this following shell command should do exactly what you need, this time with the correct syntax for your configuration:

find / -name \*.pcap -print -exec NwConsole -c login localhost:50004 admin netwitness -c cd sdk -c import {} -c logout \;

You will need to replace / with the correct starting path as the above example will walk through the entire file system not just a specific directory and will upload every file ending in .pcap to your decoder.

Hope this helps!

Regards,

Rui

[huanzhou1]

To me last time i used tcpreplay to replay the pcap files inline while the decoder is capturing.  You can download and install tcpreplay on the decoder then upload the pcap files there.