Last year, I was on an incident response engagement where we were investigating several drive-by attacks. Packet decoders were deployed and picking up the sessions rather easily but we were trying to identify the path of redirection that the malware compromises were taking. We had 'referer' meta but it was not indexed and would come in as a URL. While valuable, it was not something we would query against. However, I could extract some key elements from that 'referer' meta and help tell the story. And in case anyone is wondering why it's spelled 'referer' instead of 'referrer', please consult the Google.
The result was a Lua parser for extracting the PATH information from the referer meta. Essentially what it does is a meta-callback against the 'referer' meta key. A meta callback is simply a lookup of meta already created in the session. The parser then breaks out the key elements into individual meta keys.
Because I really only wanted the host that it came from, I created a custom key called 'referer.host' and indexed that. You could remove the comments around the other elements such as directory, filename, extension, etc but I did not find a lot of value in using those in an investigation.
The result helped tell the story and looked a little like this:
As I stated above, this was done on a packet decoder. However, you can run it on a log decoder too. Several web proxies will log the referer information in the log which would get parsed into the referer meta key. Since this parser is doing a meta callback, it can callback the meta from log sessions just as easily as it can from packets. The only difference is that log decoders need the nwll.lua (Netwitness Lua Library) file. By default, log decoders do not come with it. You can download it as a package from Live and then deploy it to your log decoders. The parser requires it for packet decoders too, but it is already there on a packet decoder.
Since this is going to use a custom meta key, you will want to add this to your index-concentrator-custom.xml on your concentrators.
<key description="Referer Host" level="IndexValues" name="referer.host" format="Text" valueMax="2500000" />
Edit that file and add the entry above. Then, save it, and restart the concentrator service to start indexing the meta.
I hope you find this useful to your investigations. Happy hunting.
Chris
