With this last post on customizing the platform I want to recall a capability already mentioned a few times in the discussions here but never emphasized enough.
As already highlighted in my previous post, log parsers (e.g. Envision parsers) are not the only mechanism available within Security Analytics for generating meta based on a log message. Flex and LUA parsers can also be used for both analyzing the raw syslog stream or for just post-processing the meta created by a log parser with the purpose of generating new piece of meta.
The ability to apply Flex and LUA parsers to the meta already generated by a log parser can be significant for many reasons:
- In some cases writing a Flex/LUA parser is easier than customizing an Envision parser (especially if the latter is very big and/or cannot be handled easily with ESI);
- Flex/LUA parsers provide more powerful operators, not all available in a log parser;
- If the same logic has to be applied for all the log parsers is of course quicker to write a single Flex/LUA parser instead of customizing them all (e.g. if we need to post-process a specific meta regardless of the parser which generated it);
A typical example is to split a URL identified by all the proxy parsers in domain, tld, directory, page, extension. Applying the logic in all the log parsers generating URLs may be possible but does not scale very well. A single Flex/LUA parser can instead do the job easily and effectively.
A few examples (some of those already shared here by @HJ_Lorentzon) are attached to this post.
To deploy the parser, upload the file to the /etc/netwitness/ng/parsers directory and reload the parsers from the Explore view (/decoder/parsers reload).
