Event Source Monitoring - Lua

[TomiReiman]

Hello,

The Security Analytics server generates syslog messages of the following kind if you setup event source monitoring and forward the notification via Syslog:

Oct 10 12:53:16 <sa-server> The Security Analytics systems have not received log events from the following devices within  <time>: Source Type=<device.type> Source Host=<device.ip> Device Name=<did>, Source Type=<device.type> Source Host=<device.ip> Device Name=<did>, ...

The list can continue on quite a bit per one log message if the amount of devices per that device type is large. I suspect you could circumvent this by creating a monitoring entry per a device.type and device.ip pair, but that would be too laborious in most of the cases. I could not find a way to parse the message with the regular, enVision-style XML parsers due to the variable length of the message. Is it somehow possible?

Also, I tried creating a LUA parser for this, but as I have zero experience with LUA before this, I could not get it working: it just isn't generating the meta I would like and to be quite honest I am not at all surprised. Attached here is the LUA parser I ended up with. The logic is quite crude also, but the main issue is that it does not work.

Has anyone got a working parser for these messages generated by the SA server? I did not find a message definition for it in the Security Analytics parser 2.0.

I would also like to know if there is any documentation for the functions provided by the architecture. The functions starting with "nw" in this file must be defined somewhere. For example, how could I check the device.ip in the session message, or any other parameters that could help me with checking if I should run through the LUA script in the first place: e.g. do not parse the payload if this message did not originate from the SA server.

[RSAAdmin]

Thanks for replies!

That's a nice example, I will look into it, in DML I would throw it all in one field.

I think of another usage - those ugly multiline text files with multiple different entry delimiters and variable field count - that is a real pain.

A little off topic, but for log files there is an interesting tag processorType with values: xml, tagvalmap, and oracle in filreader config, but no info on how to use them, so I just use generic with different delimiters (which is unusable for complex log files), do you happen to have info on those processor types? (I feel that usage of these processors can too resolve the ugly log file issues)

And another horrible thing I remembered is trouble with some encoding (ANSI, UCS) and no support for Cyrillic for delimiters, so if your field names are in Cyrillic you're out of luck (for any collection method). Are those problems relevant to LUA processing?