Help with Custom Parser

[MyronEstibeiro]

Hi, I'm trying to create a custom parser for an appliance that generates log messages in JSON format. Can anyone please  help me point at any resource that can help?

I have created custom parsers earlier that generated txt files, but I'm having difficulties in parsing the JSON text.

[ArthurCostigan1]

Myron

JSON, while being arguably an excellent logging format for applications, it logs in a multi line format.  Netwitness for Logs can only parse single line logs. So in order to accurately parse JSON format, you will need to pre-process the logs into a single line format.  This would involve looking for "braced" ({ }) lines and other delimiters and concatenating multiple entries into a single line.  

[MyronEstibeiro]

Thank you for the insight Oj2tqPhEV7pi5mR8EY3PQXHtgPuI9Vk2SNDA9WW6hKQ=‌. I appreciate  it. Do you recommend to use LUA for the parsing or ESI?

If you could you please provide any existing parsers that process JSON file, it would be of tremendous help. Could you please help?

[MyronEstibeiro]

I finally managed to create  a parser to process json based logs through ESI tool; however, I'm looking forward in developing all future parsers via LUA. Could someone please guide me to the resources available for LUA parsing development?

[DavidWaugh1]

The best way I found of learning LUA is by looking at some existing parsers and then modifying them.

There are some examples in the community is you search for LUA.

I'm very interested how you parsed your JSON logs. Would it be possible to paste some of your sample logs, you script to pre process the logs into single lines and your parser?

The think to remember in LUA is that each of our tokens can only contains 256 characters, so you will still need to split your logs into meta with an XML parser before you process them with a LUA parser.

[ArthurCostigan1]

Myron, this is a good starting point...The Programming Language Lua  and  lua-users.org