I have written a LUA parser that will detect if you are falling behind with your Check Point log collection.
Prerequisites are:
Change the header number 02 in the Check Point parser. (I have attached the modified checkpoint parser to this case)
<HEADER
id1="0002"
id2="0002"
devts="MDYTS(hmonth,hday,hyear,htime)"
content="<hmonth> <hday> <hyear> <htime> %CHKPNT-<hlevel>-<messageid>: <!payload>" />
I basically change month, day,year and time in this header to hmonth, hday,hyear and htime.
The reason for this is that I need them in text.
Add the following to you table-map-custom.xml file
<!- BEGIN Check Point Date Time Header Fields -->
<mapping envisionName="hyear" nwName="hyear" flags="None" format="Text"/>
<mapping envisionName="hmonth" nwName="hmonth" flags="None" format="Text"/>
<mapping envisionName="hday" nwName="hday" flags="None" format="Text"/>
<mapping envisionName="htime" nwName="htime" flags="None" format="Text"/>
<mapping envisionName="fld49" nwName="fld49" flags="None" format="Text" />
<!- END Check Point Date Time Header Fields -->
Add the following to your index-concentrator-custom.xml files
<key description="Seconds Delay" level="IndexValues" name="seconds.delay" format="UInt32" defaultAction="Open" valueMax="10000" />
<key description="Event Time String" level="IndexValues" name="event.time.str" format="Text" valueMax="2500000"/>
Set up your timestamp feed as detailed in the post below.
A typical checkpoint message might look something like this
May 17 2016 08:59:47: %CHKPNT-6-060020: accept,192.168.202.243,inbound,eth1,192.168.200.200,45314,54.152.120.25,443,https,tcp,8,7373,0:00:01, , , , , , , , , , , , , , ,6039,1334, , , , , ,17May2016 8:59:47,1,VPN-1 & FireWall-1, , , , , , , , , , ,17May2016 8:59:26,17May2016 8:59:27,20,9,11,11,9,1334,6039,eth1,eth0,eth0,eth1,0,0,0, , , , , ,060020, , , , , , , , , , , , , , , ,{F73441C3-A91E-4603-914C-890BBBCFB32A}, , , , ,
We compare the difference between the time in the message and the time in the header field. These are highlighted in red.
If the difference is greater than 5 seconds we create an alert "WARNING Checkpoint: Processing Delayed"
If the difference is greater than 10 seconds we create an alert "CRITICAL Checkpoint: Processing Delayed"
These are configurable in the lua parser and are probably too sensitive for a production environement.
The amount of delay is written into the new meta key "seconds.delay"
The parser should be placed in /etc/netwitness/ng/parsers on your logdecoder.
Notes:
- I only look at certain checkpoint message ID's.
- Comment the nw.logInfo lines with -- in front of them to disable debugging.
-If your header and log times are in different timezones - you will need to add or subtract multiple of 3600 seconds to the HeaderEpochTime or EventEpochTime variables so that you are comparing times in the same timezones.
NOTE: Updated Version 2 of Parser attached below:
One issue with this is that it compares the Header Date with the log Date. This is fine to detect the if there is a delay with collecting the logs, but it wont detect if the delay is with us processing the logs at the log decoder.
Unfortunately, I would really like to use the event.time meta in my LUA parser. This is however a 64 bit integer and not supported by the 32 bit version of LUA that is currently on the Security Analytics decoders.
To get around this, on the Security Analytics SA Server, I create a timestamp feed.
A cron job that runs every minute has the following:
vi /etc/cron.minute/timestamp.sh
#!/bin/bash
# Script to write a timestamp in a feed file
for i in {1..60}
do
echo checkpointfw1,$(date -u) >/var/netwitness/srv/www/feeds/timestamp.csv
sleep 1
done
I then use the output csv file as a feed that updates every minute.
In my CheckPoint events I will now have new meta called event.time.str that will be the (rough) timestamp of when the event reached the Log Decoder. Potentially this timestamp might be out by a minute or more, but is probably good enough for our situation.
In the screenshot above
event.time is 2016-05-17 14:54:12.000
event.time.str is Tue May 17 14:56:02 UTC 2016
which is good enough for our purposes.
Under the Track Column in Check Point Smart Dashboard "Account" rather than "log" will be listed.
