Event Time Function Usage

[JayShah]

Event Time Function - *EVNTTIME()

It assigns the date and time information present in the event/log to a message variable to normalize the output in a consistent TimeT format. It is part of the function tag in a message defintion.

Example in parser message: In the example we are collecting starttime and recorded_time variable.

<MESSAGE
id1="mcafeemc"
id2="mcafeemc"
eventcategory="1612000000"
functions="&lt;@msg:*PARMVAL($MSG)&gt;&lt;@starttime:*EVNTTIME($MSG,'%B %F %W %H:%T:%S.%V %L',param_starttime)&gt;&lt;@recorded_time:*EVNTTIME($MSG,'%W-%M-%DT%H:%T:%S.%V%E',param_time)&gt;"
content="&lt;param_starttime&gt;&lt;param_time&gt;&lt;msghold&gt;" />

Function definition:

<@resultant_name:*EVNTTIME([$MSG | $HDR],'Format1',Format2','FormatN',p_input1,p_input2,p_inputN)>

Multiple formats can be defined in the EVNTTIME function. Each format is processed from left to right till it finds a match.

Example:

<@event_time:*EVNTTIME($MSG,'%B %F %W %N:%U:%O','%B %F %Y %N:%U:%O','%B %F %N:%U:%O',fld20)>

As of 11.3, empty input parameters are ignored to allow for different formats in the same log without causing a parse error.

Example:

<@event_time:*EVNTTIME($HDR,'%F/%B/%W:%H:%U:%O','%W%B%F %H%U%O',hdatetime,hyear,hmonth,hday,hhour,hmin,hsec)>

Also as of 11.3, if multiple EVNTTIME() exist for a destination variable, no parse error is raised if at least one succeeds.

Format Specifiers

Character	Description	Notes
C	dates of this format: 04/20/05 14:01:57
R	Full Month Name, fixed width field: January, February, March, April, May, June, July, August, September, October, November, December
B	Abbreviated Month Name, fixed width field: Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec
M	Numeric Month, fixed width field: 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, 12
G	Numeric Month Variable width field: 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, 12
D	Numeric Month Day, fixed width field: 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, ..... , 23, 24, 25, 26, 27, 28, 29, 31
F	Numeric Month Day Variable width field: 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, ..... , 23, 24, 25, 26, 27, 28, 29, 31
H	Hour, fixed width field: 00-23
I	Hour, fixed width field: 01-12
N	Hour: Variable width field: 01-12 , 00-24
T	Minute, fixed width field: 00-59
U	Minute: Variable width field: 00-59
J	Julian day, fixed width field: 001-365
K	three letter day of week (case insensitive): MON, TUE, WED, THUR, THU, FRI, SAT, SUN
P	Alpha, fixed width field: AM or PM
Q	A.M./P.M.
S	Seconds, fixed width field: 00-59
O	Variable width field: Seconds: 00-60
Y	Year: 00-99
W	Year, fixed width field: 0000-9999
Z	Hours:Min:Sec
A	Days
X	Unix Time-Stamp (e.g.: 1424849941)	Mapped offsets or time zones are not applied and it is assumed that the generating clock is accurate.
L	Timezone Abbreviation: EST, UTC, GMT-5, etc...
E	UTC offset: Z, +/-HH:MM, or +/-HHMM	'Z' and 'z' added in 11.5 and 11.4.1
V	Fractional seconds: 0 - 999999999	Ignored in final value since the time is stored in seconds (added in 11.5 and 11.4.1)
%	The literal character '%'	Use "%%" for parsing character "%".

Populating Missing Year

If there is no year associated with a format string, the parser will attempt to populate the year heuristically.

In most cases, the value will be populated with the current year. There are however edge cases which are described below.

Parsing Logic

The current year is assigned as the message year (UTC, per the clock on the Log Decoder)....

Latent log during transition to new year

If the day resulting from assignment is more than 31 days into the future, the year is decremented.

Example:

The message date is 2017-Dec-31. The current date is 2018-Jan-1. The calculated date will be more than 31 days into the future 2018-Dec-31.

This will cause the year component to be decremented to 2017, resulting in an accurate message time.

Logs from forward time zones (or skewed clocks) just before new year transition

If the day resulting from assignment is more than 334 days into the past, the year is incremented.

Example:

The message date is 2018-Jan-1. The current date is 2017-Dec-31. The calculated date will be more than 334 days into the past 2017-Jan-1.

This will cause the year component to be incremented to 2018, resulting in an accurate message time.

Latent logs received where a leap-day cannot be successfully assigned to an appropriate leap-year

If the day (Feb-29) resulting from assignment is invalid (i.e. the assigned year is NOT a leap-year), the relative position to the current time cannot be calculated. To do this, the year is decremented in an attempt get a valid time-stamp.

Because the position of the leap-day relative to the transition to the new year, along with the expectation that no logs would be received this far into the future, we do not ever make an attempt to increment the year to produce a valid time-stamp.

After the year is decremented, the logic in bullets 1. and 2. is then followed. If the result is still an invalid day. The parser throws and a valid message time cannot be parsed.

Caveats

There is currently no mechanism to assign a year to the import of logs.

As this pertains to leap-days, if we hunted for the correct leap-year, inconsistent data would result. For example, if a set of messages is two years old and during a leap-year, we'd have a single day which was accurate, only attainable because of the limiting characteristics of that day. Because of the heuristic described above, the remaining set of messages would have time-stamps one year too new.

For the sake of data consistency, we do not hunt for valid leap years beyond the +1 month and -11 month window currently used for assignment.