Defining Log Decoder Parse Rules

A single rule is defined by the following XML elements:

  • RULE (element, required) - Each rule is enclosed in a RULE element.

    • id (attribute, required) - A name associated with the rule.

    • stop (attribute, optional) - An attribute to stop the rule.

    • order (attribute, optional) - An attribute to order the rule.

  • LITERAL (child of RULE, requires one or more) - The literal defines the string inside each log message for which Log Decoder will search. Multiple literals can be defined.

    • value (attribute, required) = The string value for which to search.

  • PATTERN (child of RULE, requires exactly one) - A pattern is defined as either a regex or a format.

    • regex (attribute, optional) - A regular expression that can be used to extract one or more values from the matched string.

    • format (attribute, optional) - A built-in format type for which to scan and extract values (see Built-In Format below).

    • range (attribute, optional) - Determines the range to be searched for the pattern before and after the found LITERAL token. It supports the following values:

      Value Description
      after Applies regex after the anchor till the end of the log. This is the default value.
      before Applies regex from the beginning of the log till the starting position of the anchor.
      all Applies regex to entire log.

      before, after

      Applies regex to entire log.
      Negative Integer (-X) Applies regex to X or available characters before the found LITERAL token.

      Positive Integer (+X)

      Applies regex to X or available characters after the found LITERAL token.
      -X, +Y (Combination of Positive and Negative Integers) Applies regex to X or available characters before the found LITERAL token to Y or available characters after the found LITERAL token. - range=”-15,+12”

      -X, after (Combination of Negative Character, and After)

      Applies regex to X or available characters before the found LITERAL token to the end of the log. - range=”-15,after”

  • CAPTURE (child of PATTERN, optional) - A capture is used when extracting data from the log that was captured by the regular expression defined in PATTERN.

    • index (attribute, required) - The index of the regex match to capture.

    • key (attribute, required) - The meta key into which to assign the captured value.

    • format (attribute, optional) - The NwType of the key to be created (For example, Text, IPv4, UInt32 and so on). Default is Text.

  • META (child of RULE, optional) - A meta element defines meta values that get created when there is a match for the pattern.

    • key (attribute, required) - The meta key into which to store the value.

    • value (attribute, required) - The value to store.

    • format (attribute, optional) - The NwType of the key to be created (For example, Text, IPv4, UInt32 and so on). Default is Text.

Note: The format defined in the CAPTURE and META elements determines the meta type of the corresponding key. If this meta key is defined elsewhere in Log Decoder (in another parser, a language file and so on) and the types do not match, then an error will be generated and parse rule could potentially be disabled.

Built-In Formats

Log Decoder Parse Rules also offer several built-in formats for the more commonly used types. These formats are:

Format Description Example
IPv4 ipv4 192.168.1.1
IPv6 ipv6 2607:f0d0:1002:51::4
MAC physical Mac address 01:23:45:67:89:ab
UInt8 unsigned 8-bit integer 0 to 255
UInt16 unsigned 16-bit integer 0 to 65535

UInt32

unsigned 32-bit integer

0 to 4294967295
UInt64 unsigned 64-bit integer 0 to 18446744073709551615

Int8

signed 8-bit integer

-128 to 127
Int16 signed 16-bit integer -32768 to 32767

Int32

signed 32-bit integer

-2147483648 to 2147483647
Int64 signed 64-bit integer -9223372036854775808 to 9223372036854775807

Float32

decimal numbers

2.71818
Float64 decimal numbers 2.71818

Email

valid email address

bob@company.com
URI universal resource identifier http://www.google.com/path/script?query=param

Hostname

RFC-1123 compliant hostname

abc.xzy.com