Event Processing Language (EPL)

This topic describes Event Processing Language (EPL), a declarative language for dealing with high frequency time-based event data. ESA uses Event Processing Language (EPL), a declarative language for dealing with high frequency time-based event data. It is used for express filtering, aggregation, and joins over possibly sliding windows of multiple event streams. EPL also includes pattern semantics to express complex temporal causality among events. It can perform, but is not limited to, the following functions:

  • Filter Event
  • Alert Suppression
  • Compute percentages or rations
  • Average, count, min and max for a given time window
  • Correlate events arriving in multiple stream
  • Correlate events that arrive out of order
  • On-Off Windows
  • Followed-by and Not Followed-by support
  • Regex filter support

Databases require explicit querying to return meaningful data and are not suited to push data as it changes. The developer must implement the temporal and aggregation logic himself. By contrast, the EPL engine provides a higher abstraction and intelligence and can be thought of as a database turned upside-down. Instead of storing the data and running queries against stored data, EPL allows applications to store queries and continuously run the data through. Response from the EPL engine is real-time when conditions occur that match user defined queries.

For best practices on writing advanced EPL rules, see ESA Rule Writing Best Practices.

Advanced ESA rules require correct character case, but in the Investigate Navigate view all characters are converted to lowercase. However, the meta may not be lowercase despite appearances in the Investigate Navigate view. To ensure you are using the correct case, you can make a strict pattern match for better performance.

Copy

Strict Pattern Match Example

@RSAAlert(oneInSeconds=0)
SELECT * FROM Event(
     (medium IN ( 1 ) AND
     filetype IN ( 'pdf' , 'windows_executable' , 'x86 pe' , 'windows executable' ))
     ).win:time(5 Minutes)
MATCH_RECOGNIZE (
     MEASURES E1 as e1_data , E2 as e2_data
     PATTERN (E1 E2)
     DEFINE
          E1 as (E1.filetype IN ('pdf')),
          E2 as (E2.filetype IN ( 'pdf' , 'windows_executable' , 'x86 pe' , 'windows executable' ))
);

Caution: Care should be taken to only add the case-insensitive toLowerCase() function on meta keys as needed. The toLowerCase() function can cause significant performance decreases. Consider checking the Investigate Events view or the Event Analysis view to see the real character case for meta fields and avoid unnecessary usage of the function.

For the purposes of online help, basic statements are used to illustrate how to set up ESA; however, for more information about writing EPL statements, the http://www.espertech.com site provides tutorials and examples.

Note: In NetWitness version 11.5, ESA Correlation supports Esper version 8.4.0. In version 11.4, ESA Correlation supports Esper version 8.2.0 and in version 11.3, ESA Correlation supports Esper version 7.1.0.