Configure an In-Memory Table Using an EPL QueryConfigure an In-Memory Table Using an EPL Query
Note: It is preferable to use Context Hub List enrichment sources instead of In-Memory Table enrichment sources for rules. Recurring In-Memory Tables are no longer supported; use Content Hub Lists as enrichment sources. For more information, see Configure a Context Hub List as an Enrichment Source.
When you use an In-Memory Table configuration in expert mode, you can create an enrichment source or named window based on an Esper query. This allows you to have more control over the content and create more dynamic content. When you do this, an EPL query constructs the named window to capture interesting states from the event stream.
WorkflowWorkflow
The following shows the workflow for creating a query using a named window:
- The event is sent to the Esper Engine.
- An EPL query is generated.
- An alert is triggered.
- The query checks to see if there is a connection between the event and the Named Window.
- If there is a connection, the query that populates the Named Window is run and populated.
- The content from the Named Window is added to the alert content and sent or displayed (depending on your settings).
PrerequisitesPrerequisites
- The meta used in the EPL statement must exist in the data.
- You must create well-formed EPL statements.
ProcedureProcedure
Note: It is preferable to use Context Hub List enrichment sources instead of In-Memory Table enrichment sources for rules.
- Go to (Configure) > ESA Rules.
The Configure view is displayed with the Rules tab open. - Click the Settings tab.
- In the options panel, select Enrichment Sources.
- In the Enrichment Sources section, click > In-Memory Table.
- Select Adhoc.
By default, Enable is selected. When you add the in-memory table to a rule, alerts will be enriched with data from it. - In the User-Defined Table Name field, type a descriptive name to describe the in-memory table.
- If you want to explain what the enrichment adds to an alert, enter information in the Description field.
This description displays when you view the list of enrichments from the Enrichment Sources view, so it's a good idea to enter a thorough description as a best practice. Doing this allows other users to understand the content of the enrichment without opening it to examine its contents. - Select Expert Mode to define an advanced in-memory table configuration by writing an EPL query.
The Table Columns are replaced by a Query field. - Select Persist to preserve the in-memory table on disk when the ESA service stops and to re-populate the table when the service restarts.
- Enter the EPL query in the Query field. The query should be well-formed, and it's a good idea to test it before entering it in the field.
- Click Save.
ExampleExample
For example, you want to know when an IPS or IDS is giving five or more inbound events with an event identified with malicious code. Additionally, you would like to know when the source IP of those events has been identified as suspicious by other sources. This information helps to more quickly triage the event and determine whether the alert is a true positive.
Step 1: Create the EnrichmentStep 1: Create the Enrichment
In this example, this enrichment is a watchlist of IPs that have been identified as suspicious by third party sources or by internal staff. The meta of threat_desc equal to ‘suspicious ip’ is generated when a match to a feed occurs. This meta can be matched and output based on a log, packet, or endpoint event.
The enrichment should look like the following:
Parameters | Description |
---|---|
Upload Type | Adhoc |
IP_Watchlist | IP_Watchlist |
Description | Dynamically populated whitelist based on a feed of IPs that are considered suspicious. |
Expert Mode | Selected |
Query |
create window IpWatchlist .std:unique(ip_src) as (ip_src string, threat_source string, threat_category string); insert into IpWatchlist select ip_src, threat_source, threat_category from Event where threat_desc = 'suspicious ip'; |
Step 2: Create Your RuleStep 2: Create Your Rule
First, you need to create your ESA Correlation rule. This example rule looks for inbound IPS or IDS log events with the event_cat_name beginning with Attacks.Malicious Code. If five or more events for the same ip_src occur within 60 minutes, then an alert will be triggered. If an ip_src from the Enrichment equals the ip_src from the alert, then that alert will be enriched with additional meta. In this case, the analyst would see the values for threat_source and threat_category in the raw alert. Threat_category would indicate the type of malware and threat_source would indicate the entity that has reported the ip as suspicious. The analyst could use this information to do additional research or escalate to the next tier for creation of a possible incident.
Rule StatementRule Statement
Rule Logic with Enrichment AddedRule Logic with Enrichment Added