ESA AnnotationsESA Annotations
This topic describes annotations that NetWitness provides to use in advanced EPL rules.
For best practices on writing advanced EPL rules, see ESA Rule Writing Best Practices.
@RSAContext Annotation (11.5 and later)@RSAContext Annotation (11.5 and later)
The @RSAContext annotation can be used in advanced rules to dynamically add or remove data from a Context Hub list after the rule fires. For example, you can create a rule that automatically adds an IP address to a blacklist and removes it from a whitelist.
You can update a single-column or a multi-column Context Hub list. The @RSAContext annotation also performs error handling when the Context Hub list cannot be reached.
PrerequisitesPrerequisites
Before deploying a rule using the @RSAContext annotation, the list must exist in Context Hub. For information on creating a Context Hub list, see the Context Hub Configuration Guide. Go to the NetWitness All Versions Documents page and find NetWitness Platform guides to troubleshoot issues.
IMPORTANT: If you rename a Context Hub list or recreate the Context Hub list with the same name, update the ESA rules that use that Context Hub list, and then redeploy the ESA rule deployments that contain those rules.
Single-Column Context Hub ListsSingle-Column Context Hub Lists
The @RSAContext annotation uses the following format for a single column Context Hub list:
@RSAContext (list="<single_column_list>", action=<DELETE_ENTRY or ADD_ENTRY>, onError = <STOP_ALL_RULE_PROCESSING_AND_WAIT or IGNORE_ERROR_AND_CONTINUE>, fields={"LIST=<meta_key>"})
The @RSA Context parameters for a single-column Context Hub List are described in the following table.
|
Parameter |
Description |
|---|---|
| list | Where <single_column_list> is the name of the single-column Context Hub list (whitelist or blacklist). |
| action |
You can add an entry to or remove an entry from the list:
You can only have one action per @RSAContext entry. |
| onError |
Identifies how to handle errors, for example, when the Context Hub list is not available or is full. You can choose one of the following options:
|
| fields | Maps a meta key value to the column in the Context Hub list. Single-column lists always show LIST= in capital letters. |
The first example deletes a source IP address from an IP whitelist. However, if the list is not available, the source IP address is not removed from the list and processing is continued.
@RSAContext (list="ip_whitelist", action=DELETE_ENTRY, onError = IGNORE_ERROR_AND_CONTINUE, fields={"LIST=ip_src"})
The second example adds a source IP address to an IP blacklist. However, if the list is not available, all rule processing stops including data aggregation, and the ESA Correlation service retries indefinitely until the entry is added to the specified Context Hub list.
@RSAContext (list="ip_blacklist", action=ADD_ENTRY, onError = STOP_ALL_RULE_PROCESSING_AND_WAIT, fields={"LIST=ip_src"})
Multi-Column Context Hub ListsMulti-Column Context Hub Lists
The @RSAContext annotation uses the following format for a multi-column Context Hub list:
@RSAContext (list="<multi_column_list>", action=<DELETE_ENTRY or ADD_ENTRY>, onError = <STOP_ALL_RULE_PROCESSING_AND_WAIT or IGNORE_ERROR_AND_CONTINUE>, fields={"<column1>=<meta_key1 or constant value1>",<column2>=<meta_key2 or constant value2>",<column n>=<meta_key n or constant value n>",})
The @RSA Context parameters for a multi-column Context Hub List are described in the following table.
|
Parameter |
Description |
|---|---|
| list | Where <multi_column_list> is the name of the multi-column Context Hub list. |
| action |
You can add an entry to or remove an entry from the list:
You can only have one action per @RSAContext entry. |
| onError |
Identifies how to handle errors, for example, when the Context Hub list is not available or is full. You can choose one of the following options:
|
| fields | Maps a meta key values or constant values to the columns in the Context Hub list. Multi-column lists show the names of each column in the fields parameter.* |
The first example deletes a user and the associated source and destination IP addresses from a multi-column IP whitelist. However, if the list is not available, the specified fields are not removed from the list and processing is continued.
@RSAContext (list="MultColumn_whitelist", action=DELETE_ENTRY, onError = IGNORE_ERROR_AND_CONTINUE, fields={"source=ip_src","destination=ip_dst","user='smith'"})
The second example adds a user and the associated source and destination IP addresses to a multi-column IP blacklist. However, if the list is not available, all rule processing stops including data aggregation, and the ESA Correlation service retries indefinitely until the entry is added to the specified Context Hub list.
@RSAContext (list="MultColumn_blacklist", action=ADD_ENTRY, onError = STOP_ALL_RULE_PROCESSING_AND_WAIT, fields={"source=ip_src","destination=ip_dst","user='smith'"})
Automatic Context Hub List UpdatesAutomatic Context Hub List Updates
When using the @RSAContext annotation in your rules, string array meta keys (shown as string[] in the Meta Key References on the ESA Rules > Settings tab) are inserted and removed as separate lines in the Context Hub list. This enables lookups in the Investigate and Respond views.
Single Column Context Hub List Update Example
A single column Context Hub list has one column with the string array meta key alias_host. If the rule using that list fires and alias_host has three values (Google, Yahoo, and RSA), ESA Correlation adds three rows to the Context Hub list:
Google
Yahoo
RSA
Multi-Column Context Hub List Update Example
A multi-column Context Hub list has three columns. If the rule using that list fires and the meta keys have the following values:
- username (string array) with values Aimee and Chris
- ip_dst (string) with value 10.10.10.10
- alias_host (string array) with values RSA, Google, and Yahoo
ESA Correlation adds six rows to the Context Hub list:
Aimee, 10.10.10.10, RSA
Aimee, 10.10.10.10, Google
Aimee, 10.10.10.10, Yahoo
Chris, 10.10.10.10, RSA
Chris, 10.10.10.10, Google
Chris, 10.10.10.10, Yahoo
@RSAAlert Annotation@RSAAlert Annotation
The @RSAAlert annotation is used to mark which EPL statements are linked to generating alert notifications. It is designed to work with the alert notification suppression feature in the Rule Builder user interface.
The @RSAAlert annotation can be useful when working with alert notifications, especially if you want to filter notifications, such as sending one notification for each user that triggers an alert.
For example, suppose you want to generate alert notifications for login failures. You could add the following statement:
@RSAAlert select * from event(msg_id=“login_fail")
| Event number | Message ID | username | src_IP | Time |
|---|---|---|---|---|
| 1 | login_fail | alice | 1.2.3.4 | 10:00 |
| 2 | login_fail | alice | 1.2.3.4 | 10:01 |
| 3 | login_fail | alice | 6.7.8.9 | 10:01 |
| 4 | login_fail | bob | 1.2.3.4 | 10:01 |
| 5 | login_fail | alice | 1.2.3.4 | 10:03 |
For the above statement, five alert notifications are generated.
However, suppose you wanted to modify the statement to generate one alert for each separate username. You can use the identifier attribute. For example, the statement @RSAAlert(identifier={”username”}) SELECT* FROM Event(msg_id=”login_fail”) generates one notification for the first alert for “bob” and one for the first alert for “alice." Subsequent alerts for “bob” and “alice” are ignored.
You can further distinguish the users by adding details via the identifier variable. For example, you can distinguish by user and IP address using the following statement: @RSAAlert(identifier={”username”, "src_ip"}) SELECT* FROM Event(msg_id=”login_fail”). Then, you would see notifications generated by user name and IP address (one alert for "alice" at 1.2.3.4, another alert for "alice" at 6.7.8.9, and an alert for "bob" at 1.2.3.4).
To use identifiers with Alert Notification Suppression:
The @RSAAlert annotation is designed to work with the alert notification suppression feature in the Rule Builder user interface. To do this:
- Create a rule in the Rule Builder user interface, and select the alert suppression feature when configuring notifications.
-
Copy the code from the Rule Builder rule into a new advanced rule.
-
Configure the advanced rule to include identifiers (as described above) and save the advanced rule.
-
Delete the original rule builder rule.
@RSAPersist Annotation@RSAPersist Annotation
The @RSAPersist annotation is used to mark a named window as an ESA managed window for persistence. By marking the named window as an ESA managed window, ESA periodically writes the contents of the window to disk and restores them back if the window is undeployed and redeployed. The systems take a snapshot just before the ESA rule deployment is undeployed and the window is removed. Conversely, it restores the window contents from the snapshot just after the deployment is redeployed. This ensures that the contents of the window are not lost if the deployment state is altered or if the ESA service goes down.
For example, consider a named window, DHCPTracker that holds a mapping from IP addresses to each assigned hostname.You can annotate the statement with the @RSAPersist annotation as:
@RSAPersist
create window DHCPTracker.std:unique(ip_src) as (ip_src string, alias_host string);
insert into DHCPTracker select IP as ip_src, HostName as alias_host from DHCPAssignment(ID=32);
Note: All windows definitions are not suitable for persistence. @RSAPersist annotation must be used with care. If the window has timed-records or if it depends on time based constraints it is very likely that the reverted snapshots will not restore it to the correct state. Also, any changes to the window definition will invalidate the snapshots and reset the window to a blank state. The system does not do any semantic analysis to determine if the changes to the window definition are conflicting or not. Note that other parts of a deployment (that is, other than the particular CREATE WINDOW call that defines the window) may change, without invalidating the snapshots.
Caution: (This caution applies only to NetWitness Platform versions 11.3.x, 11.4.x, and 11.5.0.0.) To avoid data being overwritten, if you have a rule with a named window, do not disable and re-enable it. Instead, undeploy and redeploy the ESA rule deployment that contains the rule.
@UsesEnrichment (10.6.1.1 and later)@UsesEnrichment (10.6.1.1 and later)
The @UsesEnrichment can be used in advanced EPL rules to reference enrichments. In order to synchronize enrichments with ESA, all enrichment dependencies in EPL rules must be referenced with the @UsesEnrichment annotation.
The @UsesEnrichment annotation uses the following format:
@UsesEnrichment(name= '<enrichment_name>')
For example, the following EPL references a whitelist enrichment:
@UsesEnrichment(name = 'Whitelist')
@RSAAlert
SELECT * FROM Event(ip_src NOT IN (SELECT ip_address FROM Whitelist))
@Name@Name
The @Name is the statement name defined in ESA advanced rules. It is used to dynamically generate statement names in ESA alerts. The statement name of only an alert triggering statement is displayed. This annotation has meta keys enclosed in curly brackets.
The @Name annotation uses the following format:
@Name("<static_part_of_statement_name> {meta_key1} {meta_key2}…")
For example, the following EPL references meta keys ip_src and user_name whose values will be dynamically generated.
@Name(“Login Event to {ip_src} by {user_name}”)
Note: You can specify any number of meta keys in the statement for dynamic statement name generation.
The length of individual meta key is limited to 64, after which the value is truncated and appended with “…”.
The length of the dynamic generation of statement name is limited to 128, after which the value is truncated to 128 and appended with “…”. All the remaining values post truncation will be treated as static values.
@Audit Annotation@Audit Annotation
Add the @Audit('stream') annotation to your ESA rules to print alerts to the ESA logs for troubleshooting. This is useful when debugging the Esper rules. The @Audit('stream') annotation provides debug information for the next statement in the rule. For example:
@Audit('stream')
@RSAAlert
SELECT * FROM Event((ip_src NOT IN ( '1.1.1.1' , '2.2.2.2' )));
In the above example, @RSAAlert is only necessary if the statement needs to send an alert to the Respond view.
In NetWitness Platform 11.5 and later, you can test rules in the rule builders. If you add the @Audit('stream') annotation to an advanced EPL rule, you can view the Debug Log in the test output. For more information, see Validate an Advanced EPL Rule .
