How to build logic and capture specific data from 2 types of logs?

[UtsavSejpal]

Hi Experts,

We've integrated Trend micro webproxy device with RSA SA Netwintess (10.6.2.0).

It forwards the same log under two different types. For e.g. if user A accessed website B then 2 different logs are generated with the exact time stamp. 1)Event URL Access Tracking 2)Event URL monitoring

Below are the samples:

1) EVT_URL_ACCESS_TRACKING

2) EVT_URL_MONITORING|LOG_CRIT

EVT_URL_Access log can be of allowed/denied while EVT_URL_Monitoring is the log of allowed website.

So, we wanted to achieve that if user name, time and URL (with contains and ot exact) matches then capture bytes, source and destination from EVT_URL_ACCESS_TRACKING

Match Condition:

match-user-exact

match-time-exact

URL with domain contains

Action:

Then write bytes, source, destination to Report/Chart etc which will be available from URL Access tracking logs.

Thanks

Utsav Sejpal

[JoshRandall · Accepted solution]

Hello Utsav,

Your workflow to do this will have several steps.

First, you will need an ESA advanced rule to match on the characteristics you've described.

Second, you will need an ESA Syslog output template that contains the specific values (at least) you want to report on.

Third, you will need to send the syslog alerts from the ESA to your NetWitness Log Decoder.

Fourth, you will need to create your Reporting Engine rule, chart, and report.

So then, first the ESA rule.  I don't believe it's a good idea to try to match on the exact same time between the 2 logs, because with all the variability in when logs are generated, consumed, parsed, etc. those might not always match and the ESA rule would fail to alert against them.  Instead, I believe creating a rule looking for each event followed by the other would allow you to match against all the characteristics within each that you want:

      [event A

      ->

      event B];

      [event B

      ->

      event A];

  I don't have matching logs in my lab to verify that this rule will work exactly as I think it should in your environment, but I believe it should be pretty close:

@RSAAlert(oneInSeconds=0)

SELECT * FROM PATTERN [
every-distinct(username , ip_src , ip_dst, 5 Minutes)
/* Statement: url monitor */
e1=Event(
device_type.toLowerCase() IN ( 'trendmicro' )
AND
msg_id.toLowerCase() IN ( 'evt_url_monitoring|log_crit' )
AND
ip_dst IS NOT NULL
AND
ip_src IS NOT NULL
AND
username IS NOT NULL
AND
url IS NOT NULL
)
->
/* Statement: url access */
e2=Event(
device_type.toLowerCase() IN ( 'trendmicro' )
AND
msg_id.toLowerCase() IN ( 'evt_url_access_tracking|log_info' )
AND
username=e1.username
AND
ip_src=e1.ip_src
AND
ip_dst=e1.ip_dst
AND
url=e1.url
)
where timer:within(5 Minutes)
];

@RSAAlert(oneInSeconds=0)

SELECT * FROM PATTERN [
every-distinct(username , ip_src , ip_dst, 5 Minutes)
/* Statement: url access */
e1=Event(
device_type.toLowerCase() IN ( 'trendmicro' )
AND
msg_id.toLowerCase() IN ( 'evt_url_access_tracking|log_info' )
AND
ip_dst IS NOT NULL
AND
ip_src IS NOT NULL
AND
username IS NOT NULL
AND
url IS NOT NULL
)
->
/* Statement: url monitor */
e2=Event(
device_type.toLowerCase() IN ( 'trendmicro' )
AND
msg_id.toLowerCase() IN ( 'evt_url_monitoring|log_crit' )
AND
username=e1.username
AND
ip_src=e1.ip_src
AND
ip_dst=e1.ip_dst
AND
url=e1.url
)
where timer:within(5 Minutes)
];

Second, the ESA syslog output template.  In order to be able to create RE rules, charts, and reports against something, NetWitness has to consume the data through a decoder so that it can parse it and create meta.  The available OOTB syslog templates won't contain the values you want to report against, and so NetWitness wouldn't create the meta you want.  Instead, we want to create a new syslog template that will contain these values.  The template would look something like this:

CEF:0|RSA|NetWitness ESA|10.6.2|${statement}|${moduleName}|${severity}|rt=${time?datetime} id=${id} source=${eventSourceId} <#list events as x> src=${x.ip_src!" "} dst=${x.ip_dst!" "} size=${x.bytes!" "} duser=${x.user_dst!" "} </#list>

If the username value within your logs is writing into a different metakey, then you would want to change that within the template to what is appropriate; a couple examples:

suser=${x.user_src!" "}

username=${x.username!" "}

Third, you will need to add your Log Decoder as a syslog server within Admin --> System --> Global Notifications (SA Cfg: Configure Global Notification Templates) and then choose your syslog template, notification server, and notification type within the ESA rule:

Fourth, once you have verified that the alert is firing how you want it, that the alert is getting sent to your Log Decoder, and that the meta you want is being created (bytes, ip.src, ip.dst), you can create your Reporting Engine rule.  The rule could look something like this:

I hope this helps.  Let us know if you run into any issues with this.