How to alert based on multiple meta keys?

[UmaMaheshPadise]

Hi,

Scenerio:

A log is parsed into below meta keys

   aba = 'xxxx'

   dda = 'yyyy'

   .....

I have blacklist of aba,dda pair

      blacklisted_account,aba1,dda1

      blacklisted_account,aba2,dda2

I need to alert if any log has aba=aba1 and dda=dda1

                         OR               aba=aba2 and dda=dda2

I should not alert when aba=aba1 and dda=dda2

-----------
One solution i have in mind is as below which i am not sure works.

To create custom meta key which is created while log is parsed by SA

          abadda = aba||'!'||dda

Feed is    

      tags,abadda

       blacklisted_account,aba1!dda1
       blacklisted_account,aba2!dda2

ESPER Alert

      'blacklisted_account'=ANY(tags)              

-------------

Hence i am looking for a way to create abadda (a custom meta key carved out of existing metakeys)

Or

Alternative way to accomplish above alerting.

Thanks,

Uma Mahesh | 7068402149

ADP

[Anonymous]

Would an app rule work in this case?

name=blacklisted_account    rule=  aba = 'aba1' && dda = 'dda1'

This assumes that both 'aba' and 'dda' were meta keys.

[UmaMaheshPadise]

Hi Christopher,

I have 10k+ blacklisted (aba,dda) pairs and growing.

So they have to go to a feed that has to be matched against meta key pair aba,dda in the log for the alert to fire.
hence manual addition in app rule doesn't work.

[Anonymous]

In that case,  you could look at a multi-indexed feed.  This is a feed that would match against multiple meta-keys before writing meta.  In a sense, it is like the app rule I wrote above...just a lot bigger.

Playing with this in my lab, I used the following:

===================multiblacklist.xml==================

<?xml version="1.0" encoding="utf-8"?>
   <FDF xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="feed-definitions.xsd">

      <FlatFileFeed name="multiblacklist"
      path="multiblacklist.csv"
      separator=","
      comment="#">

      <MetaCallback name="DeviceSource">
         <Meta name="nethost.src" valuetype="Text" ignorecase="true"/>
      </MetaCallback>

      <MetaCallback name="RootHost">
         <Meta name="root.host" valuetype="Text"/>
      </MetaCallback>

      <LanguageKeys>
         <LanguageKey name="alert" valuetype="Text"/>
      </LanguageKeys>

      <Fields>
         <Field index="1" type="index" key="DeviceSource"/>
         <Field index="2" type="index" key="RootHost"/>
         <Field index="3" type="value" key="alert" />
      </Fields>
   </FlatFileFeed>

</FDF>

===================multiblacklist.csv==================

#DeviceSource,RootHost,alert
iPhone6-Chris,google.com,blacklist_iphone6_google
MacbookPro-Chris,shodan.io,blacklist_mbp_shodan
iPadMini-Kailynn,googlevideo.com,blacklist_kk_youtube
iPadMini-Kailynn,youtube.com,blacklist_kk_youtube

This produced the following alert meta:

It sounds like this is a straight forward key/value pairing which is all the feed was told to do.  It is also possible to have multiple keys be checked for matches but it doesn't sound like that is needed for this specific use case.

In your use case, column 1 might be the value in 'aba'.  Column 2 might be the value in 'dda'.  Column 3 would be the meta you wish to write such as 'blacklisted_account' or something similar.

Chris

[UmaMaheshPadise]

oh wow.

the UI doesn't let me have multi index feed so i assumed it is not possible.

just realized editing xml will work to have multi index feed.

I will try and let you know.

Thanks

Have a good weekend

[Anonymous]

Correct...and I should have mentioned that.  This is not something that can be done within the Feed Wizard in the UI.  I had edited the XML by hand and deployed the feed manually.  It does appear we have that on the list of feature enhancements though.

I haven't tried deploying the compiled feed with the wizard but can try that out.

[Anonymous]

Yeah..feed wizard said it was not a valid feed xml.  I wound up compiling the feed on the decoder manually.

To do so, I ran the following command from the console in the same directory as my XML and CSV files:

NwConsole -c feed create multiblacklist.xml

Make sure you have 0 or very few invalid records.  

Then, I copied the multiblacklist.feed file into the /etc/netwitness/ng/feeds directory and issued a reload of the feeds.

To check your work, you can review /var/log/messages and look for the feed being loaded.

Chris

[UmaMaheshPadise]

me too got Invalid XML Error.
NwConsole   - is this a executable on SA Appliance? (so far i only have access to UI based admin view)

<?xml version="1.0" encoding="UTF-8"?>
<FDF>
<FlatFileFeed comment="#" separator="," path="blacklistabadda.csv" name="blacklistabadda">
<MetaCallback valuetype="Text" name="aba" ignorecase="true"/>
<MetaCallback valuetype="Text" name="dda" ignorecase="true"/>
<LanguageKeys>
<LanguageKey valuetype="Text" name="alert_tags"/>
</LanguageKeys>
<Fields>
<Field type="index" index="1" key="aba"/>
<Field type="index" index="2" key="dda"/>
<Field key="alert_tags" type="value" index="3"/>
</Fields>
</FlatFileFeed>
</FDF>

[Anonymous]

Yes.

Sent from my mobile device

[UmaMaheshPadise]

Hi Chris,
In The Feed create above, how to make the feed recurrent?