Changes to be done in cef.xml to support unknown CEF logs

[AbhishekDwivedi]

Hi All,

I am forwarding my logs from ESA to hybrid in CEF format (using syslog) . I am extracting all available  information  from alerts. I have enabled cef.xml , but logs are going as rsa_securitanyaltics_esa. 

jun 29 07:27:46 localhost CEF:2.0|RSA|Security Analytics ESA|10.5.1.0|Module_7001_Alert--0|RSA_IGSOC_CitrixNetScaler_7001_Misuse_SSLVPN Connectivity Originated from same source by different Users|5|rt=2016-06-29T07:27Z id=7cd4c968-85f2-4cd1-9cd6-6fd7bb68c4f2 source=10.68.136.105:56005:484525335521 alert="rcf_test2" bytes_src="0" city_src="Hyderabad" country_dst="India" country_src="India" dclass_r1="0.00%" dclass_r1_str="Compression_ratio_send" dclass_r2="0.00%" dclass_r2_str="Compression_ratio_recv" device_class="Application Firewall" device_group="Citrix Netscalar" device_ip="10.68.100.134" device_type="citrixns" device_type_id="168" did="blrsiemdec1" disposition="Allowed" dtransaddr="10.67.252.6" dtransport="443" duration_str="00:00:00" endtime="1467185619" esa_time="1467185060412" event_byte_size="531" event_cat="1605000000" event_cat_name="System.Normal Conditions" event_desc="TCP connection related information for a connection belonging to a SSLVPN session" event_source_id="10.68.136.105:56005:484525335521" event_time="1467185619" global_alerting="citrixns.forward" group="N/A" header_id="0001" ip_addr="203.171.211.144" ip_dst="122.15.156.5" ip_dstport="47873" ip_src="203.27.235.106" ip_srcport="64303" latdec_dst="13" latdec_src="17" level="6" log_session_id="6515" longdec_dst="80" longdec_src="78" medium="32" msg_id="SSLVPN_TCPCONNSTAT" msg_vid="SSLVPN_TCPCONNSTAT" org_dst="Vodafone India" org_src="ICICIBANK Ltd, Banking, Mumbai" rbytes="346" rid="485773059317" sessionid="484525335521" severity="Informational" size="632" starttime="1467185619" stransaddr="10.68.100.135" time="1467185058" user_dst="venuskalra" alert="rcf_test2" bytes_src="0" city_src="Hyderabad" country_dst="India" country_src="India" dclass_r1="0.00%" dclass_r1_str="Compression_ratio_send" dclass_r2="0.00%" dclass_r2_str="Compression_ratio_recv" device_class="Application Firewall" device_group="Citrix Netscalar" device_ip="10.68.100.134" device_type="citrixns" device_type_id="168" did="blrsiemdec1" disposition="Allowed" dtransaddr="10.6

Please suggest the changes what I need to do in cef.xml  .Also please suggest do I need to do some work around in case of custom parsers.

[DavidWaugh1]

Hi

In the CEF parser there is a list of mappings. Under the <ExtensionKeys>

<ExtensionKeys>

                <ExtensionKey cefName="act" metaName="action"/>

                <ExtensionKey cefName="app" metaName="application"/>

                <ExtensionKey cefName="cat" metaName="category"/>

                <ExtensionKey cefName="cnt" metaName="event_counter"/>

                <ExtensionKey cefName="destinationDnsDomain" metaName="ddomain"/>

                <ExtensionKey cefName="destinationServiceName" metaName="service"/>

                <ExtensionKey cefName="destinationTranslatedAddress" metaName="dtransaddr"/>

                <ExtensionKey cefName="destinationTranslatedPort" metaName="dtransport"/>

                <ExtensionKey cefName="deviceCustomDate1" metaName="cs_customdate"/>

                <ExtensionKey cefName="deviceCustomDate2" metaName="cs_customdate"/>

                <ExtensionKey cefName="deviceDirection" metaName="direction">

                        <device2meta device="rsaflow" metaName="param_direction"/>

You need to make sure that the metakeys you have in your output are defined in the CEF parser.

What this means for example is that entries with deviceDirection=Blah in your CEF syslog go into the meta key direction.

[DavidWaugh1]

Also make sure that you unsubsribe from the CEF parser in live, or make the cef.xml file immutable. Otherwise the changes will get overwritten.

[AlexeyFedorov]

Hello David,

Can I create my custom CEF parser? I have my own event source send logs by CEF. I wish add some new mapping but I don't want to unsubscribe from CEF parser in RSA Live.

[SoumyajitDhara]

Hi David,

Is it really worth, if we subscribed as auto-update. Could you let me know how often it updating from RSA Live.

[DavidWaugh1]

Hi I think only the standard default keys are defined in the CEF parser, so it wont get updated that often.

[DavidWaugh1]

I havent tried this but you could copy the parser and create a new one perhaps called aaacef.xml (must be alpabetically above cef.xml) and then make your modification there.

[SalSanshez]

tqIdCMJpxOoRfittDAIUsDgjytWvZmuXrUPP6BnAqps=‌ alphabetically above because it will be loaded first in memory? I've heard this theory before. 

thanks

[JoeGumke]

Hi David,

I thought that live subscriptions only notified you of updates through a dashlet on a dashboard? I wasn't aware that subscriptions actually pushed out parsers automatically for you?

[EricPartington]

If you are forwarding the logs from the ESA appliance to a log decoder via syslog in CEF format then you will need the CEF parser enabled and the messages will be extracted as device.type=rsa_securityanalytics_esa (as you mentioned in the post).  This is the expected and correct action.

If you are asking for more meta to be extracted from the message payload that you posted then you need to leverage the metakeys that are already referenced in the cef.xml parser to map the field name in your alert, to the field name in cef.xml that maps to the field name in table-map(-custom).xml on your log decoder.

For instance you reference destination IP as one field you want as meta from the alert

Your alert needs to have the key named dst=<ip address>

[root@ldecoder6 cef]# cat cef.xml  | grep dst
                <ExtensionKey cefName="dst" metaName="daddr"/>

dst from your alert maps to daddr in the cef.xml template

[root@ldecoder6 cef]# cat ../../table-map.xml | grep daddr
        <mapping envisionName="daddr" nwName="ip.dst" flags="None" format="IPv4" envisionDisplayName="ForeignAddress|DestinationAddress|DestinationIPAddress|LocalAddress" failureKey="ipv6.dst" nullTokens="(null)|-"/>
        <mapping envisionName="daddr_v6" nwName="ipv6.dst" flags="None" format="IPv6" envisionDisplayName="DestinationAddressv6|ClientAddressv6" failureKey="host.dst" nullTokens="(null)|-"/>

then the table-map takes daddr and maps to ip.dst (which you see in the UI from the concentrator).

Not all the fields from your alert will be available as meta via CEF as not all of them are mapped (you cant override the rsa_securityanalytics_esa device.type with the device.type from your alert.  The point of these alerts are to provide enough information to record the event and allow you to pivot from the alert into the original sessions where the original events occurred.  So make sure your alerts have those pivot points (source or destination IP, group, alert name or other key meta that allows you to link the events together)