NetWitness Community

DavidWaugh1 · ‎2015-12-18

I recently had a customer who needed to parse McAfee Advanced Threat Detection Logs. This is not officially supported by Security Analytics so it was necessary to write a custom parser.

The log format for McAfee ATD is similar to the following:

Dec 15 03:27:05 localhost ATD2ESM[13207]: {"Summary": { "Event_Type": "ATD File Report","MISversion": "3.4.4.2.43772","SUMversion": "3.4.4.2.43772","OSversion":"win7sp1x64","fileId": "Not Available","Parent MD5": "Not Available","ATD IP":"10.213.248.17","Src IP": "10.213.248.69","Dst IP": "10.213.248.107","TaskId":"37","JobId": "37","JSONversion": "1.001.0718","hasDynamicAnalysis":"true","Subject": {"Name": "http://10.213.248.107/Apoorv/samples/automation_samples/vtest64.exe","Type": "PE32+ executable (console) x86-64","md5":"6AF8F4E3601156A59F050AAB4FAB5153","sha-1":"11BBBA1E7B39E1E193C6740B61F2A32E30ADD01A","size": "56832","Timestamp": "2014-12-15 11:24:12","parent_archive": "Not Available"},"Selectors": [{"Engine":"Sandbox","MalwareName": "Malware.Dynamic","Severity": "5"}],"Verdict":{"Severity": "5","Description": "Sample is malicious"},"Stats": [{"ID":"0","Category": "Persistence, Installation Boot Survival","Severity": "5"},{"ID":"1","Category": "Hiding, Camouflage, Stealthiness, Detection and Removal Protection","Severity": "0"},{"ID": "2","Category": "Security Solution / Mechanism bypass, termination and removal, Anti Debugging, VM Detection","Severity": "5"},{"ID": "3","Category": "Spreading","Severity": "2"},{"ID": "4","Category":"Exploiting, Shellcode","Severity": "0"},{"ID": "5","Category":"Networking","Severity": "3"},{"ID": "6","Category": "Data spying, Sniffing,Keylogging, Ebanking Fraud","Severity": "4"}],"Behavior": ["Created content under Windows system directory","Deleted AV auto-run registry key","Created a socket bound to a specific service provider and listen to an open port","Installed low level keyboard hook procedure","Deleted a key from auto-run registry entry","Altered auto-run registry entry that executed at next Windows boot"]}}

This is JSON and in a more human readable format would look as follows:

{

"Summary": {

"Event_Type": "ATD File Report",

"MISversion": "3.4.4.2.43772",

"SUMversion": "3.4.4.2.43772",

"OSversion": "win7sp1x64",

"fileId": "Not Available",

"Parent MD5": "Not Available",

"ATD IP": "10.213.248.17",

"Src IP": "10.213.248.69",

"Dst IP": "10.213.248.107",

"TaskId": "37",

"JobId": "37",

"JSONversion": "1.001.0718",

"hasDynamicAnalysis": "true",

"Subject": {

"Name": "http://10.213.248.107/Apoorv/samples/automation_samples/vtest64.exe",

"Type": "PE32+ executable (console) x86-64",

"md5": "6AF8F4E3601156A59F050AAB4FAB5153",

"sha-1": "11BBBA1E7B39E1E193C6740B61F2A32E30ADD01A",

"size": "56832",

"Timestamp": "2014-12-15 11:24:12",

"parent_archive": "Not Available"

},

"Selectors": [

{

"Engine": "Sandbox",

"MalwareName": "Malware.Dynamic",

"Severity": "5"

}

],

"Verdict": {

"Severity": "5",

"Description": "Sample is malicious"

},

"Stats": [

{

"ID": "0",

"Category": "Persistence, Installation Boot Survival",

"Severity": "5"

},

{

"ID": "1",

"Category": "Hiding, Camouflage, Stealthiness, Detection and Removal Protection",

"Severity": "0"

},

{

"ID": "2",

"Category": "Security Solution / Mechanism bypass, termination and removal, Anti Debugging, VM Detection",

"Severity": "5"

},

{

"ID": "3",

"Category": "Spreading",

"Severity": "2"

},

{

"ID": "4",

"Category": "Exploiting, Shellcode",

"Severity": "0"

},

{

"ID": "5",

"Category": "Networking",

"Severity": "3"

},

{

"ID": "6",

"Category": "Data spying, Sniffing,Keylogging, Ebanking Fraud",

"Severity": "4"

}

],

"Behavior": [

"Created content under Windows system directory",

"Deleted AV auto-run registry key",

"Created a socket bound to a specific service provider and listen to an open port",

"Installed low level keyboard hook procedure",

"Deleted a key from auto-run registry entry",

"Altered auto-run registry entry that executed at next Windows boot"

]

}

The difficulty I had with these messages is that the Behaviour fields and Stats field could contain any number of items. In this case the behaviour field contains 7 or more entries ID:0 to ID:6 and the behaviour field contains 6 entries. These are not necessarily maximum values and actual test logs would be needed to determine the maximum and minimum possible entries.

To parse this log I did the following:

Create a single message for each possible Stats field. Eg a Message for ID=0, A message for ID=0 ,ID=1 all the way up to the possibility that there were 7 entries in the stats field.
Used the {a | a,b | a,b,c|a,b,c,d|a,b,c,d,e|a,b,c,d,e,f} to handle the behaviour part of the field.

Other possibilities that I tried were:

Using a JSON parser. Unfortunately, the maximum amount of meta that a variable can hold is 256 characters and so I was unable to capture the complete string. The JSON parsers I found also used functions that were not supported by Security Analytics so could not be imported.

Here is the necessary table-map-custom.xml keys to use with the parser. These can be mapped to meta fields as desired.

tom86 · ‎2016-12-07

Hi,

thank you for the parser. What version did you use? McAfee Advanced Threat Defense Version:3.6.2.21.57069 but it does not work. Do you have an updated version of this parser?

Thanks in advance

NetWitness Community

Parsing McAfee Advanced Threat Defense Logs