This website uses cookies. By clicking Accept, you consent to the use of cookies. Click Here to learn more about how we use cookies.
Accept
Reject

NetWitness Community

  • Home
  • Products
    • NetWitness Platform
      • Advisories
      • Documentation
        • Platform Documentation
        • Known Issues
        • Security Fixes
        • Hardware Documentation
        • Threat Content
        • Unified Data Model
        • Videos
      • Downloads
      • Integrations
      • Knowledge Base
    • NetWitness Cloud SIEM
      • Advisories
      • Documentation
      • Knowledge Base
    • NetWitness Detect AI
      • Advisories
      • Documentation
      • Knowledge Base
    • NetWitness Investigator
    • NetWitness Orchestrator
      • Advisories
      • Documentation
      • Knowledge Base
      • Legacy NetWitness Orchestrator
        • Advisories
        • Documentation
  • Community
    • Blog
    • Discussions
    • Events
    • Idea Exchange
  • Support
    • Case Portal
      • Create New Case
      • View My Cases
      • View My Team's Cases
    • Community Support
      • Getting Started
      • News & Announcements
      • Community Support Forum
      • Community Support Articles
    • Product Life Cycle
    • Support Information
    • General Security Advisories
  • Training
    • Blog
    • Certification Program
    • Course Catalog
    • New Product Readiness
    • On-Demand Subscriptions
    • Student Resources
    • Upcoming Events
  • Technology Partners
  • Trust Center
Sign InRegister Now
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
NetWitness Community Blog
Subscribe to the official NetWitness Community blog for information about new product features, industry insights, best practices, and more.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
  • NetWitness Community
  • Blog
  • LUA - Common Event Format (CEF) Script Template

LUA - Common Event Format (CEF) Script Template

IanRedden
Employee IanRedden
Employee
Options
  • Subscribe to RSS Feed
  • Mark as New
  • Mark as Read
  • Bookmark
  • Subscribe
  • Email to a Friend
  • Printer Friendly Page
  • Report Inappropriate Content
‎2016-10-19 06:14 PM

The CEF helper script attempts to fill the gap of CEF extensions that the system parser does not currently parse. For a complete list of what we do parse, please refer to Supported CEF Meta Keys - RSA Security Analytics Documentation .

 

This CEF helper template was written to be highly configurable as well as adhere to revision 16 of the Common Event Format (CEF) standards document. Its aim is to be able to parse meta data into any meta key in RSA Netwitness from any security appliance with the least amount of programming.


This is performed by several routines within the code that associates “csX” and “cnX” within a CEF extension with their appropriate “csXLabel” and “cnXLabel”. If no “csLabel” exists, it will use “csX” or “cnX” as the key name (where X is a numeric value). For example, a CEF message containing “cs3=name cs3Label=Ian” will set the key name to “name” and the value to “Ian” whereas a CEF message containing “cs3=Ian” will set the key name to “cs3” and the value to “Ian”. However, using scripts configuration, you can translate cs3 to any meta key name you want (i.e. username). This is configured in the t_keys_to_use variable.  More information on that below!

 

As CEF messages can also contain carriage returns (\n), if a carriage return is found, it will parse each line into a separate meta value of the same key name. For example, a CEF message containing “cs2=user.names cs2Label=iredden\nepartington” will result in 2 meta values (user.names) containing “iredden” and “epartington”.

 

The template can be downloaded at the bottom of this article.  It is configured for a FireEye HX appliance but can be easily modified for anything!

 

 

Configuration

The main configuration of the script is in 2 variables.  These variables are t_keys_to_use and b_debug.

t_keys_to_use – Is a LUA key/value table containing which keys to parse.

 

b_debug – By default, this variable is set to true.  This means that no meta will be created.  Instead, output will be provided for debugging to logs.

 

You also need to configure the cefhelper:setKeys() section of the script.  It needs to contain all the same keys from the t_keys_to_use table.  For example:

cefhelper:setKeys({
   nwlanguagekey.create("vx.threatscore"),
   nwlanguagekey.create("vx.detection"),
   nwlanguagekey.create("virusname"),
   ...
   nwlanguagekey.create("alias.host"),
   nwlanguagekey.create("vx.filedesc")
})



You can access logs on a Log Decoder via the REST API:

http://de.co.der.ip:50102/logs?msg=pull&force-content-type=text/plain&expiry=600&count=50 

 

Example - Accellion File Transfer (FTA)

 

The CEF system parser from RSA Live parses numerous CEF extension keys.  However, there are a few meta keys that we need a helper for to be able to parse CEF logs from Accellion FTA:

CEF Extension

Description

System Parser

NetWitness Key Name

msg

Message

Yes

msg

src

Source

Yes

src

deviceFacility

Device Facility

Yes

deviceFacility

fname

File Name

No

filename

fsize

File Size

No

n/a

location

Location

No

directory

type

Transfer Type

No

action

seconds

Transfer Time - # Seconds

No

n/a

suser

Source User

Yes

username

 

Helper Configuration:

local t_keys_to_use = {
        ["fname"] = "filename",
        ["location"] = "directory",

        ["type"] = "action"
}

cefhelper:setKeys({
   nwlanguagekey.create("filename"),
   nwlanguagekey.create("directory"),
   nwlanguagekey.create("action")
})

local b_debug = false

 

Example CEF Message:

CEF:0|Accellion|FTA|FTA_9_12_80|203|Download|1|msg=(172.16.20.45) test.txt downloaded by ian.redden@rsa.local (23 bytes, 3.5 sec) (Type: ssl_download, Location: ) src=172.16.20.45 deviceFacility=local5 fname=test.txt fsize=23 cs2Label=location cs2= cs1Label=type cs1=ssl_download cn1Label=seconds cn1=3.5 suser=ian.redden@rsa.local

 

Example - Payload Security's VxStream

 

The CEF system parser from RSA Live parses numerous CEF extension keys.  However, there are a few meta keys that we need a helper for to be able to parse CEF logs from Accellion FTA:

CEF Extension

Description

System Parser

NetWitness Key Name

Threat Score

Threat Score

No

vx.threatscore

AV Detection Rate

Detection Rate

No

vx.detection

Malware Family

Malware Family Name

No

virusname

fileHash

File Hash Value

No

vx.filehash

fname

File Name

No

vx.fname

fsize

File Size

No

vx.fsize

fileType

File Type

No

vx.filetype

flexString1

Comments

No

vx.comments

Client

Client

No

vx.client

Indicators

Source User

No

vx.indicators

Contacted Domains

Contacted Domains

No

alias.host

Contacted Hosts

Contacted Hosts

No

alias.host

Compromised Hosts

Compromised Hosts

No

alias.host

File Description

File Description

No

vx.filedesc

 

Helper Configuration:

local t_keys_to_use = {
   ["Threat Score"] = "vx.threatscore",
   ["AV Detection Rate"] = "vx.detection",
   ["Malware Family"] = "virusname",
   ["fileHash"] = "vx.filehash",
   ["fname"] = "vx.fname",
   ["fsize"] = "vx.fsize",
   ["fileType"] = "vx.filetype",
   ["flexString1"] = "vx.comments",
   ["Client"] = "vx.client",
   ["Indicators"] = "vx.indicators",
   ["Contacted Domains"] = "alias.host",
   ["Contacted Hosts"] = "alias.host",
   ["Compromised Hosts"] = "alias.host",
   ["File Description"] = "vx.filedesc"
}

cefhelper:setKeys({
   nwlanguagekey.create("vx.threatscore"),
   nwlanguagekey.create("vx.detection"),
   nwlanguagekey.create("virusname"),
   nwlanguagekey.create("vx.filehash"),
   nwlanguagekey.create("vx.fname"),
   nwlanguagekey.create("vx.fsize"),
   nwlanguagekey.create("vx.filetype"),
   nwlanguagekey.create("vx.comments"),
   nwlanguagekey.create("vx.client"),
   nwlanguagekey.create("vx.indicators"),
   nwlanguagekey.create("alias.host"),
   nwlanguagekey.create("vx.filedesc")
})

local b_debug = false

 

Example CEF Message:

Mar 29 15:42:59 192.168.1.100 CEF:0|Payload Security|VxStream|6.20|Sample Analysis Result - Malicious|Sample Analysis Result - Malicious|100|end=03/29/2017 21:28:55 cn1=100 cn1Label=Threat Score cn2=88 cn2Label=AV Detection Rate cs1=Trojan.Generic cs1Label=Malware Family cs2=100 cs2Label=EnvironmentID cs3=Windows 7 32 bit cs3Label=Environment Description fileHash=8271d841b9971f04d6a48804d06ecd7185d71ed8546988b1697fbe01741a8572 fname=8271d841b9971f04d6a48804d06ecd7185d71ed8546988b1697fbe01741a8572 fsize=357888 fileType=exe request=https://www.hybrid-analysis.com/sample/8271d841b9971f04d6a48804d06ecd7185d71ed8546988b1697fbe01741a8572/?environmentId\=100 msg=Malicious flexString1= flexString1Label=Uploader Comment cs4=zpr5huq4bgmutfnf.tor2web.org \nzpr5huq4bgmutfnf.onion.to \ncrl2.alphassl.com \nipinfo.io \nkosdfnure75.op1gifsd05mllk.com \ngfdkotriam.fo4j4wnq51hepa.com cs4Label=Contacted Domains cs5=84.200.69.80:53 \n34.196.176.140:80 \n185.100.85.150:443 \n192.36.27.5:443 cs5Label=Contacted Hosts cs6=84.200.69.80 \n185.100.85.150 \n192.36.27.5 cs6Label=Compromised Hosts cs8=2812134 \n2015576 \n2812134 \n2020116 \n2020716 \n2016810 \n2016810 cs8Label=ET Alerts cs9=api-12:1:4 \napi-75:1:10 \nsuricata-2:2:10 \nnetwork-21:2:10 \napi-10:1:6 \ntarget-4:2:10 \nnetwork-0:0:1 \nnetwork-2:0:5 \ntarget-25:0:3 \nnetwork-22:1:10 \nregistry-25:1:3 \nstatic-3:1:10 \ntarget-58:2:10 \nstatic-6:1:10 \navtest-3:2:10 \nsuricata-1:1:10 \napi-25:1:7 \nnetwork-27:2:10 \nregistry-27:1:10 \nnetwork-14:2:9 \napi-16:0:1 \nregistry-36:1:8 \nstring-14:1:3 \napi-51:1:5 \nregistry-35:1:5 \napi-37:0:10 \napi-76:1:10 \nnetwork-24:2:6 \nstatic-17:1:10 \nregistry-1:1:10 \nnetwork-5:1:7 \ntarget-14:2:8 \nhandle-0:1:7 \nstatic-0:1:10 \ntarget-3:0:5 \napi-27:1:10 \napi-77:1:10 \navtest-0:2:8 \nnetwork-1:0:1 \napi-55:0:7 \napi-9:1:4 \napi-42:2:7 \nstatic-60:1:10 \napi-39:0:8 \nregistry-61:1:10 \nstring-13:1:7 \nnetwork-15:2:5 \napi-26:1:10 \navtest-5:2:5 \nmutant-0:0:3 \nregistry-55:0:10 \napi-11:0:2 \napi-6:1:4 \nstatic-1:1:1 cs9Label=Indicators priority=9 cs10=PE32 executable (GUI) Intel 80386 (stripped to external PDB) \n for MS Windows cs10Label=File Description

 

Example – FireEye HX

FireEye HX is FireEye’s endpoint protection appliance.  There are a few meta keys that we need to parse from FireEye HX:

 

CEF Extension

Description

NetWitness Key Name

IOC Name

IOC Name / Malware Family

risk.warning

Target OS

Target Operating System

OS

dhost

Destination Hostname

alias.host

dst

Destination IP

ip.dst

dntdom

Destination Domain

ad.domain.dst

 

Helper Configuration:

local t_keys_to_use = {
        ["IOC Name"] = "risk.warning",
        ["Target OS"] = "OS”,
        ["dhost"] = "alias.host",
        ["dst"] = "ip.dst",
        ["dntdom"] = "ad.domain.dst"
}

cefhelper:setKeys({
   nwlanguagekey.create("risk.warning"),
   nwlanguagekey.create("OS"),

   nwlanguagekey.create("alias.host"),
   nwlanguagekey.create("ip.dst"),
   nwlanguagekey.create("ad.domain.dst")
})

local b_debug = false

Example CEF Message:

CEF:0|fireeye|hx|3.1.3|IOC Hit Found|IOC Hit Found|10|rt=Sep 29 2016 02:39:54 UTC dvchost=lab.rsa.local categoryDeviceGroup=/IDS categoryDeviceType=Forensic Investigation categoryObject=/Host cs1Label=Host Agent Cert Hash cs1=YUGh0fvlBG5ewBBahhbEZH dst=172.16.10.50 dmac=aa-bb-7a-fa-75-d8 dhost=victim-a3c696c8 dntdom=WORKGROUP deviceCustomDate1Label=Agent Last Audit deviceCustomDate1=Sep 29 2016 02:39:49 UTC cs2Label=FireEye Agent Version cs2=21.33.0 cs5Label=Target GMT Offset cs5=PT0H cs6Label=Target OS cs6=Windows 7 Professional 7601 Service Pack 1 externalId=1049717 start=Sep 29 2016 02:39:28 UTC categoryOutcome=/Success categorySignificance=/Compromise categoryBehavior=/Execute categoryTechnique=Exploit act=Detection IOC Hit msg=Host victim-a3c696c8 IOC compromise alert categoryTupleDescription=A Detection IOC found a compromise indication. cs4Label=IOC Name cs4=SANDSTORM (FAMILY)

 

DISCLAIMER: As always, this script is provided as is.  If you have any questions, feel free to reach out to me at ian.redden@rsa.com.

Happy Hunting!

 

Update 5/26/2017 - Updated CEF template fixing several bugs.

Update 6/14/2017 - Added sample for VxStream

ceftemplate.lua.zip
  • cef
  • fireeye
  • Lua
  • NetWitness
  • netwitness logs
  • NW
  • NWP
  • RSA NetWitness
  • RSA NetWitness Platform
  • security analytics logs
ceftemplate.lua.zip
7 Likes
Share

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

  • Comment
Latest Articles
  • Agent Tesla: The Information Stealer
  • Threat Analysis: Detecting “Follina” (CVE-2022-30190) RCE Vulnerability with Netwitness Endpoint
  • Introducing NetWitness Vision XDR
  • Introducing NetWitness Platform XDR v12.0
  • Atlassian Confluence Zero-day Vulnerability (0-Zero) CVE-2022-26134: What You Need To Know
  • ‘Follina’ CVE-2022-30190 0-Day: What You Need To Know
  • CVE-2022-1388: BIG-IP iControl REST RCE Vulnerability
  • Ragnar Locker Ransomware: The Rampage Continues…
  • Ransomware Email Attacks: Beware of BazarLoader
  • Detecting Impacket with Netwitness Endpoint
Labels
  • Announcements 54
  • Events 2
  • Features 9
  • Integrations 6
  • Resources 57
  • Tutorials 21
  • Use Cases 21
  • Videos 116
Powered by Khoros
  • Blog
  • Events
  • Discussions
  • Idea Exchange
  • Knowledge Base
  • Case Portal
  • Community Support
  • Product Life Cycle
  • Support Information
  • About the Community
  • Terms & Conditions
  • Privacy Statement
  • Acceptable Use Policy
  • Employee Login
© 2022 RSA Security LLC or its affiliates. All rights reserved.