Packet Parsers

This topic discusses and describes the packet (Lua) parsers available in RSA NetWitness Platform. If you need a parser that does not already exist, you can Request a Parser.

Note: More information on each of these parsers is available in Live. Navigate to Live search, and select RSA Lua Parser in the Resource Types field. From the results, select any parser and click to display all the information for the parser.

Context

Packet parsers identify the application layer protocol of sessions seen by the Decoder, and extract meta data from the packet payloads of the session.

Every packet parser is able to extract meta from every session. For example, a webmail session will be parsed by both an HTTP parser which identifies the session as HTTP and extracts meta from HTTP headers, and by a MAIL parser which extracts email-related meta from message headers. Further, if the session were to contain an executable file, its presence would be detected by a windows executable parser.

Packet parsers in RSA NetWitness may be broadly classified as:

  • System or Native parsers: These are compiled into the Decoder base code. Updates are delivered along with updates to RSA NetWitness. Many system parsers have Lua equivalents. In these cases, generally, the native parser may perform faster, while the Lua parser may extract more meta.
  • Lua parsers: these are written in the Lua programming language, and delivered via Live. Customers can write their own custom Lua parsers.
  • Flex parsers: these were written in a proprietary scripting language, Flex, and delivered via Live. These are now discontinued, and no longer delivered in Live. Every existing Flex parser has a better Lua equivalent, and all customers using NetWitness should not be using Flex parsers.

Packet Parsers in NetWitness

The following table describes the Lua parsers delivered with RSA NetWitness Platform.

Parser Name

Description

apt_artifacts

Detects possible apt WMI and windows registry manipulation.

Avamar

Identifies Avamar Backup and Recovery, TCP port 28001.

BGP_lua

Identifies BGP Routing Protocol.

bittorrent_lua

Identifies the bittorrent protocol and registers the name of the file being downloaded.

Canon_BJNP

Identifies Canon printer discover protocol BJNP.

cerber

Detects potential Cerber ransomware beaconing.

china_chopper

Detects cleartext China Chopper sessions.

creditcard_detection_lua

Attempts to detect possible credit card numbers and validate with Luhn's Algorithm.

CustomTCP

Detects CustomTCP beaconing activity. Registers C2 domain and victim hostname as alias.host meta.

db2_lua

Extracts queries from DB2 database protocol sessions.

DCERPC

Extracts action and Kerberos authentication from Microsoft's DCERPC protocol.

Derusbi_Server_Handshake

Detects Derusbi server handshake.

DHCP_lua

Identifies DHCP (BOOTP) and DHCPv6, extracts hosts and addresses.

DNP3_lua

DNP3 Distributed Network Protocol (SCADA).

DNS_verbose_lua

Identifies DNS sessions. Registers query and response records including record type. Registers protocol error messages.

dr_watson_lua

Detects Dr Watson crash report and registers name of crashed process.

duqu_lua

Detects binaries that may be related to the duqu threat.

DynDNS

Detects dynamic DNS hosts and servers.

ein_detection_lua

Attempts to detect Employer Identification Numbers.

ethernet_oui

Determines the manufacturer of eth.

Evilgrab

Detects possible Evilgrab APT malware activity.

exif

Extract longitude and latitude coordinates from exif data embedded in JPEG files.

fingerprint_7zip

Detects 7zip archive files.

fingerprint_access_db_lua

Identifies Microsoft Access database files.

fingerprint_apple_dmg_lua

Detects Mac OS X Disk Copy Disk Image files.

fingerprint_apple_ios_lua

Detects Apple IOS App files.

fingerprint_apple_iwork_lua

Detects Apple iWork files (Pages, Numbers and Keynote).

fingerprint_appleExec_lua

Detects MAC OSX executable binary files.

fingerprint_bmp

Detects BMP format image files.

fingerprint_cab

Identifies cabinet files (cab).

fingerprint_cad_lua

Detects Autodesk Autocad DWG, DXF, and DWF files.

fingerprint_chm_lua

Identifies Microsoft Compiled Help files, and detects potentially suspicious elements within.

fingerprint_flash

Detects Adobe Flash (swf) files.

fingerprint_font

Identifies font files: embedded opentype (eot), web open format (woff), opentype (otf), and truetype (ttf).

fingerprint_gif_lua

Identifies GIF files.

fingerprint_gzip

Detects files which have been compressed using the gzip family of compression programs (gzip, bzip, etc).

fingerprint_java

Detects Java JAR and CLASS files.

fingerprint_javascript_lua

Detect javascript, and suspicious javascript actions and anomalies.

fingerprint_job

Identifies windows job task scheduling files.

fingerprint_jpg_lua

Detects JPEG image files.

fingerprint_lnk_lua

Identifies lnk files and detects possible exploit characteristics.

fingerprint_msi_lua

Identifies Microsoft OLE / Compound Document Format Windows Installer files.

fingerprint_mssql_lua

Detects Microsoft SQL Server database files.

fingerprint_office_lua

Identifies Microsoft Office 95-2007 Word, Excel, and Powerpoint documents.

fingerprint_pdf_lua

Identifies PDF files and detects risky characteristics.

fingerprint_pff

Detects Microsoft Outlook Personal File Folder objects such as pab, pst, and ost.

fingerprint_pkcs12_lua

Detects PKCS #12 format private key files.

fingerprint_png_lua

Detects PNG image files.

Fingerprint_Private_Key

Detects SSH and PGP private key files.

fingerprint_rar_lua

Detects RAR archive files.

fingerprint_rtf_lua

Detects RTF files.

fingerprint_unix_script_lua

Identifies shell, perl, ruby, and python scripts.

fingerprint_webm

Detects webm and matroska video files.

fingerprint_zip

Detects PK format zip files, and extracts the names of files contained in the archive.

FIX_lua

Identifies the Financial Information Exchange Protocol. Form_Data_lua Extracts submitted values from HTTP POST actions.

Form_Data_lua

Extracts submitted values from HTTP POST actions.

FTP_lua

File Transfer Protocol (FTP) RFC 959.

ghost

Detects likely Ghost Rat beacon sessions.

glass_rat

Detects the network communication used by the GlassRAT Trojan identified by RSA Research.

gnutella_lua

Identifies the Gnutella file sharing protocol.

HTML_threat

Detects common HTML threat techniques such as hidden frames and embedded objects.

htran_lua

Identifies the error message generated by the htran redirection tool.

HTTP_lua

Extracts values from HTTP protocol request and response headers.

HTTP_lua_options

Use this file to influence the behavior of the HTTP_lua parser. For details, see HTTP Lua Parser Options File.

HTTP_SQL_Injection

Detect possible injection of SQL commands in HTTP requests.

ICMP

Provides types and codes from ICMP packets.

IDN_homograph

Detects punycode-encoded internationalized domain names which use non-Latin Unicode code points whose glyphs resemble those of Latin Unicode code points. Registers the decoded homograph as analysis.service meta.

Reference the RSA Link blog post from RSA Research for more details about this threat: Dissecting PunyCode - Not All Characters are Created Equal.

IMAP_lua

Identifies IMAP, registers commands, errors, usernames, and passwords.

IRC_verbose_lua

Expanded IRC parsing.

ISAKMP

Identifies ISAKMP Internet Security Association and Key Management Protocol).

iSCSI

Identifies SCSI-over-IP.

JSON-RPC

Identifies JSON-RPC 2.0 streams. Will not identify JSON-RPC 1.0 streams, and may not identify JSON-RPC over transports such as HTTP.

Kerberos

Extracts meta from the Kerberos network protocol.

LDAP

Lightweight Directory Access Protocol, and extensions.

LDAP_options

Lightweight Directory Access Protocol, and extensions. Use this file to influence the behavior of the LDAP parser. For details, see LDAP Parser Options File.

Lync

Identifies Microsoft Lync (formerly Microsoft Office Communicator, Windows Messenger).

MAIL_lua

Extracts values from email messages, such as email addresses, subject, and client.

Mail_lua_options

Use this file to influence the behavior of the Mail_lua parser. For details, see Mail Lua Parser Options File.

Mitozhan

Detects Mitozhan malware command and control.

modbus

Identifies MODBUS TCP/IP, extracts commands, errors, and device identifications.

MSU_rat

Detects MSU RAT activity.

NetBIOS_lua

NetBIOS over TCP/IP: NBNS, NBDS, NBSS.

NFS_lua

Identifies and parses RPC-related protocols NFS, MOUNT, and PORTMAP.

NTLMSSP_lua

Extracts Active Directory user information from NTLM HTTP headers from proxy authorization.

ntp_lua

Identifies Network Time Protocol.

OCSP_lua

Extracts certificate information and status from OCSP messages.

Packers

Detects specific packer used to pack executables.

phishing_lua

Registers the host portion from each URL found within an email.

plugx

Detect PlugX malware.

Poison_Ivy

Detects Poison Ivy RAT activity.

POP3_lua

Post Office Protocol version 3.

Proxy_Block_Page

Parses proxy denied exception pages.

pvid

Detects PGV_PVID malware activity. PGV_PVID is a cookie string the actor put into the malware's POST routine.

pwdump

Detects output from Windows password dumping tools such as pwdump.

QQ_lua

Identifies QQ (OICQ protocol) sessions.

radius

Remote Authentication Dial In User Service.

RDP_lua

Identifies the Microsoft Remote Desktop Protocol.

rekaf

Detects a variant of rekaf and derives the xor key (crypto) and name of the infected host.

ripng_lua

Identifies the RIP routing protocol.

rlogin

Identifies Remote Login protocol.

rsync

Identifies the RSYNC ;Network Protocol.

rtmp_lua

Real Time Messaging Protocol.

RTSP

Identifies the Real Time Streaming Protocol.

SCCP_lua

Cisco Skinny Client Control Protocol.

Search_Engines

Extracts search terms from search engine queries.

sekur

Detects the initial handshake of the Sekur/Anunak Trojan.

session_analysis

Analyzes session characteristics such as bytes transmitted vs bytes received, TCP flags seen, etc.

shadyrat_lua

Identifies potential artifacts related to shadyrat command and control traffic.

Signed_Executable

Extracts the Certificate Authority, Subject, and Serial Number from the first x509v3 certificate in the certificate chain of a signed executable.

SIP_lua

Session Initiation Protocol (SIP).

SMB_lua

Parses the Microsoft SMB/CIFS protocol, versions 1 and 2.

SMTP_lua

Parses the SMTP protocol (RFC 5321).

SNMP_lua

Parses SNMP versions 1, 2c, 2p, 2u, and 3.

socks_lua

Identifies Socks protocol version 4 and 5.

SoulSeek_lua

Identifies the SoulSeek file sharing protocol.

spectrum_lua

Determines which sessions are sent to Malware Analysis, based upon file types seen in the session, and total session size.

SSH_lua

Identifies SSH protocol.

struts_exploit

Detects a possible Remote Code Execution attack when using the Struts REST plugin with XStream handler to handle XML payloads.

supercmd

Detects SuperCMD Trojan beaconing. For details on the SuperCMD Rat, see the SUPERCMD RAT RSA blog post.

teredo

Identifies teredo tunneled sessions. Performs identification only. No meta is extracted.

TDS_lua

Identifies Microsoft SQL Server 'Tabular Data Stream' protocol.

TFTP_lua

Identifies Trivial File Transfer Protocol, extracts names of files transferred.

TLD_lua

Extracts the top-level domain and second-level domain portions from hostnames.

TLD_lua_options

Use this file to influence the behavior of the TLD_lua parser. For details, see TLD Lua Parser Options File.

TLS_lua

Identifies SSL 2.0, SSL 3.0, TLS 1.0, TLS 1.1, and TLS 1.2.

TN3270E_lua

Identifies IBM TN3270E sessions.

traffic_flow

Provides subnet names for internal networks, and directionality of the session (inbound, outbound, lateral).

traffic_flow_options

This is an optional file for use with the traffic_flow Lua parser. If used, this file provides a way for customers to configure internal subnets as described within the full product documentation for this parser. For details, see Editing the Options File in the Traffic Flow Lua topic.

vCard_lua

Extracts fullname and email values from vCard, xCard, jCard, and hCard formats.

VNC

Identifies the Remote Framebuffer protocol used by VNC and its derivatives.

windows_command_shell_lua

Identifies Microsoft Windows command shell sessions.

windows_executable

Identifies windows executables, and analyzes them for anomalies and other suspicious characteristics.

X11_lua

Identifies the X11 protocol (RFC 1013).

xor_executable_lua

Detects executables that have been xor or hex encoded.

Discontinued Packet Parsers

The following table lists the Lua parsers that have been removed from the system.

Name Description Notes

AIM_lua

OSCAR protocol used by AIM (AOL Instant Messenger) and ICQ, and AIM-express web client.

As of December 15, 2017, AOL Instant Messenger products and services have been shut down and no longer work.

BITS

Identifies Microsoft BITS Protocol.

BITS was added to HTTP_lua, making the standalone BITS parser redundant. BITS parsing in HTTP_lua is also much more complete than it was in the standalone parser.