NetWitness Log Parser Improvements

[Anonymous]

The RSA NetWitness Log Parsing team has reviewed the Top 50 Log Parsers that generate the most number of incoming “Unknown Message Defect” support cases.

Summary for Top 50 Event Sources	Total	# of Log Parsers Released on Live
Number of Event Sources - IMPROVED & RELEASED	38	38 (as of 21 Jun 2017)
Number of Event Sources - CANNOT BE IMPROVED	12	Not Applicable
Final Count	50	38

These enhancements are part of a strategic initiative to drive improvements to Log Parsers.

Benefits from these improvements result in:

• Fewer Unknown Messages
• Improved Device Discovery
• Better Adaptability to newer versions of an Event Source
• Reduced Parser Maintenance

To take advantage of these improvements, you will need to download the latest versions of the Log Parsers listed below from the NetWitness Live Portal.

A Look into the Details

50 Log Parsers were identified as the ones that were causing almost 80% of ALL the incoming requests for fixing unknown messages & minor defects. All of them were reviewed in detail and checked for scope of adding any improvements.

Design Considerations

The team investigated the reasons behind generation of high volume of incoming unknown message tickets.

The factors considered were:

• Common patterns seen across all unknown message tickets for a particular event source.
• Certain Log formats that generate Unknown messages
  
• Were these logs result of a Configuration Error?
• Were these logs generated after upgrading to a newer version of the Event Source?
• Backwards Compatibility Impact of the modifications:
  • If the solution is re-designing the parser in any way (re-writing a few or all message definitions or Header definitions)  the 'after-effect' on the Meta Key Footprint should be nothing to very minimal.

Log Parsers were updated/Re-designed only If the Backwards Compatibility impact from above parameters were negligible.

Here is a brief highlight of the design improvements-

• The Log Parsers have been redesigned to identify all events generated by the event source. They have been made future proof as much as possible to parse newer events.
• Log Parsers which parse logs with structured formats that use tag=value functionality, have been updated to accommodate New/Unknown tags, which significantly reduces the number of unknown messages.
• These updates were also done to cover as many new changes as possible that may be introduced in Newer versions of the product.

Here is the list of 38 Log Parsers that were Improved and released to NetWitness Live –

	Log Parser Name
1	Bit9-Bit9 Security Platform
2	Blue Coat-Blue Coat ProxySG SGOS
3	Check Point-Check Point Security Suite, IPS-1
4	Cisco-Cisco Adaptive Security Appliance
5	Cisco-Cisco IronPort Email Security Appliance
6	Cisco-Cisco IronPort Web Security Appliance (WSA)
7	Cisco-Cisco Secure Access Control Server & Cisco-Cisco Identity Services Engine
8	Cisco-Cisco Secure IDS or IPS
9	Cisco-Cisco Wireless LAN Controller (2100 Series and 4400 Series)
10	F5-F5 Big-IP Application Security Manager
11	FireEye-FireEye Web Malware Protection System
12	Fortinet-Fortinet FortiGate
13	IBM-IBM AIX
14	IBM-IBM DB2 Universal Database
15	IBM-IBM iSeries AS400
16	IBM-IBM ISS SiteProtector
17	IBM-IBM WebSphere
18	Juniper-Juniper Networks SSL VPN
19	Lancope-Lancope StealthWatch
20	McAfee-McAfee Email Gateway (formerly known as CipherTrust IronMail)
21	McAfee-McAfee Network Security Platform (Intrushield)
22	Microsoft-Microsoft Exchange Server
23	Microsoft-Microsoft Internet Information Services
24	Microsoft-Microsoft SQL Server
25	Microsoft-Microsoft Windows using Eventing Collection
26	Microsoft-Microsoft Windows using: Adiscon Event Reporter
27	Microsoft-Microsoft Windows using: Intersect Alliance SNARE
28	Oracle-Oracle Access Manager
29	Oracle-Oracle Database
30	Oracle-Sun Solaris
31	SNORT/SourceFire
32	Trend Micro-Trend Micro Control Manager
33	Tripwire-Tripwire Enterprise
34	UnboundID - UnboundID Identity Data Store
35	Vmware-VMware ESX/ESXi
36	Vmware-VMware vCenter Server
37	Voltage SecureData
38	Websense-Websense Web Security

Here is the list of 12 Log Parsers that cannot be improved further -

	Log Parser Name
1	Check Point-Check Point IPSO (nokiaipso)
2	Cisco-Cisco Router/Switch
3	Citrix-Citrix NetScaler
4	F5-F5 Big IP (Local Traffic Manager)
5	Infoblox-Infoblox NIOS
6	Juniper-Juniper Networks JUNOS
7	McAfee-McAfee ePolicy Orchestrator
8	McAfee-McAfee Web Gateway
9	Palo Alto Networks-Palo Alto Networks Enterprise Firewall
10	Red Hat Linux (RHEL)
11	RSA Authentication Manager/UCM  (rsaacesrv)
12	Symantec-Symantec Endpoint Protection

Most of these contain highly unstructured Log formats. Due to several Backwards Compatibility / Performance impact issues, these couldn't be improved.

Please note that these 12 Log Parsers are expected to generate unknown messages. The team depends on the incoming support requests for updating these parsers. Once these requests are received, the team will get them updated as soon as possible.

The RSA NetWitness Log Parsing team will be closely monitoring any incoming requests for the improved Log Parsers and further improve them as applicable. They will continue to power these improvements to other Supported Log parsers in the library.