How NetWitness Investigate Works

[#1]

How NetWitness Investigate Works

NetWitness Investigate provides analysts the means to analyze events that have been captured by NetWitness. Using Investigate, analysts can examine packet, log, and endpoint data, and identify possible internal or external threats in their environment. There are several different views available to analysts to gain different perspectives into the data in their environment. A key element that all the views have in common is metadata.

Metadata, Meta Keys, Meta Values, and Meta Entities

NetWitness audits and monitors all data communications in an environment. One type of service--a Decoder--ingests, parses, and stores the original packets captured on the network, logs forwarded by a device, and endpoint events seen by the endpoint agent. The configured rules, parsers, and feeds on the Decoder create metadata that analysts can use to investigate the ingested logs, packets, and endpoint data. Another type of service, called a Concentrator, indexes and stores the metadata, making it more efficient to search through all types of metadata.

The metadata is created to give analysts valuable points of reference associated with the original data. This allows analysts to quickly get a sense of what has transpired without being required to examine every detail of an event. The metadata is in the form of a meta key and meta values for the key. For example, ip.src is a meta key, and an IP address (192.168.1.1) that is the source of the traffic is a meta value tagged as ip.src. When you view data in Investigate, you see the meta key ip.src and all of the IP addresses (meta values) that are tagged with that key. Some meta keys are built-in and others may be custom keys specific to your environment and defined by the administrator. All metadata, no matter the source of the data, is normalized into the Unified Data Model for NetWitness Platform to keep similar metadata concepts grouped together into like meta keys (see https://community.netwitness.com/t5/netwitness-platform-unified-data/tkb-p/netwitness-udm).

A meta entity is an alias that groups together the results from other meta keys. Meta entities organize similar meta keys into a single, easier to use, meta type. For example, the default Core database language includes distinct meta keys for IP source and IP destination. One of the built-in meta entities named ip.all represents the combined set of all IP sources and destinations. Some meta entities are already included by default, and the administrator can create custom meta entities. Analysts can use a meta entity in a query, a meta group, a column group, and a query profile. Parallel coordinates visualizations do not support meta entities. Administrators can use meta entities to define a query prefix to apply to a user role and a user as described in the System Security and User Management Guide. The Decoder Configuration Guide provides additional information about creating meta entities and how they can be used in rules.

Note: Meta entities need to be configured on all upstream Concentrators. If any Concentrator does not have a meta entity configured, that meta entity will be empty when you query the Broker.

Analysts usually query the Broker or Concentrator to discover threats. The Concentrator handles queries, only going to the Decoder for raw logs or endpoint events or a full reconstruction of network events. ESA, Malware Analysis, and Reporting Engine also query the Concentrator, where they can quickly get all the pertinent metadata associated with an event and generate information about the event without having to query each Decoder. In some special cases, analysts may query a Decoder.

Triggers for an Investigation

These are a few examples of triggers for an investigation:

• You receive intelligence about a new active directory hack. Starting in the Events view, you use that intelligence to run a search across all of your raw Active Directory log data for the last 24 hours.
• You are asked by the SOC manager to find any Pokemon Go malware due to its popularity. Starting in the Navigate view, you craft a query to look for an HTTP session using a specific user agent related to the malware that your SOC manager found on a security blog.
• An incident responder escalates a ticket that shows some odd indicators related to a host. Starting in the Hosts view, you examine that host to find specific details.
• You are looking for the next zero day attack and start drilling into the network metadata in the Navigate view (or the Filter Events panel in the Events view) to find any abnormal automated sessions leaving the enterprise.
• You are asked by your SOC manager to find any information related to user jarvis, an employee who was just let go. Starting in the Users view you can filter for that username to make sure there is no longer any activity for that user and see if that user deviated from their typical behavior prior to being let go.

• A phishing attack detected has an associated attachment, and you want to know what devices in your environment have seen that file by searching for the file hash in the Files view.

• A malicious file has been automatically found in your environment, and you want to review the static and dynamic analysis done on that file along with how many systems it has been transmitted to or from. Starting in Investigate > Malware Analysis you can see the analysis results.

Workflow of an Investigation

Analysts can investigate data captured by NetWitness, and deep dive from information on a NetWitness dashboard, the Springboard (Version 11.5 and later), a NetWitness Respond incident or alert, a report created by the NetWitness Reporting Engine, or a third-party application. During the course of an investigation, analysts can move seamlessly between views: the Navigate view, the Events view, the Legacy Events view, the Hosts view, the Files view, the Users (Entities) view, and the Malware Analysis view. By default, the Navigate view and the Legacy Events view are disabled.

This figure illustrates the Version 12.4.2 menu, which is optimized with new NetWitness logo and renamed our product as NetWitness Platform .

Note:
- Specific user roles and permissions are required for a user to conduct investigations and malware analysis in NetWitness. If you cannot see a view, the administrator may need to adjust the roles and permissions configured for you.

The views are closely integrated to reduce the need to jump from one view to another. Each use case determines the starting point for your investigation; every situation is unique in terms of the types of information you are attempting to find. Many investigations start in one view, and end in a different view as you learn something and then need to follow that result to a different line of questioning. Experienced analysts frequently begin an investigation in the Navigate view or the Events view. Less experienced analysts can begin in the Dashboard, Respond view, or the Springboard, in which clickable incidents and alerts link to detailed information and analyses in the different views.

Go to...	Focus
Navigate view	The Navigate view  works well for providing a high-level vantage point of what has been seen in your environment for a specific time range by showing meta keys and meta values for log, endpoint, and network events. After drilling into meta values, go to the Events view to see the raw event (See Filter Results in the Navigate View.)
Events view	The Events view (default Investigate view) is the workflow for analysts interacting with events, presenting different facets of the same data in adjacent panels. There is no need to go to the Navigate view to drill into meta values as the Filter Events Panel in the Events view provides this functionality. (See Refining the Results Set, Reconstructing and Analyzing Events, and Downloading and Acting Upon Results.)
Legacy Events view	The Legacy Events view was the original workflow for looking at event details in Version 11.0 to 11.3.x.x. The Legacy Events is replaced by the 11.4 and later Events view and it is hidden unless the administrator enables it. (See Refining the Results Set, Reconstructing and Analyzing Events, and Downloading and Acting Upon Results.)
Hosts view	Hosts on which the NetWitness Endpoint agents are running are listed. For every host, you can view processes, drivers, DLLs, files (executables), services, anomalies, and autoruns that are running, and information related to logged-in users. (See the NetWitness Endpoint User Guide.)
Files view	Unique files in your deployment, such as PE, Macho, and ELF, are listed. For each file, you can view details such as file name, reputation status, file status, risk score, signature, checksum, and others. (See the NetWitness Endpoint User Guide.)
Malware Analysis view	If you are running a Malware Analysis appliance, you can scan files and see results of four types of analysis: network, static, community, and sandbox. If a file is malware, you can go to the Hosts view to see which hosts downloaded the file. (See the Malware Analysis User Guide.)
Users view	The Users view provides visibility into risky user behaviors across your enterprise using NetWitness UEBA. You can view a list of high-risk users and a summary of the top alerts for risky behavior for your environment, and then select a user or an alert and view details about the risky behavior, and a timeline during which the behaviors occurred. NetWitness Platform  users assigned the Administrators or UEBA Analysts role have access to this view. (See the NetWitness UEBA User Guide.)

Focus on Metadata, Query, and Time

The following figure depicts the workflow for an investigation with focus on metadata, a query, and time range.

Analysts use Investigate to hunt for events that drive the incident response workflow and to do strategic analysis after another tool has generated an event. Beginning in the Navigate view, Events view, or Legacy Events view:

1. Start by executing a query on a service for a specific time range, filter results to get a subset of events, reconstruct or analyze an event, and repeat the process to reconstruct or analyze another event. Built-in query profiles, meta groups, and column groups provide a good starting point. For example, you can choose the RSA Email Analysis query profile to see only metadata that is useful when investigating email risks.
2. When an event bears a closer look, view the context around the event, and decide whether to create an incident or add the event to an incident. If you decide not to add the event to an incident, you can run another query to gain further insight, which starts again at the beginning of the workflow.
3. If you notice suspicious activity or files on a specific host in the network, gather additional information about the host and files found on the host in the Hosts and Files view, or in a standalone NetWitness Endpoint server.
4. If you find a file or event that potentially contains malware, do a Malware Analysis scan of the file or open Malware Analysis and start a scan of the service on which the event was seen.

Here is one simple use case: If there is a concern regarding suspicious traffic with certain countries, the Destination Country meta key reveals all destinations and the frequency of the contact. Drilling into those values yields the specifics of the traffic, such as the IP address of the originator and the recipient. Checking other metadata can expose the nature of attachments exchanged between the two IP addresses. When suspect IP addresses are identified, looking at the addresses in the Navigate view or Events view with a broader time range can provide clues about what happened before and after the event being investigated.

Another use case is to investigate an alert to discover a malicious insider in the network who is exfiltrating intellectual property or other sensitive data from a specific IP address. The investigation begins with this meta value: Upload without change request followed by download alert. Start in the Navigate view or Events view by filtering the values to the IP address during the time range in which the alert was generated. Alerts metadata shows risk indicators as meta values, and you can click on different meta values to filter the events list and then reconstruct the event. Next extract files and examine the files to understand what happened. With this information, you can filter on the same IP address and broaden the time range to see activities before and after the event.

Focus on Respond View Incidents and Alerts

An analyst who is working on an incident or an alert in Respond can open the incident in Investigate to do a deeper analysis of the event or alert.

• The workflow to respond to an incident typically begins in the Respond view, where the analyst who is investigating an incident needs to gather intelligence about the incident in Investigate. Hover over an underlined entity in an incident or alert, such as an IP address, and select the action Pivot to Investigate. The Events view opens and is filtered for the selected entity. Defined meta keys are queried and the captured packets, logs, and endpoint events are displayed in the Events view.
• If you find events that are relevant to the incident, add the events to the incident in Respond. You can also create a new incident in Respond based on one or more events found in Investigate.
• From the Incident Details view Indicators panel in Respond, open the Events view to get a better understanding of an indicator event.

NetWitness Investigate Views

This section provides a brief description and example of the Navigate view, Events view, and Legacy Events view, as well as the context information, event details and reconstructions available in those views. Refer to the Malware Analysis User Guide for information about features and functions of the Malware Analysis view.

Navigate View

Note: By default, the Navigate view is disabled as the Events Meta Panel in the Events view provides this functionality. To enable the Navigate view, see Configure the Navigate View and Legacy Events View.

The Navigate view provides the capability to drill into and query metadata for network, logs, and endpoint events on a Broker, Concentrator, or Decoder (though investigating a Decoder is not typical). For certain configured meta keys, such as IP address, or hostname, you can see additional context information around a value using the Context Hub. The Navigate view also provides a sequential visualization of the data in a timeline. This figure illustrates the Navigate view.

• Each meta key listed displays the top 20 values based on how many events have those values.
• Drilling into the meta values by successive left or right clicking of values applies each clicked value as an additional filter to your query. As you drill, the subset of metadata seen grows smaller based on the filters you have applied. For example, if you filter to only show HTTP (service=80), all the remaining metadata presented will be what is contained inside those HTTP events.
• With a smaller set of refined results, you can go to the Events view to examine further event details, or perform other lookups inside or outside the platform to gain further insight.

  

Events View

The Events provides the ability to view events sequentially, analyze raw event data and metadata, and (in Version 11.5) drill into metatdata as you can in the Navigate view.

• In the Events panel, Network, endpoint, and log events are listed in order by time. You can view the raw event, filter, sort, search, look at details and reconstructions, and download events. Clicking an event opens the Event Details panel for the event. Different reconstructions are available in the Events view (packets, text, files, email, and web) with helpful cues to identify points of interest, such as interesting bytes, file types, and encoded data.

• In the Event Meta panel, you can view related metadata for an event that is open in the Event Details panel. Analysts reviewing the metadata can change the order of the metadata to better track down what they are looking for. The items in the list of metadata can be grouped by the sequence they were generated or alphabetically.
• In the Filter Events panel, you can drill into meta values for the listed events with your actions reflected in the Events panel. When the Filter Events panel is expanded to the full width of the browser, you can drill into meta values to hunt for specific information before listing the events in the Events panel (comparable functionality to drilling into data in the Navigate view).
  (Version 11.5.1) In the Filter Events panel, the meta values result threshold is 100000. If results are above the threshold, it is indicated using either ~ or >. For example, (>100000) indicates that the results are sorted based on count and are greater than the threshold. Similarly, (~100000) indicates that the results are sorted based on size and are greater than the threshold.
• In the Event Details panel, you can view details of a network, log, or endpoint event and safely reconstruct an event in a format similar to the original format. The tabs are Text, Packet, Files, Hosts, Users, Email, and Web.
• From various points in the Events view, you can pivot to standalone Endpoint, look up in Live, and do other internal lookups. External lookups allow you to search the internet for meta values with which you interacted, determine passive DNS information related to an IP address, check if a URL is blacklisted, and other third-party context integrations.

• If network data is enriched with endpoint data from an Endpoint deployment and the Endpoint Agent is configured for expanded network visibility, host information for network events is also displayed in the Events view in the header and in the Hosts tab.

  Note: For Expanded Network Visibility to work, ensure the service user account used for aggregating Endpoint Log Decoder data to Endpoint Concentrator is assigned with the decoder.manage permission. For more information on how to assign roles and permissions, see "Services Security View - Aggregation Role" in the Hosts and Services Getting Started Guide for NetWitness Platform.

• For certain configured meta keys, such as IP address, or hostname, you can search for additional context information around a value using the Context Hub. The additional context may include incidents, alerts, Threat Intelligence, and other sources where the value was mentioned.
• You can export different types of data. When viewing files, you can export files in a zip archive to your local file system. When viewing email reconstructions, you can download attachments. You can download logs from the text reconstruction, and export packets from the packet reconstruction. You can download multiple events from the Events list.
• (Version 11.6) To view the download jobs, click . Unlike the icon in the Legacy Events and Navigate page, it does not open the Jobs tray. It opens the Jobs page where all jobs are listed. To display the Jobs tray, go to Investigate > Navigate (Version 11.5 and earlier) or Investigate > Legacy Events (Version 11.3 and earlier), and click the (Jobs) icon.

This figure is an example of the Events view with a network event selected in the Events list, analyzed in the middle panel, and related metadata in the right panel.

This figure illustrates the Events view with the Events Meta panel open on the left side to filter the results set by drilling into meta values.

Legacy Events View

The Legacy Events view was the original user interface for analysts examining raw event data. The Legacy Events view is no longer needed in Version 11.4; it is hidden unless the administrator enables it as described under "Configure Investigation Settings" in the System Configuration Guide. When the Legacy Events view is enabled, both the Events view and the Legacy Events view are visible in the menu bar. The Legacy Events view provides a view of packet, log, and endpoint events in list form so that you can view events sequentially and reconstruct events safely.

• You can open the Legacy Events view for a meta value that you see in the Navigate view.
• For analysts without sufficient privilege to navigate a service, the Legacy Events view is a standalone investigation view in which analysts can access a list of network, log, and endpoint events from a NetWitness Core service without having to drill down through metadata first.
• The Legacy Events view presents event information in three standard forms, a simple list of events, a detailed listing of events, and a log view.
• For certain configured meta keys, such as IP address, or hostname, you can see additional context information around a value using the Context Hub. The additional context may include incidents, alerts, and other sources where the value was mentioned.
• You can export events and associated files, and create an incident from an event.

This figure illustrates the Legacy Events view.

Contextual Information for an Event

In the Navigate view, Events view, and Legacy Events view, the Context Lookup panel shows details about elements associated with an event in the Context Hub for these meta types: IP Address, User, Host, Domain, MAC Address, Filename, and File hash. In addition, you can right-click all meta keys except time to see additional context.

You can interact with the elements of an event to get further insight including related incidents, alerts, custom lists, Archer assets, active directory details, NetWitness Endpoint IIOCs, and STIX data sources (namely File, TAXII Server, and REST Server). (See Look Up Additional Context for Results.)

Note: Archer assets and active directory details are available in the Events view context lookup. Endpoint context lookup is available for NetWitness Endpoint 4.4.0.2 or later hosts, but not for the NetWitness Endpoint hosts.

The following figure shows the Context Lookup panel, which opens to the right of the Events panel in the Events view.

The following figure shows the Context Lookup panel, which opens to the right of the Events list in the Legacy Events view.

Reconstruction and Event Analysis

When you discover an event that merits additional investigation, you can analyze it in a form that best suits the content of event. Some forms of analysis involve safely reconstructing the event in a form similar to its native form, for example, packets, text, email, and web content. The rendering of events restricts the use of dynamic or active code that may be contained in the event to limit adverse outcome to your system or browser. Using cache improves performance when viewing previously viewed events. Each analyst has a separate cache of reconstruction data, and you can only access reconstructed events in your own cache.

Some network events are enriched with host data if you have an Endpoint deployment and the Endpoint Agent is configured for expanded network visibility. For such an event, you can view the host details.

The Events view gives you the ability to interactively analyze an event, looking at raw data, meta keys, meta values. This figure is an example of a network event rendered as packets in the Events view.

The Event Reconstruction in the Legacy Events view presents the raw data and the meta keys and meta values for an event in a list form. This figure is an example of the Event Reconstruction.

Previous Page	Next Page