Group Events from Split and Related Sessions in the Events and Legacy Events Views
In the Events view listing of events, events from split and related sessions are listed in the order they were parsed; for this reason, they are not always listed together in the Events list. In Version 11.4.1 and later, the Group Events option allows you to change the order in which the events are listed so that you can more easily detect relationships in the captured data. When events are grouped, we refer to the first event as the leading event.
The user interface is designed to help you identify grouped events. Solid row lines delineate different groups of related events while dotted lines depict the events that are part of the same related group. In a group of events, the leading event is first and subsequent events are nested under the leading event with indentation and relationship icons for subsequent events. The numbers next to the relationship icon distinguish the session split count.
If the leading event is not included in the current data set, the subsequent events are still grouped together under the first subsequent event. Only the leading event or the first subsequent event, in the case of no leading event, is sorted; indented events are not sorted. Hovering over the subsequent event marker (
If events are related based on session fragments, when you select a subsequent event and open the reconstruction, you can see the session.split meta key in the Event Meta panel.
Split Network Session
If you see a tool tip like this, the listed events are part of a split network session:
The event is part of a split session (session.split: #) matching these parameters: ip.src=127.0.0.1 AND ip.dst=127.0.0.1 AND tcp.srcport=25 AND tcp.dstport=1234.
The reason for the split is one of the following:
- The original event was split into sub-parts by creating additional events for each transaction inside the original event.
- The original session was split as it was ingested into the Network Decoder because the size was larger than the Assembler Maximum Size (default=32 MB).
- The original session was split as it was ingested into the Network Decoder because the time was longer than the Assembler Timeout Session (default=60 sec).
Session Size and Time Split
Network Decoders are configured with a default session size (assembler.size.max) and a session timeout (assembler.timeout.session). The configuration is described in "Configure Session Split Timeouts" in the Decoder Configuration Guide. When a session exceeds either limit, the Network Decoder splits the session, and all subsequent packets become part of a new session, fragmenting the actual network session across multiple Network Decoder sessions. To process session fragments with the context that they are fragments of the larger network session and to improve association of source and destination addresses, ports, and application protocols, Network Decoders parse contextual fragments and highlight session fragments.
Note: In the Legacy Events view, you can find session fragments and export all packets viewed in the Events list to a single PCAP as described in Find and Combine Fragments in the Legacy Events View.
The Network Decoder completes session parsing before splitting the session based on the configured maximum session size (default = 32 MB) or the configured timeout (default = 60 seconds). When parsing is complete, the parsed results include the proper address directionality and application protocol, which are propagated to each subsequent session fragment to ensure consistency with the logical network session they represent.
Transaction Handling Split
Administrators can configure a Network Decoder to subdivide incoming sessions into smaller transaction sessions when using Lua parsers that are designed to create transactions. The configuration is described in "Configure Transaction Handling on a Decoder" in the Decoder Configuration Guide. The Decoder service configuration node has a parameter that controls the behavior of the Network Decoder when a parser defines a transaction within a network session: /decoder/parsers/config/parser.transaction.mode. If the mode is set to split, a large session with multiple application-level transactions is split when a parser generates an application-level transaction such as an email. An example of this is a large session containing multiple emails. For each email (transaction), a new session item (split session) is created, network meta items are copied into the new session, and meta items marked in the transaction are copied from the original session to the new session.
Transactions require a parser update to function, and initially they only support SMTP and HTTP pipelining use cases. This is an example of the reconstruction of an email that is separated based on the individual emails inside the original event. Each transaction highlights a single email and the metadata associated with the transaction is only related to that email. To provide this functionality, the original packets are stored on the Network Decoder as usual for a network event, but the new related transaction events are created on the Concentrator. As a result, analysts see visual queues in the user interface, and they also have the ability to query to find only specific emails or attributes of emails that were previously all bundled together. To eliminate the original event from the query results, the session.split meta key has been indexed. When there are transaction splits, the original event does not have that meta key associated with it while all the related transactional events do.
Session Fragments Highlighting
Both types of session fragments have an additional meta key: session.split. The earliest fragment is session 0 and sessions with a later time stamp are incrementally numbered 1, 2, 3, and so on. The session.split meta key indicates the number of preceding sessions fragments; however, it does not always indicate that there are subsequent session fragments, even with a value of 0. It is also possible for the first fragment of the session to not have session.split metadata if the session is parsed before exceeding the maximum session size.
Transaction splits start with a session.split value of 1. When viewing sessions, the session.split meta key clearly identifies sessions that are fragments in the Events view and the Legacy Events view (Events List view and the Events Detail view).
If this was a session size and timeout split, you can view the session fragments and determine the maximum session size or session timeout necessary for parsing to combine the split sessions into one again. For example, if you have four fragments at 32 MB, you need to configure your test Decoder (usually a virtual machine set up separately from the production service) with a maximum session size greater than 128 MB. The steps are the same to find all fragments based on a session timeout.
Related Network SessionRelated Network Session
If you see a tool tip like this, separate events processed by the Network Decoder share a set of four values that identify the IP source, IP destination, source port, and destination port:
The event is related to a previous session matching these parameters: ip.src=127.0.0.1 AND ip.dst=127.0.0.1 AND tcp.srcport=25 AND tcp.dstport=1234"Second category: Related Network Session
In this case, the Network Decoder has not inserted a split, and there is no session.split metadata associated to any of the events. The reason these events grouped together is to highlight the events that are worthy of scrutiny based on a pattern. Each event has the same source IP address, the same destination IP address, the same source port, and the same destination port. The grouping of related events does not occur if any of the four meta keys is obfuscated for data privacy reasons.
These are the combinations of meta keys that must match to categorize an event as a Related Network Session:
- ip.dst, ip.src, tcp.dstport, tcp.srcport
- ip.dst, ip.src, upd.dstport, udp.srcport
- ipv6.dst, ipv6.src, tcp.dstport, tcp.srcport
- ipv6.dst, ipv6.src, upd.dstport, udp.srcport
In Version 11.6, the group events option that retrieves events from logs, network and endpoints is disabled when data privacy is enabled. For example, if the 'ip.src' is blacklisted or restricted by the admin, then the option shown in the following image is disabled.
This option is disabled if one or more of the following 12 metas are restricted:
'sessionid'
'nwe.callback_id'
'medium'
'session.split'
'ip.dst'
'ip.src'
'ipv6.src'
'ipv6.dst'
'tcp.dstport'
'tcp.srcport'
'udp.dstport',
'udp.srcport'
Use Cases for Viewing Events from Split and Related Network Sessions
These are examples of practical use cases for viewing events from split sessions:
- A Network Decoder that is inline with a proxy server receives a lot of email connections that get bundled into a single session based on event time as NetWitness sees them. The single session has multiple meta values for subject, email.src, email.dst, and other meta keys relevant to email, which are difficult to map together correctly. Seeing the session organized as leading and subsequent events, gives the analyst a clear idea of the details of each email.
-
An analyst is attempting to understand what IP address out of all the metadata associated with a session resulted in the generated metadata or an alert, but the IP address is not in the output. For example, a feed that is parsing for an indicator of compromise may have many triggers in a session that has many IP addresses. By viewing the complete event organized as leading and subsequent events, the analyst can understand which IP triggered the alert.
- An analyst wants to know which file was deleted from which directory versus which file was read out in which directory, but the session encompasses multiple files and directories. For example, an HTTP connection with commands: directory /keep/, directory /temp/, filename foo.txt, filename me.doc, action delete, and action read. Viewing the leading and subsequent events tells the analyst that /temp/me.doc was deleted and /keep/foo.txt was read. Now the analyst or analytics can form an opinion about how bad these actions really were.
-
An analyst is attempting to retrieve a large file related to an event that triggered a suspicous alert. However, the file that was transferred was so large that the network decoder split it into 100 separate sessions. When viewing this group related split sessions, the analyst can download a PCAP of the sessions, then extract the original file by either running it through a Decoder with larger assembler settings or a third-party tool.
Show and Hide Relationships in the Events List
For both types of related events, you can see the relationship of events in the Events view Events list. When the Events list is first displayed, you can tell if set of results includes related events by looking at the Group Events switch at the top of the Events list. If the results do not include related events, the switch is dimmed as shown in the following figure.
To look for related events in the Events list
- Go to Investigate > Events and submit a query.
If the results include related events, the Group Events switch is active, but not enabled. The figure below illustrates a set of results that include split sessions, and the Group Events switch is disabled. The related events are not nested. - Click the Group Events switch.
Related subsequent events are nested below the leading event. The subsequent events are indented and marked by an icon. Clicking on the icon explains why the event is grouped.
Find and Combine Fragments in the Legacy Events ViewFind and Combine Fragments in the Legacy Events View
From within the Legacy Events view, you can find fragments of a session using the Refocus > Find Session Fragments context menu option. NetWitness composes a query using the source and destination addresses and ports of the selected session and displays all sessions that match that query within the current time window.
To find session fragments
- In the Legacy Events view, right-click any of the source and destination address and port values: ip.src, ip.dst, ipv6.src, ipv6.dst, tcp.srcport, tcp.dstport, udp.srcport, and udp.dstport) as well as session.split values.
The context menu is displayed. - Select Refocus > Find Session Fragments or Refocus New Tab > Find Session Fragments.
NetWitness repopulates the Events list with session fragments for a single session within the current time range. Depending on the option you selected, the refocus replaces the current view or opens in a new tab. (All data is used in these examples but is not recommended on production systems). - If necessary, adjust the time range to include any session fragments that may precede or follow the current time window. You can tell that the time range needs to be expanded if the fragments occur near the time boundary, especially if the first visible fragment does not have a split value of 0 (or none). Alternately, inspecting the packets of the last visible session may lead you to believe that the session continues. Here is an example:
- If you are looking at fragments that are obviously not the first fragment, for example, 1, 2, 3, and 4 in time range 10:30 to 10:35, there should be a fragment 0. You can increase the time range to start earlier (for this example, 10:25) to find the additional fragment.
- If the session size of last fragment is close to maximum session size (12 MB in this example), look for additional fragments by increasing the time window to include a later time (for this example, 10:40).
When all of the session fragments of a network session are included within a single Events list, the list can span multiple pages.
- (Optional) To export the packets for every session fragment to a single PCAP file, select Actions > Export All PCAP.
A message informs you that the PCAP is being downloaded. When download is complete, PCAP file includes the entire network session that was fragmented.
