Configure Session Split TimeoutsConfigure Session Split Timeouts
The default behavior of the Decoder is to automatically end sessions that exceed a configured size or have been inactive for a period of time. When the session is ended due to timeout, any subsequent packets received in that session appear to be stored in a new session. You can mitigate the effect of session splitting due to long periods of inactivity between packets using this procedure.
When a Decoder session exceeds a configured size (32MB by default, the /decoder/config/assembler.size.max) or has been inactive for a period of time, the session is split. NetWitness has the previous packet and the next packet and can propagate session state from the initial session fragment to the subsequent session fragment.
Each session fragment is annotated (session.split meta) such that it can be identified and associated with other fragments from the actual network session. Directionality as determined by the initial session reduces the occurrence of fragments having reversed directionality.
If there is a gap in time between packets large enough that there are no longer any packets for the session in memory, the session is removed from the Decoder. If a subsequent packet shows up after this occurs, a new session is created with no context to the preceding session. The issue is the inability to continue a session when we encounter a gap between packets of a session that is larger than the packets we are buffering (based upon available memory and timeout configurations). Once the last packet of a session is removed from memory, the session is also removed, and with it the necessary context for ensuring consistent directionality.
There are two timeout settings in a Network Decoder, /decoder/config/assember.timeout.session and assembler.timeout.packet. Both default to 60 seconds. The setting assembler.timeout.session controls how long a session lives in Assembler without receiving another packet. The setting assembler.timeout.packet controls how long a session waits before getting parsed. If the session is kicked out of Assembler before this timeout, then it automatically goes to parsing.
The session timeout is the number of seconds since the last packet was added to that session. Therefore, this timeout resets on every packet added to that session. The packet timeout is the number of seconds since the very first packet for that session was added (in other words, the packet that created the session). This is never reset and once the timeout expires, the session is parsed.
The important point is a session can be parsed but still remain in Assembler. A session in Assembler can still have packets added to it, even if it has already been parsed. Packets added after the session is parsed will never be seen by parsers, but they will be attached to the session and can be viewed by a subsequent /sdk content or /sdk packets call.
After a session is parsed, the session AND its metadata are written to disk. At this point, they can be aggregated and "seen" by sdk commands. Packets are written in order of capture and are not reordered by what session they belong to. Nor are they necessarily written when the session and meta data are written.
You can disable both timeout nodes, /decoder/config/assembler.timeout.session and assembler.timeout.packet, by setting them to zero in the Services Explore view.
If both timeouts are disabled, the sessions are still split due to time or size expiration. However, the Decoder keeps track of the network stream for as long as it has sufficient memory. Thus, when more packets arrive on the same network stream, the Decoder adds split meta items to the subsequent sessions. Using a combination of the split metadata and the stream key, it is possible to reconstruct the network stream from the multiple sessions.
The length of time for which sessions are tracked is limited by the number of session pool entries available on the Decoder, and therefore the actual time window varies according to the rate at which new sessions are added. If new sessions are added at a high rate, the size of the time window decreases. The size of the pool is set using the configuration entry /decoder/config/assembler.session.pool, which sets the maximum number of sessions that will be tracked at a time.
The /decoder/stats/assembler.timespan statistic allows you to see when the Decoder is no longer tracking session splits because the ingest rate is too high and the Decoder does not have enough memory to track. This statistic shows the number of seconds tracked within the session table, which is the effective time window in which the Decoder can link together sessions. Under normal operation this statistic matches the value of /decoder/config/assembler.timeout.session, but when running in Time Split mode, the /decoder/stats/assembler.timespan statistic grows or shrinks depending on the ingest rate.
To configure Time Split mode, set the following configuration parameters and restart the Decoder:
- In the
(Admin) > Services view, select the Decoder service and
> View > Explore.
- In the Services Explore view select decoder > config.
- Click in the Value column next to the parameter and set these two parameters :
/decoder/config/assembler.session.flush = 0
/decoder/config/assembler.timeout.session = 0 - To see when the Decoder is no longer tracking session splits because the ingest rate is too high and the Decoder does not have enough memory to track, view the /decoder/stats/assembler.timespan statistic, in the Services Explore view select decoder > stats.
