Rollover

This topic describes the two rollover mechanisms. The database operates as a first-in, first-out (FIFO) queue. New data is always appended to the database, and the oldest data is automatically removed as needed. Data that is in the middle of the database is immutable, meaning it cannot be modified.

There are two mechanisms to for rollover: synchronous and asynchronous.

Synchronous Rollover

Synchronous rollover refers to rollover settings that are applied in response to a write operation on the database. That means data is removed from the database in direct response to the need to write new data. Synchronous rollover is configured by setting size values on the configuration for packet.dir , meta.dir , session.dir , and index.dir .

Synchronous rollover on the packet, meta, and session databases can occur within any write operation. Synchronous rollover on the index occurs when the index is saved.

Asynchronous Rollover

Asynchronous rollover refers to database file removal that occurs when an explicit rollover command is issued to the database. Most commonly this type of rollover is scheduled to run periodically using the built-in scheduler of the Core service. The user can also explicitly request it.

The asynchronous rollover command is the sizeRoll message present on the /index and /database nodes of the configuration tree. The message on the /database node does size rollover on packet, meta, and session databases only, while the message on the /index node can do simultaneous rollover on both the index and the packet, meta, and session databases.

The sizeRoll command has the following parameter syntax: 

				
size-roll-params    = {type-param, space}, (max-size-param | min-free-param | max-percent-param), {max-size-warm-param, space}
type-param          = "type=", {type-flag} , { ",", type-flag } ;
type-flag           = "packet" | "meta" | "session" ;
max-size-param      = "maxSize=", number, {space}, unit ;
max-percent-param   = "maxPercent=", number, {space}, unit ;
min-free-param      = "minFree=", number, {space}, unit ;
max-size-warm-param = "maxSizeWarm=", number, {space}, unit ;
unit                = "t" | "TB" | "g" | "GB" | "m" | "MB" ;
number              = ? decimal number ? ;
percentage          = ? number between 0 and 100 ? ;

The type parameter controls the databases to consider for removing the oldest data based on total size or space remaining. If type is not specified on the /index sizeRoll, only the index is considered for rollover operations.

The maxSize parameter sets a current maximum size of the database or index. If the database is larger than this size, oldest data is deleted first (or moved to the Warm or Cold tier, depending on the configuration) until total size is less than maxSize . The sizeRoll operation determines which data is oldest out of all the databases and the index based on session IDs. Sessions or index entries with lowest session IDs are deleted first, possibly including removing meta and packet databases that are orphaned by removing entries from the session database. The index data is rolled out if the sessions that it refers to are removed.

The maxSizeWarm parameter sets a current maximum size on the Warm tier, but otherwise behaves identically to the maxSize parameter. When data is rolled out on the Warm tier, it is moved to the Cold tier (if configured) or deleted.

The maxPercent parameter sets a maximum percentage of all the volumes of all databases passed in type parameter combined. When exceeded, oldest data is deleted first until total size is less than maxPercent of total volumes.

The minFree parameter sets a minimum allowed free space on the volumes before oldest data is deleted.

Each call to the sizeRoll operation provides a single pass through the database to delete files. When the operation completes, the current size utilization of the database will have met the criteria specified by the maxSize , maxPercent , or minFree parameters and the optional maxSizeWarm . Therefore, this operation can be scheduled periodically to ensure that the database can continue to operate uninterrupted.

Example

The following example shows a typical sizeRoll scheduler entry for an Archiver:

				
pathname=/index minutes=5 msg=sizeRoll params="type=meta,session,packet maxSize=25TB maxSizeWarm=150TB"

This scheduler entry specifies that every five minutes the database ensures that the max size of the meta, session, packet, and index does not exceed 25 terabytes on the Hot tier and does not exceed 150 terabytes on the Warm tier.