Rebuilding the Index

Under normal operation, changes made to the index configuration for a service are only applied to new data that enters the collection. Rebuilding the index over all the data in the collection is a time-consuming process because it requires all of the meta database storage to be read from disk.

In latest version, it is possible to rebuild the index while the service is online. Current version services rebuild indexes in the background whenever the service detects that part of the session and meta databases is unindexed.

Activating the Background Reindexer

The background reindexer is activated whenever the service starts. During startup, the indexer checks for gaps between sessions that are indexed and sessions that are present in the session and meta database. If any gaps are found, the background reindexer begins reindexing the session and meta database on the service.

Examples of events that may activate the background reindexer:

  1. A power failure or crash occurred, rendering the last slice of the index corrupt. The corrupt data is deleted at startup, leaving a gap in the index.
  2. Index data is forcibly deleted, either by doing an index reset or deleting files from the filesystem.

Controlling the Background Reindexer

The operation of the background indexer is controlled by the configuration node /index/config/reindex.enable . If reindex.enable is set to true, the next time the service starts the reindexer will operate. If reindex.enable is set to false, the reindexer will not start the next time the service starts, but will continue to operate until the service is restarted.

Background Reindexing Algorithm

The operation of the background indexer is as follows:

  1. The index examines the ranges of sessions that are present in the index and compares them to the ranges of sessions that have valid meta data. Any discrepancies between the two are considered gaps.
  2. The gaps in the index are subdivided into slices based on the current value of /index/config/save.session.count .
  3. For each slice that is missing, a temporary index is created in one of the directories specified by /index/config/index.dir . The slices are reindexed in reverse numerical order. Thus, the most recently collected sessions are indexed first.
  4. Once the slice is completely reindexed, it is moved into its valid location in the online index. If the reindexed slice belongs on the warm tier, it is moved to the warm tier.
  5. The newly indexed data appears as part of the collection.

Background indexer status

The stat node /index/stats/updater.state indicates the current state of the background reindexer. This node will say running , not running , or failed . If the status is failed , check the service log for more diagnostic information.

Effects on Aggregation

Services that perform aggregation utilize the index to keep track of sessions that have already been aggregated. If the index does not have enough information to begin aggregation, aggregation will be offline until enough slices have been reindexed. During this time the aggregation status for the upstream device will indicate that it is waiting on aggregation.

Forcing A Reindex

To force the index on a service to be rebuilt:

  1. Ensure that /index/config/reindex.enable is true.
  2. Reset the index by using the reset message on the service. For example: /concentrator/reset index=1 will restart the service and delete all the index files.
  3. Wait for the service to restart. Background reindexing will start.
  4. The most recently collected data will be available for queries as soon as the index slice representing those sessions has been reindexed.