NwConsole makemeta msgpack format

[RichardB]

We find the NwConsole makemeta command (which is not described in the User Guide) quite useful. It has an output format msgpack that looks interesting. What tools can parse this msgpack format?

Is there a way to export metadata for long time storage so it can be re-ingested and analyzed again later?

The NwConsole makemeta help:

RSA NetWitness NextGen Console 11.5.3.0
Copyright 2001-2021, RSA Security Inc.  All Rights Reserved.

Type "help" for a list of commands or "man" for a list of manual pages.
> help makemeta
Usage: makemeta {source=<pathname>} [dest=<pathname>] [filenum=#[-#]]
                [metaid=#[-#]] [format=<text,json,xml,msgpack>]
                [delimiter=<string>] [eol=<string>] [header=<true,false>]
                [delete=<0,1>]
Convert meta database files to text, json or xml formats.  NOTE: Meta in the DB
is stored in a very compressed format.  Converting to any of these formats
will require space many times greater than the size of each meta DB file.  It
will also take some time.

    source    - Required, the directory where the meta db files reside
    dest      - The destination directory for output files, uses source dir by
                default
    filenum   - The meta db file numbers to operate on or range of file
                numbers, use 999999999 for no limit on upper range
    metaid    - Only extract the meta IDs within the specified range
    format    - Output format type
    delimiter - For text format, the field delimiter (defaults to tab \t)
    eol       - For text format, the record delimiter (defaults to newline \n)
    header    - For text format, outputs a header at the start of each file
                (defaults to true)
    delete    - If true, deletes the db file after processing, default is false

[JoshRandall · Accepted solution]

You might find more information here: https://msgpack.org/index.html

[RichardB]

Thanks! So it actually is a standarized format, like BSON. When exported in msgpack format, can any NetWitness tools read it again later?

[JoshRandall]

This is all just going to be the metakey=metadata information, and cannot be imported back into NW.

If you wanted the information available again in NW (assuming it's rolled off of storage), you would need to replay/re-ingest the original raw logs/pcaps.

[RichardB]

Ok, so when the raw logs/pcap are gone because of retention and/or disk space and we want to preserve the metadata for prosperity (and later analysis) what is the recommended method in NetWitness?

[JoshRandall]

Any of the makemeta options can be used to export the meta for longer term retention. You can also export meta from the UI.

The key here, though, is that exported meta is not re-importable into NW.