In NWFL, we make use of transient meta, which is meta that is available for use by Feeds, App rules, and Basic Event Correlation Rules, but is not written to the meta database and therefore not available in Investigator or Informer. The reason for the use of transient meta is mainly because of the amount of meta produced per log message being much greater that the amount of logs. With packet data the amount of meta is about 10% of the amount of packets, but it is reversed with logs where each log might produce 8 times the amount of meta per log message. Therefore transient meta is used to help to reduce the amount of capacity needed for the meta database and also because much of the log meta that is generated is not necessarily needed for monitoring and investigative purposes.
However, we have made the assignment of transient configurable by the customer, with the understanding that as one moves from transient to non-transient, the amount of retention in the metaDB goes down. Here is how you would go and make the necessary changes:
In 9.7, you need to edit the table-map.xml file that is in /etc/netwitness/9.0/envision/etc directory.
In 9.8, same file but you find it in the /etc/netwitness/ng/envision/etc directory.
The table-map.xml file is a "map" that takes the enVision variable fields and maps them to an appropriate Netwitness Nextgen Meta Field. What you have to do is to go in and change the flags="Transient" to flags="None" for the variable/field that you are interested in. Here is an example:
Once you restart the nwlogdecoder service, from that point ON, the meta will be written to the meta database.
(NOTE: When you update your log parsers (enVision Content) via Live, your table-map.xml will get clobbered. To preserve your changes copy the table-map.xml to /etc/netwitness/9.0/envision directory and the nwlogdecoder service will use that table-map.xml rather than the one below it in etc).
If you want to be able to see the data in Investigator or want to be able to use it in the WHERE clause in Informer (i.e. the value does not need to be indexed to be used in the SELECT portion of the query, but if it isn't the query performance could be pretty bad.), you have to index the value or key of the meta field in question. This is generally done on the concentrator, by editing the index-concentrator.xml just like you would for packet meta. You may have to add the meta field or change from level="None" to level="IndexValues" or level="IndexKeys".
Regards,
Paul
Paul W. Stoecker, Ph.D.
Principal Software Engineer
Content Analytics Research Team
RSA, The Security Division of EMC
o: 508.599.2743 | c: 302.379.3375 | e: paul.stoecker@rsa.com<mailto:daniel.reich@rsa.com
