Is there anyway to prevent duplicate meta keys being generated for a session?

[david_waugh]

Sometimes in a session the same meta keys and values are generated.

For example if you had a session that contained multiple jpg files, the forensic fingerprint meta would fire for each jpg file in the session.

There doesn't seem to be much to gain from this,as your would have multiple 

filetype = 'jpg'

saved for the session.

If this is repeated many times over many sessions then eventually this has an issue with packet retention as you are saving duplicate meta per packet over lots of packets.

Is there a way that if a session has already been tagged with MetaKey MyMeta=MyValue then further occurrences of MyMeta=MyValue are not registered?

[david_waugh]

For example just looking and the inv.context key seems to contain lots of duplicate information

[EricPartington]

I don't think that the impact on storage is as big as it appears.  per slice I think you will only have one value in the metadb and the sessiondb will link that unique value to each session where it occurred.  I dont think we store each duplicate unique value in the DB.  For a value like checksum where the meta is random and highly unique it does not normalize well and in that case you will have a large index, but for entries where the values are common and not highly unique those should dedup nicely and reduce the on disk usage in metadb

[KaranDesai]

David Waugh wrote:

For example just looking and the inv.context key seems to contain lots of duplicate information

Hi David, I kindly request you to please elaborate this and explain in deep, It seems I have same problem. my bk experience  kroger feedback