nwindex file size

[NaushadKasu1]

On a Concentrator, the *.nwindex files, what is the difference  between ones that are 448 bytes and 1048576 bytes?  I believe both are not indexed (I have many other nwindex files that are very large i.e. indexed).  Just curious the difference between these 2 sizes for what I believe are both not indexed keys (or don't have values).

[EricPartington]

small *.nwindex files I think indicate that there are less (or no) values for that metakey for that slice (period of time/sessions).  I would cross reference the name of that index with what is in your concentrator(-custom)-index.xml and see what the index level is set to (none, keys,values) as well as investigator on that key and see how many unique values of data exist in there.  You could compare that to a key that have a large *.nwindex size and see if the larger index file equates to more values in it (i think it does).

[NaushadKasu1]

Thanks for the response Eric.  I suspect that the 448 may be for keys that are not indexed i.e. IndexNone while the 1048576 could be IndexKeys level?

[EricPartington]

Exactly

[ShawnOstapuk]

I think that is not correct. Meta that is IndexNone should not have an nwindex file at all. IndexKey is 448 bytes and IndexValues will vary depending on how many unique values there are. The actual list of sessions associated with each key is stored in page.db and the actual values for IndexValues is stored in summary.db

[NaushadKasu1]

Shawn,

What you're saying makes sense.... but I find it coincidental that many keys are of (exact) file size 1048576.  I'm fairly sure that 448 bytes is IndexKeys and I see ip.src and other large keys with several megabytes in size but many keys are still 1048576.  Trying to understand exactly what a key with size 1048576 means.

[DougLaing]

I thought that the 1048576 size files were in use.  They include whitespace until they are closed out and the next file is created.  If they end up empty they are likely are small, but exist which could account for that 448 bytes size.  Someone please correct me if I am wrong about that one...

[KennyKim]

There is another way using REST API to retrieve relevant information of the meta key.

You might know 'index-concentrator.xml' or 'index-concentrator-custom.xml' where it contains definitions of index level for each meta key.

If the .nwindex' filename is ip.dst, you can check and retrieve information in the index file by issuing this command.

1. Go to administration > services > concentrator > view > explore

2. select 'index' node, populate properties dialogue by right clicking on the node

3. select 'inspect' method among drop-down list.

4. type key=ip.dst in Parameters text field and click the Send button.

Seeing the response, you can't miss how many keys or values indexed at end of the response.

totalKeys=1,

totalValues=106265

H0pe this is useful for you.

Thanks.