Event.time index and search - Bad idea?

[MaxBoaventura]

There are many reasons to execute a search over the event.time values (I'm talking about reporting basically).

In the last two years using the SA I've faced the same situation many times and the solutions were always provided by some king of inefficient search over the raw log.

After open a case I received the following answer and I can't see this as the final word. The answer is also now an article in the knowledge base (000032469 - RSA Security Analytics - How to index "event.time" meta if its required).

"Technically it is possible to index event.time but it create problems with the index. What happens is basically having unique values created in your index which will grow massively so leading to performance issues and a lot of other side effects. "

Following is advised to index the meta on IndexKey level that is useless in my opinion in this case.

What I just can't accept is the fact we are talking about the "same" time already indexed by the solution. Same I mean datatype and not the same meta.

I see the point as a DATEANDTIME/ TIMESTAMP field to the indexed. Simple like that.

index-concentrar.xml

       <!-- time needs to always be indexed at value level -->

        <key description="Time" format="TimeT" level="IndexValues" name="time" valueMax="0" />

[DavidWaugh1]

Hi Max,

Thanks for writing on the community. Can you give an example of what you are trying to achieve here? If I understand correctly (and I might not here!) you have a difference between the time a log is generated (the event time) and the time that the log is actually processed on the log decoder? Is this what is causing you to use event time?

Normally the two should be close enough together as not to cause too many problems, but I think you have a specific case that I want to understand further to see how we can help.

[MaxBoaventura]

Hi David,

Thanks for your help.

Situation1:

Problems in the event source. After 3 days the situation is under control.

After 6 months you look for a event happened minutes after the failure. You won't find the event on the day.

Situation2:

Day 0 of a event source. Import everything or just the new ones?

Import every event bring us to the same situation. The day of SA is day 0. Period.

What about all those events happened before the the SA? Discard is not an option and search them is a must.

Those are two in many in my opinion.

[DavidWaugh1]

Hi I know product management are working on a solution to this. Maybe they will be able to comment. In the short term though I think you will need to index event.time.