Moving Average

[RenatoGoncalves]

Hello guys,

I need to create a chart thats show traffic for a determined ip address in a time windows ( for example a week ) and that can give me have moving average. With that i can see the spikes of that average and i can investigate a possible DoS or DDos attack if the alerts rules doesnt work or even delete the existing rules.

Its possible to do that like splunk has?

[BobDredger]

Renato,

    If I've interpreted your question properly then, yes.

     I run a chart on my Dashboard that is monitored by all network defenders throughout the day to identify evidence of a DoS or DDoS. Unfortunately it only covers up to the last 36 Hours due to Chart limitations but since it is continually monitored this is more than sufficient for DoS identification.

       Essentially what I did was first created a Feed tagging all publicly accessible web servers (as they are most likely to be DoS'd). Lets say that all of those servers tagged in the feed populate to the "Web Servers" (webserver) MetaKey. I then created rules to show what services (service) are most frequently observed in conjunction with the webserver MetaKey in the Reporting Module. Based upon my example, the logic of that rule would look something like:

                SELECT = service

                WHERE = webserver exists

                SORT BY = descending

                LIMIT = 10

            Then with that Rule tested and validated.  I created a Chart using the Rule which "Charts Value Over Time". Then I loaded that Chart into a Dashlet. This produces a nice wavy graph showing the frequency at which the Top 10 most services are utilized. In the event of a DoS or DDoS. The sudden increase of traffic would be so prevalent that a clear spike would be indicated on the graph. We've been able to test this several times with great success. It also is useful in identify instances where people are running vulnerability scanners against our web servers. Due to the speed at which those products send requests, obvious spikes are indicated on the graph.

              To investigate the spikes, I make note of the time at which the spike peaked and the service which the spike was associated. I then pivot into the Investigate -> Navigate view and specify the time frame to be +/- 5 minutes of the peak and apply the advanced query:

                webserver exists && service = (Whatever the Service identified Common Port Number is)

                When that data loads, I open up the visualization to ensure I see a similar spike in the Investigative -> Navigate visualization model, verifying my time frame is accurate. I then analyze the most active Source IPs, usually one of the IPs has a much higher session count than the others and is the culprit of the traffic spike.

               One additional element I did personally was to essentially whitelist known good IPs from the initial Reporting Engine Rule. If I knew I had certain scanners or high traffic systems that were legitimate and may cause a false positive, I simply added ip.src != X.X.X.X. If you have several such systems, I'd recommend creating another Feed so that you can simply whitelist them by a meta condition such as, webserver != 'known scanners' or something.

                 I hope this helps!

V/R,

Bob

[RenatoGoncalves]

Hello Bob,

I gonna try that but i have doubt to put:

 "Essentially what I did was first created a Feed tagging all publicly accessible web servers (as they are most likely to be DoS'd). Lets say that all of those servers tagged in the feed populate to the "Web Servers" (webserver) MetaKey. I then created rules to show what services (service) are most frequently observed in conjunction with the webserver MetaKey in the Reporting Module. Based upon my example, the logic of that rule would look something like:"

How do create the feed?

Regards

[RenatoGoncalves]

Anyone has more ideas on how to create a moving average?

[RenatoGoncalves]

Hello Robert Dredger‌,

Can you help with you response? Did you create a metakey? Where?

[BobDredger]

All,

    Apologies for the delayed response.

     Yes, I created a custom MetaKey named "pfwserver" with a display name of "Public Facing Web Servers". This key should be created in the index-concentrator-custom.xml file. It can be found by browsing to Admin -> Services -> Concentrator -> Config -> Files. Select the "index-concentrator-custom.xml" file from the drop down list. It is very important that you create custom MetaKeys on the "index-concentrator-custom.xml" file and NOT the "index-concentrator.xml" file.

    In most cases, you can copy the syntax of an existing MetaKey and simply change around the Key Name and Display name to whatever you prefer. Make sure you add this key to the individual "index-concentrator-custom.xml" files on all of your other Concentrators. Once this is done, either restart the service or give the system a little while to re-load the indexs and the key will appear in the Investigator View. You can verify its presence by trying to add it to a new MetaGroup.

      Once your key is created, simply create a new feed using an excel file with two columns. Column one should be a list of IP addresses associated with your web servers and column two can be the plain text name of that specific server. Column two will end up being the MetaValue that populates under the MetaKey of "Public Facing Web Server".

       Feel free to let me know if you have any other questions and I will make sure I respond in a more timely manor!

V/R,

Bob