Decoder Log Stats - CSV extract

[KeithDixon]

Recently I stumbled across the Log Stats tab on my decoders Stats page. When I discovered this, I thought "This is great information, but how can I REALLY make efficient use of this as I am working to put the final nails in the coffin of Envision?"

REST API...

With a few email exchanges and such, I found it was possible to extract the information. But to get it into a consumable format the challenge was on.

Here is the final "sanitized" results of my script. A caveat first;

• A coder I am not. My best coding comes from "Google plagurism" (piece together the right pieces from Google searches)

I have listed two options, for SSL and non SSL decoders.

All values inside < > are specific to your environment.

curl -k -u admin:<admin password> "https://<Non SSL decoder1 of choice>:50102/decoder/?msg=logStats&force-content-type=text/plain">Output1.csv

sed -e 's/forwarder= /local/' -e 's/device=//' -e 's/source=//' -e 's/count=//' -e 's/time=//' -e 's/\ \ */\ /g' -e 's/[:space:]+/,/g' -e 's/forwarder=<SA Log Collector hostname>/<SA_Log_Collector_Friendly_name>/' -e 's/forwarder=<Envision LC IP>/Envision-<Insert name>-LC1/' -e 's/forwarder=<Envision LC IP>/Envision-<Insert name>-LC2/' Output1_LogStats.csv > Output1_LogStats_fixed.csv

sed -e 's/\ /\,/g' -e 's/^.//' Output1_LogStats_fixed.csv > Output1_LogStats_Cleaned.csv

curl -u admin:<admin password> "http://<SSL decoder2 of choice>:50102/decoder/?msg=logStats&force-content-type=text/plain" > Output2.csv

sed -e 's/forwarder= /SA/' -e 's/device=//' -e 's/source=//' -e 's/count=//' -e 's/time=//' -e 's/\ \ */\ /g' -e 's/[:space:]+/,/g' -e 's/forwarder=<SA Log Collector hostname>/SA_Log_Collector_Friendly_name/' -e 's/forwarder=<Envision LC IP>/Envision-<Insert name>-LC1/' -e 's/forwarder=<Envision LC IP>/Envision-<Insert name>-LC2/' Output2_LogStats.csv > Output2_LogStats_fixed.csv

sed -e 's/\ /\,/g' -e 's/^.//' Output2_LogStats_fixed.csv > Output2_LogStats_Cleaned.csv

• curl -k -u admin:<admin password> VS. curl -u admin:<admin password> <-- SSL (without an accepted certificate) vs. non SSL
• sed -e 's/forwarder= /local/' <-- If you have a forwarder that shows blank, you can edit the "local" to say something other than if you wish.
• -e 's/source=//' -e 's/count=//' -e 's/time=//' <-- To clean up the source, count and time entries and leave only the data you want
• -e 's/\ \ */\ /g' <-- To clean up extra whitespace
• -e 's/[:space:]+/,/g' <-- To replace white space with a comma
• -e 's/forwarder=<SA Log Collector hostname>/<SA_Log_Collector_Friendly_name>/' <-- Adjust accordingly to your environment (<SA Log Collector hostname>) and your specifications (<SA_Log_Collector_Friendly_name>)
• -e 's/forwarder=<Envision LC IP>/Envision-<Insert name>-LC1/' <-- YMMV on this, if you're migrating away from Envision, you may find this statement useful (and duplicate however many LC's you have), if you're not using Envison as well, this statement isn't needed
• sed -e 's/\ /\,/g' -e 's/^.//' Output1_LogStats_fixed.csv > Output1_LogStats_Cleaned.csv <-- Refer back to my original caveat listed. I had to put this in as no matter what I did, I still had a comma as the first character for each line. This cleans that up.

logstats.zip