NetWitness Community

AlexeyFedorov · ‎2016-03-16

Hello,

Somobody has solution how I can transformate data from user.agent metakey (Log and Packet) to some like Device, OS, Application via Feed or LUA Parser? I found some FlexParser in RSA Live, but it is not I want. I found some site like Complete List of iOS User-Agent Strings | Enterprise iOS with DB of iPhone/iPad/IPod, but data is old and I can't create suitable Feed.

I found interesting progect GitHub - piwik/device-detector: The Universal Device Detection library will parse any User Agent and detect the browser,… and his web version Device Detector Demo

It is what I want, but how transformate it in LUA Parser or Feed?

AlexeyFedorov · ‎2016-03-18

I solve all issue what I wrote into previous email.

DavidWaugh1 · ‎2016-03-18

Thats good news.

For "fun" I downloaded a large list of useragents from Massive list of user agents for User Agent Switcher by Chris Pederik http://forums.chrispederick.com/categories/user-age…

I extracted all the user agents with:

cat useragentswitcher.xml |grep "useragent description" |cut -f3 -d "=" | cut -f2 -d "\"" |grep -v -e '^$' >useragent-strings.txt

And then used the following script to send them to an internal web server so that the traffic was captured by security analytics.

spoof-useragent.sh

while IFS='' read -r line || [[ -n "$line" ]]; do

echo "Text read from file: $line"

curl -A "$line" http://192.168.200.30/virustest

done < useragent-strings.txt

Here is a picture of the results. The feed file is also attached to this post.

I've also attached my latest cron job file and script.php.

The new cron job file only sends the string to be processed if the client has not been seen before.

AlexeyFedorov · ‎2016-03-21

Hello David,

I found issue. PHP script doesn't detect apple user agent strings like "apple-iphone8c2/1304.15", meanwhile Device Detector Demo detect it!

DavidWaugh1 · ‎2016-03-22

Hi Alex, Thanks for letting me know. Looks like a little gremlin! Looking into it now.

AlexeyFedorov · ‎2016-03-22

Hello David,

I just disable $UsefulInfo into script.php and all ok. Now I use output:

{

echo $argv[1];

echo "^";

if(!empty($clientInfo["name"])) {echo $clientInfo["name"];} else {echo "";}

if(!empty($clientInfo["version"])) {echo " ",$clientInfo["version"];} else {echo "";}

echo "^";

if(!empty($osInfo["name"])) {echo $osInfo["name"];} else {echo "";}

if(!empty($osInfo["version"])) {echo " ",$osInfo["version"];} else {echo "";}

echo "^";

echo $device;

echo "^";

echo $brand;

if (!empty($model)) {echo " ",$model;} else {echo "";}

echo "\r\n";

}

DavidWaugh1 · ‎2016-03-22

I think your code is much better!

DavidWaugh1 · ‎2016-03-22

Here are the latest files.

The scripht.php now includes botinfo.

Thank to Alex for pointing out the errors in the original script.

useragentfeed.csv - example feed output

script.php - DeviceDetector script takes Useragent as a string as the first parameter and output a feed friendly line

UserAgentInfoScript2.sh - put in /etc/cron.hourly to generate an updated feed.

DeviceDetector.xml - feed definition file

The additional keys that I used in my index-concentrator-custom.xml file were:

NetWitness Community

User Agent to Device/OS/Application