This idea started out as a POC to see if there was a way to implement substring matches for a large number of items related to useragent strings.
There are a number of locations that have published known bad UA strings over the years but they have always had to be limited to exact matches in RSA NetWitness (at least for scale reasons).
Here are a few:
- https://community.rsa.com/community/products/netwitness/blog/2013/03/19/detecting-malicious-and-suspicious-user-agent-strings
- sigma/proxy_ua_malware.yml at master · Neo23x0/sigma · GitHub
- EMC Community Network - DECN: Import This Malicious User-Agent String Feed
The Feed function only works with exact matches, application rules could be used for substring matches but for a large number of elements that option wasn't really scalable.
What about Lua?
Could Lua be used to create a single package of content that could be applied to Log and Packet decoders that could search for exact matches and substrings efficiently (as possible) and write out something useful to indicate further investigation is necessary?
With some help from internal RSA resources and after reviewing some of our parsers we decided to build this function in a lua parser. The basic idea was two sections that enabled exact matches (replacing feed requirement for 1/2 the solution) and a substring match section. Optimizing performance as much as possible led to a two table structure for the substring section (make sure you update the two related tables if you add or remove items).
As usual this is POC code. It works in a small lab - your performance may vary in a large corporate environment. Use at your own risk!
Attached is the Lua parser that can be loaded to both log and packet decoders which reads from the client metakey and writes out to the IOC and analysis.session keys when there is a match.
The output looks like this:
ioc='suspicious_useragent'There are two potential ways of displaying a match in analysis.session:
Exact matches are written like this:
ua_match_metasploit old rare
ua_match_metasploit
Susbtring matches are written like this:
ua_substring_sql injection >> ^.*sqlmap.*$
ua_substring_badly scripted >> ^mozilla/3%.0 .*$
This way I was able to understand from the meta written what was matched and why (exact or substring). This also showed the potential threat it was related to. The ^ and $ lock the match to beginning and end of the string and .* are your wildcards which are translated from the * entry in the tables to keep things simple on input.
For instance the two entries for the sqlmap look like this:
"*sqlmap*",
and
["*sqlmap*"] = "SQL Injection",
The Lua parser is listed here and any updates made will be placed here. Ideally this will have the tables contained in an *options file so that the configuration and updates are separate.
At the moment changes can be made from the decoder > config> files section as this is a cleartext lua parser. You can change/add from there and push the file to all your other decoders.
** keep in mind that most log parsers write the user agent string to user.agent and not client on log decoders. you can either change the lua parser to read from there, or build a lua utility parser to move user.agent to client then this parser will work evenly across both log and packet decoders.**
3 Comments
