There is no denying the power that Security Analytics (SA) brings to the table. However, knowing where to start or providing the tools to get an analyst started down the path of finding the nasty bits is an area we would like to improve. In SA 10.4 the Incident Management definitely does help facilitate this once issues or combinations are known, but without that information how does an analyst know where to start hunting?
What follows is a basic primer on how to investigate certain situations in SA 10.4. The focus is on determining which meta values are of importance for each use case and the different ways each can be tackled using the attached profiles and groups.
First example use case: file analysis.
Most files have extensions and those typically indicate what type of file it is. That is if you believe everything you see which from a security standpoint SA does not. SA looks at the extension (if there is one) and tracks it in the meta key called extension. However to determine the actual file type SA uses much further analysis. The main reason being if a malicious actor is trying to get a piece of malware through the network and past some security controls one simple way is to modify the extension to make it seem like another less harmful file type. Malware in general is some form of an executable meaning it would, if not trying to be inconspicous, have an extension of EXE, DLL, etc. Of course there are many other ways to get around security defenses (like using JavaScript to manipulate a file after it has traversed the network) then just changing a file extension but for now we will focus on this simple case.
If you want to see a file as it is represented by the user or application you can examine meta keys filename and extension but if you really want to validate what type of file is being transmitted as that filename then the filetype meta key should be used. There are a lot of complex parsers, that I will not go into here, that compare the official documented sections for different file formats (portable executables, PDFs, Office Docs, etc) to what is actually in the file being transmitted. This additional parsing allows for finding such items as file magic numbers, if objects are at the appropriate offsets, if the file is encoded a certain way, or if there is Javascript embedded in your PDF file. Therefore having these meta keys along with some pertinent contextual meta like source and destination IP addresses or countries will provide a good picture of what types of files are being transmitted through an environment.
To make this a little easier you can either enable malware analysis (MA) in your Security Analytics deployment since it does all this file analysis for you or you can upload the attached column groups, meta groups, and profiles which contain the file analysis templates. If you choose the later, which I will add is still useful in file analysis cases even if you have MA deployed, here are some screen shots of what the outcome can look like.
The file analysis meta group shown here provides a view of the four main meta keys to focus on during doing file analysis - extension, filename, forensic fingerprint, and destination country. Of course these can be added to if you prefer to have additional information like the source/destination IP addresses or other meta keys you think relevant. In below I have enabled the file analysis meta group.
I have then drilled into the filetype (or forensic fingerprint) windows executable to see all the executables traversing the environment since malware needs to execute so usually in some form of an executable; unless very obfuscated that is.
Below is the same example but using the file analysis column group in the event view to compare values versus the navigate view.
Second example use case: web analysis.
Most malicious traffic is either overt and hoping to blend in with the sea of network traffic or it is covert and attempting to stay undetected by using techniques like encryption and obfuscation to avoid detection. In either case for the most part there are rules of engagement for networking protocols just like there are for file formats. This handy trinket of information can be very useful to an analyst and in general how a lot of our network parsers view the world. The RFC for HTTP (RFC 2616, 7230-7235 - HTTP/1.1) dictates that a host field is required in HTTP/1.1 (was optional in HTTP/1.0) and if it is missing or an IP address is there instead of a FQDN then possibly will lead to something interesting. Now that could be interesting in that the header was crafted by a lazy programmer at a commercial vendor or a malware author. If you have applied content from Live there are parsers that will generate meta like for this example direct to ip http request or http direct to ip request or http1.0 unsupported host header. These meta values will be depicted in the risk.informational, suspicious, or warning meta keys that are included in the web analysis meta and column groups. The profile web analysis utilizes these as well as limits the traffic to only service=80 which will find HTTP traffic on any port. This is because the decoder parsing is port agnostic and will determine it is HTTP whether on a non-standard port or on tcp-80 and tcp-8080; primary and secondary ports for HTTP. This very specific indicator of abnormality can lead to other indicators of compromise and help find those malicious sessions that are more challenging to find directly because their payload is XOR encoded or obfuscated inside javascript inside a PDF file.
This is an example why the risk meta keys were chosen just like why the client application meta key was included in this meta group to find indicators of abnormality such as internet explorer spelled out versus MSIE 10.0 or Trident/6.0 (some normal representations of IE web browser in an HTTP header).
Third and final example use case: querying for IP addresses.
The idea behind the meta groups is not only to use them to limit your view into the data but to also use them to query against. This allows an inexperienced analysts or someone who has better things to do then learn our rule syntax to execute a query against all relevant meta keys. The meta groups are exposed for use in this way in the investigation query pulldown menu provided right before the list of all available meta keys as shown in the figure below.
Now if I wanted to query for an IP address it is much simpler to select Query IPs then to know all the different possible meta keys that could contain an IP address and search each one of those. All the possible meta key candidates for IP addresses are shown as Query IPs in below figure.
There are several listed as Query <a value> and these have been modified to allow for querying against these using all operators by default. Now what this means is that Query Files will not actually query for a file in the filename meta key where it would obviously be provided. The reason for this is because by default the filename meta key is set to indexKeys in the index database (can be modified in index-concentrator-custom.xml) which limits a query to using only exists or !exists. The Query Files does search through some additional meta keys that their relation to files is not as obvious. Now the concept here is to simplify the ability to query without the user knowing all the meta keys so you ask how could they know if the key index has to change? A new user would not know this and why we have these Query groups versus the other groups like File Analysis which has filename contained in it because that group is more focused on limiting the user view into the data versus provided an easy query capability. The filename will still show up in that case, but in a closed state because it is by default indexed at the key level and showing all the values requires the index to be built for that key on the fly which is obviously slower. I must also mention that the Query groups are not the best when it comes to performance either since effectively executing a single query against multiple meta keys.
Remember these are all attempts to make it easier for an analyst when first getting familiar with the product and possibly with long-term use. If you have any suggestions or comments around these groups please let me know.
Logistics:
Now that was all fine and dandy you say but how do I get these groups into my SA system for use? Well, your in luck because they are attached to this blog and there is a way to import/export these JSON files into SA in the areas where they apply. Start by importing the column groups in the investigation event view area. Then in the investigation navigate area import the meta groups followed by the profiles. If you do not follow this order the imports will not work because the profiles are dependent on the other groups to function. These have been tested (in general terms) by me in SA 10.4 and above without issue minus the aforementioned import order limitation and not being able to import if an existing group or profile has the same name.
Attachments:
CustomColumnGroups_ootb.jsn
MetaGroups_ootb_w_query.jsn
Profiles_ootb.jsn
6 Comments
