Preface
This blog post explains how to import external intelligence as a NetWitness recurring feed. As an example, the Locky Ransomware C2 domain blocklist from https://ransomwaretracker.abuse.ch is used.
We will use the SA Server Head Unit to perform all the operations described in this blog post.
Downloading and pre-processing external content
At first the list offered by the external provider must be downloaded to the local environment:
curl https://ransomwaretracker.abuse.ch/downloads/LY_C2_DOMBL.txt > locky.txt
The file contains a comment block at the top followed by one domain per line:
##########################################################################
# Locky Ransomware C2 domain blocklist (LY_C2_DOMBL) #
# Generated on 2016-10-18 09:10:02 UTC #
# #
# For questions please refer to: #
# https://ransomwaretracker.abuse.ch/blocklist/ #
##########################################################################
wrubyjtvqhxaqkh.pw
jfmiondv.xyz
tswsgajtwhqkosd.su
xofguhypjgvxrm.pw
yofkhfskdyiqo.biz
[...]
To be able to schedule the feed import for the decoders, we create a path dedicated for feed deployment on the local webserver:
mkdir /var/netwitness/srv/www/rsa/feeds
Now we need to perform the following operations on the file:
- Remove the comments
- Add the values to be written to Meta Keys at the end of each line
- Write the result to a new file
The following command will do all those tasks at once. It may look very complicated at first, but all parts of the command will be explained below:
cat locky.txt | grep -v '#' | awk -F $'\n' '{print $1",Locky Ransomware,ransomwaretracker.abuse.ch,Ransomware"}' > /var/netwitness/srv/www/rsa/feeds/locky.csv
Explanation:
- cat locky.txt: Read the file locky.txt in the current folder
- grep -v '#': Remove all lines that include the hash symbol
- awk -F $'\n' '{print $1"[...]"}': for each line return the first column (which is the domain name) and add the static text within the double quotes (which are the meta values to be generated later).
- > /var/netwitness/srv/www/rsa/feeds/locky.csv: Save the result to the locky.csv file to the path created before
As these lists usually change quite often (some providers update their feeds every 30 minutes), a bash script can be created and scheduled to automate the process (some minimal error handling is added in case the file could not be downloaded):
- Create the bash script (the file can also be found in the attachments to this post)
vi /root/lockyC2.sh
#!/bin/bash
curl https://ransomwaretracker.abuse.ch/downloads/LY_C2_DOMBL.txt > locky.txt
firstLine=$(head -n 1 locky.txt)
if [[ ${firstLine:0:1} == '#' ]]; then
cat locky.txt | grep -v '#' | awk -F $'\n' '{print $1",Locky Ransomware,ransomwaretracker.abuse.ch,Ransomware"}' > /var/netwitness/srv/www/rsa/feeds/locky.csv
else
echo "File not downloaded successfully"
fi
- Schedule the script to run every 30 minutes
crontab -e
*/30 * * * * /root/lockyC2.sh
Creating a custom feed
After completing the steps mentioned above, a recurring custom feed can be created in Live > Feeds:
- Provide a name for the feed and add its URL, make sure the “Recur Every” schedule matches the update interval that you set in the cronjob (to have the data updated in the same intervals).
- Select the decoders to push the feed to
- Select “IP” if the feed is based on IP addresses or “Non-IP” otherwise. Select the correct index column (the column where the “search” parameter can be found) and the callback key(s) which are the meta keys where you want to find the value in. Define which meta keys the remaining columns should be written to by selecting from the drop-down above each column.
- Verify your settings then click “Finish”
