Live - Custom Feed

[EvanRamos1]

Version SA 10.6.1 - 

This might be a stupid question but, is it possible to setup a recurring custom live feed with a single column (list of IP addresses). Example could be something simple like this IP list; https://rules.emergingthreats.net/blockrules/compromised-ips.txt  - i dont believe RSA provides this as a feed in Live.

When i add a Key (ip.dst) on the column, SA says well you gotta have an Index. So I set an Index on column , wipes the Key value , and SA says well you gotta have a key. ~loop~

I assume its not possible? 

[JohnSnider]

Evan,

All feeds require at least 2 columns.  in your case you have a list of IP's, for an IP related feed the index is the IP address column, (that is compared to ip.dst & ip.src for any matches), the additional columns are used to write a values into other meta keys, when there is a match to the IP address.

You will need to massage your list to add the other 2 columns, (comma separated), which you be pretty easy to do with a script. 

Example:

#Index	risk.warning	threat.source
1.1.1.1	compromised_ip_hit	emerging_threats
2.2.2.2	compromised_ip_hit	emerging_threats
3.3.3.3	compromised_ip_hit	emerging_threats

[EvanRamos1]

Okay, figured that would be the case...the additional columns make sense given meta key requirement. 

Thanks for the info && help John!

[JohnSnider]

#!/bin/bash
#
#Pull file from remote host and create CSV for NW feed
#
mkdir -p /var/netwitness/srv/www/feeds/ETcompromisedIPs
cd /var/netwitness/srv/www/feeds/ETcompromisedIPs
touch compromised-ips.csv
wget https://rules.emergingthreats.net/blockrules/compromised-ips.txt
for i in `cat compromised-ips.txt`;do echo -n "$i,compromised_ip_hit,emerging_threats";echo"";done >> /var/netwitness/srv/www/feeds/compromised-ips.csv

this works as a shell script to do what you want.  just schedule it to run with cron and create your recurring feed to access it at http:localhost/feeds/compromised-ips.csv 

[EvanRamos1]

Heyyyyyyy my guy, you wrote a shell script for me?! Cool. 

I'll give this a shot today - thanks man!

[EvanRamos1]

Appending the two fields worked. However, since our environment doesn't allow direct internet access from this particular subnet i had to re-write the shell script in powershell and host on an IIS website in our core network. Then setup a reoccurring Feed in SA pointing to the url...  Anyway , no big deal. Essentially doing the same thing you did in bash just in posh ;

$outfilecsv = "C:\SAFeeds\Feeds\compromised-ips.csv"
Invoke-WebRequest -Uri "https://rules.emergingthreats.net/blockrules/compromised-ips.txt" -OutFile $outfilecsv
(Get-Content $outfilecsv) | % {$_ + ",compromised_ip_hit,emerging_threats"} | Set-Content $outfilecsv

Thanks again!

[CSIPiemonteSOC]

hi 

how can I import this list of IP addresses in security analytics? 

[NaushadKasu1]

It needs to be stored on a Web Server that NW/SA can then reference via the recurring feed in Live module.  NW/SA can host this file itself as John suggested putting it under /var/netwitness/srv/www/feeds... then you can point your Live custom feed to localhost.

[EvanRamos1]

Yep. In my case, i hosted on a web server and used the feeds in SA UI to setup a reoccurring job.

PS - For the Windows portion, i setup a schedule task (instead of cron job) to do the polling to the threat intel site.