Table of Contents
Simple RCE
The attack: CVE-2018-0171 (Cisco Smart Install RCE). We will simply be pulling the startup-config. This CVE can be exploited to do more damage but that is not the point of this post.
Why is this simple? Well, this exploit has public proof-of-concept code available, it requires no authentication, and it can be executed externally.
Why should you care? Based on some crude Shodan searches, there are tens of thousands externally facing switches (most of which are in the U.S) that are potentially vulnerable to this exploit. A good friend of mine brought this exploit up to me based on results from a recent penetration test which tells me that it is a relevant attack still.
Quick Tips
- Don't test this exploit outside of a lab.
- The contents in the video are educational only and as owner of the switch, I consented to pwning it.
Finding Abnormal Traffic
This is a packets only post. In my lab I have captured the attack from a Kali box(10.1.1.20) that is external to the switch (192.168.10.254) via a span port on the switch that was abused (ironic eh?). In my lab the switch is not internet facing for obvious reasons but kali is on a network external to the switch to simulate an external attack.
The first step is to understand your network. As an analyst, you should understand the network in which you are protecting (this knowledge comes over time). This is important because you need to be able to find what traffic does not fit into the normal flow. For instance, you most likely won't see TFTP being used often legitimately as the traffic is cleartext. You're more likely to see SCP or SFTP. That makes TFTP an investigation point, regardless if the traffic appears harmless.
In this lab, I'd expect to see SSL, DNS, NTP, HTTP, and SSH as part of the normal flow. When starting to look for abnormal traffic, pick a direction. Picking a direction is easy if you have traffic flow direction setup.
As a consultant one of the first things I like to setup once an environment is deployed is "Traffic Flow". You can find this under the Live Content tab by searching for "traffic flow". It will need to be configured with internal/external subnet information to function. Please let us know if you want more information on that piece.
By picking a direction, we have narrowed the scope of the hunt. Looking at all the data is like finding a needle in a haystack. Since we are looking for an external attack, I selected inbound traffic. As kali is external to the switch's LAN, any connection made by kali to the switch will show up in this traffic direction. Immediately in the screen shots above TFPT stands out. TFTP is a UDP transfer protocol on port 69. In my lab this is abnormal but TFTP does exist in some environments so its presence by itself is not indicative of an attack but it is a point of investigation. By clicking the green (1) next to TFTP we can view the event and reconstruct the packet.
Clearly, something isn't right here. We have a few different breadcrumbs to follow.
- Why is a startup-config file from a Cisco switch moving in cleartext across the network?
- Why is it moving outside the LAN?
- What is 10.1.1.20?
Without having per-existing knowledge of the attack, I'd start with answering these questions. We can go about this a couple ways. You can take a gamble on the CVE showing up in a vulnerability scanner or we can piece meal what happened. I'm starting with investigating 10.1.1.20. If you're new to NetWitness, you can further the investigation by tacking on a another query in the investigation tab. The idea here is to carve out the data we don't need so that we get closer to finding the information we do.
Seeing traffic in OTHER can often turn into a rabbit-hole of data but looking at it when its been carved out a bit can often lead to interesting information. By reconstructing the packet the same way as before we can see relevant data. We can see a TCP connection on port 4786 (Cisco Smart Install) and what appears to be a remote command calling for the startup-config we saw transferred over the wire. We can look at this a little more granular via the web tab.
I personally think the Hex view is the most useful and we can see most of the same information here as before. Although not necessarily more useful in this specific scenario, you'll find it easier to reconstruct fragmented packets (like what is found in the more recent bad neighbor exploit) in this view. The default view will also try to reconstruct any web pages.
So far we know that an external device pulled the startup-config file from the switch via TFTP after an initial TCP connection via port 4786. This is enough information to do a google search and the query "cisco port 4786" will give you plenty of results on the exploit you just witnessed. Its fair to say that the switch was attacked but your job doesn't end there. We saw a password in cleartext and the presence of inbound HTTP traffic which tells me that this switch most likely has web GUI. Using the same steps as above we can look at the HTTP traffic.
Some pieces of web content will automatically be reconstructed while investigating.
While looking at the event meta for the HTTP traffic, it is pretty clear that a login attempt was made.
At this point we have a compromised device. This is the part where remediation begins and an incident report gets created. This is also where as analysts, we will begin to think about how to make our response faster in the future. This attack did not show up in the Cisco logs. Depending on your organization's size, you may have multiple devices that are vulnerable to the same attack (hopefully not many externally facing). This becomes tricky and time intensive for large organizations with network devices spread across multiple sites. We also don't want to make a routine of checking for this attack going forward every day. Part of remediation should be to take an inventory of devices at risk and create a plan to mitigate the threat but that can take time and we need to monitor until that process is complete. We have other threats to look for and the point of NetWitness is to save you valuable time. To help monitor the threat during remediation, we can create an alert to fire based on a set criteria.
Content Creation
Part of an analyst's role is to be proactive rather than solely reactive. To help monitor the situation created by this hunt, we can create content to alert us when a possible attack is detected. By automating this task, we can spend more time looking for other threats. This section will cover creating a basic ESA rule to create an alert. The bonus section will cover extending functionality with NetWitness Logs, a simple Shodan integration, and a simple Censys integration.
ESA Rule Builder
The most effective way to handle this threat will be creating an ESA Correlation Rule. We can do this by heading to the configuration section on the NetWitness main navigation bar. From there, you can click on "ESA Rules" as depicted below and you'll see the rule library by default. The view on versions earlier than 11.5 will be slightly different. Instead of icons, the word "Configure" will be in the top navigation bar.
At this point we can create a new rule using the rule builder.
The rule name will be what shows up in Respond and also what we will be using for the log aspect further down. I personally use CVE names when I can. The reason for this is because there are a lot of threats and looking up the CVE is quicker when the alert is named after it. We will utilize this further down with a Shodan lookup. Always enable "Trial Rule" for a new rule. It will protect the ESA from crashing if a bad rule is created. We do want to alert off the rule so ensure that is checked off (should be by default) and adjust the severity to your organizations guidelines. Once that is done, we will add two conditions as this attack had two distinct criteria.
The first condition will look for Cisco Smart Install connections on TCP port 4786. Based on what was seen in the investigation, the attack starts with a connection to 4786 and is immediately followed by the TFTP transfer over UDP port 69.
The second condition will look for TFTP traffic.
Once that is complete, we need to configure more information for the conditions. On the first condition, set the "Connector" to be "followed by", "Correlation Type" to "SAME", and the "Meta" to "ip_dst. The "followed by" means as the name implies. It is different than an "AND" type in that the following condition must occur after the first. We are looking to group by "ip_dst" because in the event you have multiple switches attack at the same time, we make sure the alerts are organized by each device. Due to the immediate TFTP transfer, we will set the "Occurs Within" to 1 minute.
Optional: If you have a SIEM, you can send the alert to the SIEM via syslog. This requires that a syslog server is configured in the global notification settings under the system settings in the administration menu.
At this point, we can save the rule. If you click "show syntax", you will be able to see the Esper syntax. This is the format in which advanced rules are written. When you hit save, if there is an issue with the rule, it will not save and the issue will be highlighted in red. Once the rule is created, it needs to be added a deployment.
If the attack is detected again, it will show up in respond as shown in the video below.
Bonus
We will extend the base functionality in a couple of ways. The first being a fun trick I picked up if NetWitness Logs is also in the environment. As seen in the ESA Rule builder, the rule can send a notification via syslog. In this instance, I configured my Endpoint Log Hybrid as a syslog server destination. So when the ESA rule triggers, a notification is sent to the log decoder service as syslog. A NetWitness Log Decoder will parse syslog sent by another NetWitness appliance (ESA in this case) by default and will identify the source accordingly as seen below (look at "Device Type"). The important fields here are "Device Type" and "Event Description". With these fields we can create an app rule on the log decoder. The app rule will populate the "alert" meta with the CVE name.
Decoder App Rule
On 11.5, the admin settings are accessible by the tools icon on the right hand side of the navigation bar. On all other 11.x versions, there is an Admin tab on the navigation bar. Once there, go to services and go to the packet decoder(s) config page.
Make sure that you use the same name as the ESA Rule. This way when you see the alert, you'll know exactly what triggered it. Now earlier I mentioned how I like to use the CVE names for this kind of content. Part of this is because there are millions of CVEs in existence and a shortcut I came up with was to integrate with Shodan's exploit lookup.
We can accomplish this by setting up a context menu action. This is completed in the administration menu and only needs to be completed once per meta value. So going forward, we will be able to right-click a value in the alert meta and look it up in Shodan.
Add a new menu option by click on the red plus sign and edit the values as below. You will need to use your own Shodan API key.
Once you hit save, either reload a tab that has investigation open or open a new one. Any value in the alert meta will have a right click option for "Shodan Exploit" going forward. Its a quick function that saves time when trying to figure out what CVE is what. If you really don't want to use CVE names, that is fine too. Use a name associated with the exploit. In this instance, I could use "Cisco Smart Install" and Shodan will query various resources (Ex. ExploitDB) and provide me links. Use of the CVE name just provides me an immediate result without having to look through search results.
Packet Bonus
In the previous section, we went over adding a Shodan right-click function to alert meta, but what if you only have packets. Surely you can extend that too. You'd be correct. One common extension of functionality that I like to do is to add Censys and/or Shodan to the IP related meta (ip.dst, ip.src, ip.addr). There are more tools we can add but we'll cover those in another post later on. We're going to move away from the attack for this section because I did not make my vulnerable switch internet facing for obvious reasons (I did not want to get pwned by a stranger/bot). I'll show the Censys integration on an external IP as most tools will not be able to provide data on my internal LAN. Here is a use case though. You want to quickly determine how many of your externally facing switch could potentially be vulnerable to the attack we discussed above. Wouldn't it be nice to look at the IP meta in NetWitness, right-click the externally facing switch IP and instantly get a print out out on which ports are open? Specifically if port 4786 is open. No API key, nmap, or vulnerability report required.
I redacted some information from the results but you can see what is important. If a vulnerable switch had be queried, we would see 4786 as open. Setting this up is easier than Shodan as you won't need an API for the base lookup functionality. Following the same steps as above, create another Context Menu Action.
Afterthoughts
Here is a high-level overview of what this post covered:
- Switch gets pwned
- We review hunting basics
- We find said pwning during routine hunt
- We have notified operations and hopefully a change order is in the works to remediate this threat
- Until remediation is done, we have to monitor for further pwning and identify vulnerable devices
- We can make monitoring more effective with the alerting functionality within Respond
- ESA rule building
- We can extend the NetWitness out-of-box functionality with third-party integrations such as Shodan and Censys
Everything we have gone over can be expanded on further in some shape or form. Both Censys and Shodan have more functionality than what was discussed here. I chose these two as they are often not utilized by a blue team. Part of being an analyst is understanding the organizations infrastructure. Another part is to think outside the box and look at the infrastructure from an attacker's point of view. These two tools are often used during a reconnaissance phase of an attack. It is beneficial for a blue team to use these tools as well. Shodan is often used to find vulnerable devices but we are able to re-purpose it to make our vulnerability lookup more efficient. The Censys lookup provides us a with a quick way to see what devices may be vulnerable. The information is the same as what the attacker sees. I hope you found this post interesting and learned something new. Anything involving attacks is always a personal interest to me and I enjoy providing my customers (and you the reader) some information that I have learned over time.
We will be discussing this process on the December 16, 2020 webinar:
