On July 2nd a supply chain attack was initiated using the software of a US company called Kaseya reminiscent in scale of the SolarWinds incident discovered in late 2020. Unlike with SolarWinds, the intention of this supply-chain attack has been destructive by deploying ransomware at a massive scale. Kaseya provides IT solutions to help other businesses manage computers within their networks making it an optimal target to launch this type of attack.
Three days after the deployment of ransomware, on July 5th, Kaseya reported that fewer than 1,500 businesses were affected by the attack. Many of those companies were small and midsize businesses with little direct exposure to consumers.
However, based on third-party reports, it appears that a much wider attack unfolded with a significant impact on the US and European retail market. For example, it has been reported that a Swedish supermarket chain was forced to close some outlets for several days after the attack due to the effects caused by the ransomware.
To make matters worse, on July 7th, Malwarebytes reported a malware spam campaign capitalizing on the awareness and distress caused by this ransomware attacks by spreading bogus links purporting to be a Microsoft security update when in fact it was a dropper for a Cobalt Strike payload.
While this second attack was opportunistic and based on traditional social-engineering techniques, the original one was sophisticated, using zero-day vulnerabilities that targeted the Kaseya virtual systems/server administrator (VSA) software.
Kaseya VSA is used by companies of various sizes and technology-service providers and it is adopted for remote management, software patching, and monitoring of systems on computer networks. This type of software, by its nature, requires broad access and elevated trust on the systems it monitors, making it especially attractive to attackers – made worse by the ability to compromise so many customers at one time. The attacker was able to exploit vulnerabilities in the VSA software and leverage it to disseminate ransomware across the Kaseya customer base.
Based on a public announcement, the infamous REvil gang claimed to be the orchestrator of this attack. Subsequent analyses from the security community corroborated the threat actor's attribution.
Figure 1: REvil blog announcing the successful Kaseya attack
During REvil’s public announcement, the attacker requested $70 million for the decryption key to unlock the computers affected in this widespread ransomware attack.
The attack was triggered via an authentication bypass vulnerability in the Kaseya VSA web interface which allowed REvil to upload its malicious tools to any exposed Kaseya server.
Figure 1: Initial exploitation via VSA Web Interface exploit
An additional vulnerability, possibly based on a SQL injection vulnerability, was then utilized against the page /userFilterTableRpt.asp of the Kaseya web interface. This allows the execution of the malicious DLL payload (KUpload.dll) uploaded when the server is successfully exploited.
The DLL offers upload functionality and logs to a file KUpload.log which can be used, if recovered during the investigation, as a starting point for any analyst looking into a suspected breached Kaseya server, to understand when and how the attack was executed.
The log is used by the attacker for troubleshooting purposes.
Figure 2: SQL Injection and subsequent activation of the second stage of the attack
The UI authentication vulnerability, along with other potential flaws in Kaseya VSA, were originally discovered, in March, by Dutch researchers and reported to Kaseya with a responsible disclosure: https://csirt.divd.nl/cases/DIVD-2021-00011/
Despite this, REvil was able to trace the same vulnerability from a different source and used it to target Kaseya and its customers.
Once the attacker was able to interact with the Kaseya Server, he dropped a crypter, based on REvil ransomware (named agent.crt), and was ready to infect internal systems via the VSA server update feature.
Notably, in the recorded cases investigated by the Netwitness IR team, and also confirmed by other researchers, there was no evidence of data exfiltration before the dissemination of the ransomware. This is a notable aspect that differentiates this attack from other attacks connected with REvil threat actor.
Figure 3: Malicious tools upload
With the successful upload of the crypter to the VSA Server, the attacker proceeded to disseminate it to the victim’s network.
As such, the attacker sent the update command to all the connected Kaseya Monitoring Agents (the agent is a monitoring service/application) which in turn caused the download of the agent.crt from the initial drop folder and the Batch script to execute the additional actions needed to activate the ransomware.
The script, once executed, moves the Ransomware from the download folder:
<kayseya web root>\VSATicketFiles\agent.crt
to the agent working directory:
The script finalizes the attack by computing the date with the activation date (July 2) via SQL query against the Server database. This value is later used in conjunction with the ping sleep timer described later on.
Figure 4: Malicious Crypter/Ransomware dissemination leveraging on on-premises VSA server
These vulnerabilities and their exploitation have allowed the attacker to organize the dissemination of ransomware at a massive and indiscriminate scale. Each compromised VSA server inside an affected network attempts to push ransomware on VSA agents running on managed Windows devices.
The execution of the ransomware on the victim system is initiated by the VSA Monitoring service (AgentMon.exe) and controlled by several commands carried out by a malicious batch script connected with the setup of agent.crt.
The entire execution is protected by Kaseya AgentMon.exe process which is trusted by traditional security applications such as the antivirus.
Figure 5: Task Manager showing Kaseya VSA Monitoring agent
This gave REvil a huge advantage over the cybersecurity measures enforced in the network as the exploitation of the VSA server allowed the initial compromise through a trusted channel and leveraged trust in the VSA agent code. This is reflected in anti-malware software exclusions that Kaseya requires during the setup for its application and agent “working” folders.
Anything executed by the Kaseya Agent Monitor is ignored by antivirus because of those exclusions—which allowed REvil to deploy its dropper without scrutiny.
The VSA Server sends “agent.crt” package to the internal Windows systems together with a Batch script which, and once executed, moves the .crt file to the agent working directory, decrypt it and rename it as “Agent.exe”.
"Agent.exe" contains two binaries embedded in its body (particularly in the resources section):
On execution, Agent.exe writes (with system privilege) both files into C:\Windows, then executes MsMpEng.exe that eventually loaded the malicious "mpsvc.dll" file exploiting the typical DLL Side-loading.
The Zero-Day Vulnerabilities
The attacker exploited a small subset of zero-day vulnerabilities to upload files and execute subsequent commands from vulnerable Kaseya VSA servers.
Authentication bypass vulnerability in the Kaseya VSA web interface
While the details about the exploitation are still lacking several core elements, the way the attacker carried out the initial attack sequence confirmed the authentication bypass was most likely the initial exploit used to land upon the VSA Server.
SQL injection attack
The SQL Injection attack was enabled by the initial drop of the following file
As initially clarified by a Huntress report, the file contains a significant amount of potential SQL injection vulnerabilities, which would offer an attack vector for code execution, an essential complement to the authentication bypass vulnerability that allowed the attacker to access the VSA server in the first place.
In addition, the SQL injection phase allowed the REvil threat actors to execute the following actions on the VSA agents:
Figure 6: Second stage of the attack (triggered by the SQL Injection)
Process tracking data from a targeted system would show these actions carried out by the attacker:
del /q /f c:\kworking\agent.crt C:\Windows\cert.exe
Once the agent file runs, REvil uses DLL sideloading to inject the REvil ransomware DLL into Windows Defender activating the encryption on the machine and removing Kaseya logs to reduce the risk of being detected.
Contrary to previous REvil attacks, no further commands, have been reported by the security community.
In this case, none of the reported commands, nor anything inherent to the ransomware executable allows data to be stolen from the network.
As such, this appears to be a fully automated attack that allowed the malicious actors to operate at a remarkable scale, albeit forcing REvil to modify their usual strategy, limiting the impact and the potential chance of negotiation against single victims.
Some of the most effective techniques adopted in the past by REvil and other Malware-as-a-Service (MaaS) actors have been based on the previous exfiltration of data and the promise to leak it via the Internet if the victim did not pay the requested sum.
Crypter: REvil Ransomware
REvil ransomware is not new. It has been advertised on underground forums since 2019 and it is one of the most prolific Ransomware-as-a-Service (RaaS) and MaaS providers.
From a cybersecurity perspective, the crew and its affiliates registered a record year in 2020 when they harvested more than $100M USD.
As a malware family, REvil/Sodinokibi is typical ransomware with some peculiarities.
It encrypts all files on local drives except those listed in their configuration file, but more importantly, during the initial activation of the malware, there is a check on the system keyboard: via User32.dll's “GetKeyboardLayoutList” function. This check inspects the keyboard identifier and will terminate execution if the result ends in a value between \x18 through \x44 inclusive, which are values linked to East European keyboard layouts.
The actual version, used in this attack is version 2 and it is deployed in conjunction with the already discussed batch command, which in turn, is executed through the Kaseya agent.
The ransomware in general does not have any functionality to evade security products. The batch command was already responsible for disabling the security products and uses the certutil.exe utility to unpack the final payload, as a DLL together (mpsvc.dll) with a vulnerable Microsoft binary: an older version of Microsoft Defender (MsMpEng.exe).
The DLL is then loaded by calling this older Microsoft Defender executable using the DLL side-loading technique (T1574.002).
REvil Attacker Details
The actor is also known as "GOLD SOUTHFIELD” (G0115) on the MITRE ATT&CK Framework.
REvil is a cybercriminal threat group that authors and operates the REvil ransomware (also known as “Sodinokibi”) on behalf of various affiliated threat groups. RSA has reported it as a MaaS operator since April 2019.
The crew is responsible for authoring REvil and operating the backend infrastructure used by affiliates (also called partners) to create malware builds and to collect ransom payments from victims.
REvil partners are recruited through Russian-speaking underground forums where a limited and exclusive partnership is offered to potential affiliates that demonstrate effective capabilities to penetrate high-value Western organizations.
REvil affiliates distribute ransomware through a variety of means including exploit kits, scan-and-exploit attacks, publicly accessible RDP and remote management and monitoring (RMM) servers, and backdoored software installers.
Since December 2019, similar to other crew like Maze, Ryuk, and Nefilim, REvil launched a website accessible via TOR that publishes stolen data from intrusions to generate additional leverage against victims.
Between the typical and common attacking techniques, the following ones are characteristic of REvil attacks (Courtesy of MITRE ATT&ck Framework.):
GOLD SOUTHFIELD has staged and executed PowerShell scripts on compromised hosts. (Citation: Tetra Defense Sodinokibi March 2020)"}
GOLD SOUTHFIELD has exploited Oracle WebLogic vulnerabilities for initial compromise. (Citation: Secureworks REvil September 2019)"}
GOLD SOUTHFIELD has used publicly accessible RDP and remote management and monitoring (RMM) servers to gain access to victim machines.(Citation: Secureworks REvil September 2019)\t"}
GOLD SOUTHFIELD has executed base64 encoded PowerShell scripts on compromised hosts. (Citation: Tetra Defense Sodinokibi March 2020)"}
GOLD SOUTHFIELD has conducted malicious spam (malspam) campaigns to gain access to victim's machines. (Citation: Secureworks REvil September 2019)"}
GOLD SOUTHFIELD has used the remote monitoring and management tool ConnectWise to obtain screen captures from victim's machines. (Citation: Tetra Defense Sodinokibi March 2020)"}
GOLD SOUTHFIELD has distributed ransomware by backdooring software installers via a strategic web compromise of the site hosting Italian WinRAR. (Citation: Secureworks REvil September 2019) (Citation: Secureworks GandCrab and REvil September 2019) (Citation: Secureworks GOLD SOUTHFIELD)"}
GOLD SOUTHFIELD has breached Managed Service Providers (MSP's) to deliver malware to MSP customers. (Citation: Secureworks REvil September 2019)"}]
Table 1: REvil/Sodinokibi ATT&ck profile
To detect this ransomware attack and develop actionable indicators, NWIR re-created the attack in a lab using events from cases investigated. This process aimed to extract significant content to complement the initial IOCs.
The following Lab setup has been created to achieve this goal:
Figure 7: Lab overview
The NetWitness packet solution was running software version 11.6.
NetWitness Endpoint was running software version 11.6
The Desktop was running Windows 10.1909 and both the domain controller and the Kaseya server were running Windows 2016 Server Standard.
Our team focused on the actions carried out after exploiting the VSA vulnerability to drop and execute the ransomware file called ‘agent.crt’ that is activated via malicious batch commands.
The command was captured in NetWitness Endpoint in the form of several Process and Console events. The specific console event with the entire one-line command is shown in the figure below:
Figure 8: Malicious Batch Command
First, the executed command effectively achieves a sleep timer by running the Ping utility targeting localhost with a high number of echo requests.
Subsequent commands would not be executed until the high number of pings were completed. In the experiment, this number was reduced to expedite analysis. However, in instances in the wild, the number of pings was in the thousands which could take over an hour to complete.
Figure 9: Ping with number of echo requests
The second command ran PowerShell to disable Windows Defender. This specific process event was flagged by NetWitness for disabling security tools.
Figure 10: Disabling Windows Defender
Next, the command used several masquerading techniques before executing the CertUtil.exe Windows utility to evade potential monitoring of this program. CertUtil.exe was copied and renamed to cert.exe and then a few random bytes were appended to the end of the file so that any monitoring based on a cryptographic hash would be unsuccessful.
Figure 11: Creation and Execution of Masqueraded CertUtil
Once copied and modified, CertUtil was leveraged to create an executable called “agent.exe” by decoding the content of the agent.crt file. Finally, after deleting cert.exe and agent.crt, the command executes agent.exe.
Figure 12: Creation of Agent.exe
‘Agent.exe’ drops a vulnerable version of Windows Defender and the REvil DLL. The malicious DLL is loaded and executed through a DLL Sideloading vulnerability in MsMng.exe after being executed by agent.exe. Once this process is executed, the REvil ransomware began encrypting targeted files.
Figure 13: Agent.exe dropped files and MsMpEng.exe execution
Detection via NetWitness
The NetWitness Platform is able to detect the occurrence of a Kaseya breach by querying the system against specific IOCs in three different areas:
In fact, the following queries can be utilized within NetWitness to identify known atomic indicators associated with the REvil attack.
The search can be directed against Domains and IP Addresses, the latter is exemplified below:
Figure 16: Query for selected IP Address related to Kaseya breach
Additionally, a Zscaler blog published a list of more than 1,000 domains associated with the ransomware variants observed in this attack. These values can be queried against the ‘alias.host’ and ‘domain.all’ metakeys.
The network configuration was disabled in the REvil samples observed in the Kaseya attack, so the traffic produced in this experiment was initiated with a modified version of the ransomware.
The screenshot below contains DNS events that match the domains associated with REvil beaconing.
Figure 17: REvil C2 Match
The query in the table below will identify potential events within packets, endpoint, or logs that are associated with these REvil C2 domains.
At the endpoint level, we can leverage IOCs related to the Tampering of Microsoft Defender.
As reported, the attack relies on PowerShell to disable it, so we can query the endpoint platform for the following action:
Figure 18: NWE query for PowerShell used to tamper Microsoft Defender
We can also search for specific local paths and files.
We reported the following filenames and paths during the analysis:
kworking\agent.crt – Encoded malicious content
kworking\agent.exe – Decoded contents of agent.crt
mpsvc.dll – REvil encryptor payload
In addition, a Registry key is created when the ransomware is dropped on a system:
It can be added to our search at the endpoint level to look for REvil activities against the system:
Figure 19: NWE query for local files, folders and Registry key
To complete the review of the potential mechanism to detect REvil activities linked with this campaign, we can leverage on Logs collected via Netwitness.
In particular, we can focus on the following event codes located within the Microsoft-Windows-Windows Defender/Operational event log:
Figure 20: Event ID 5007 – Windows Defender Disabled
We can search events in Windows Security Event Logs with code 4688 where the process command line presents references to the “\\kworking\\agent.exe*"
Figure 21: Event ID 4688 – Process Creation
The Kaseya ransomware attack is among the largest and most impactful cybersecurity events to date. The use of ransomware as a service (RaaS) in a supply chain attack combines two particularly pernicious techniques.
However, while the combination of techniques is novel, the underlying tactics are typical of modern advanced persistent threats (APTs), which NetWitness Platform is designed to detect. NetWitness recommends carrying out periodic Proactive Hunting for Lateral Movement and Ransomware Attacks as foundational hygiene using NetWitness to Detect Ransomware Attacks: