Malware Analysis fails to process the Yara Rules in NetWitness Platform 12.4.0.0 version or later

[#1]

Article Number

000003238

Applies To

NetWitness Product Set: NetWitness Platform
NetWitness Product/Service Type: Malware Analysis
NetWitness Version/Condition: 12.4 and later
Platform: AlmaLinux

Issue

Malware Analysis (MA) fails to process Yara rules from version 12.4.0.0 and above due to Python version incompatibility. The system automatically selects Python 3.6.8, while the Yara rules require Python 2.7.18.

Cause

The incompatibility arises because the system defaults to using Python 3.6.8, which does not support the Yara rules written for Python 2.7.18. As a result, the Malware Analysis component fails to process these rules correctly.

Resolution

Follow these steps if Malware Analysis is not processing the Yara files:

1. SSH to the MA host.
2. Check the Python version installed on your MA host: 

   python --version
   

   Expected Output: 

   Python 3.6.8

3. Check the Yara version: 

   yara --version
   

   Expected Output: 

   4.2.3

4. Configure the alternate Python version by selecting the number corresponding to python2: 

   alternatives --config python
   

   Expected Selection: 

   There are 4 programs which provide 'python'.
   Selection    Command
   *+ 1           /usr/libexec/no-python
      2           /usr/bin/python3
      3           /usr/bin/python2
      4           /usr/bin/python3.9
   
   Enter to keep the current selection[+], or type selection number: 3
   Here we are selecting option 3, because python2 is showing at number 3.

5. Verify the Python version is switched to python2.7: 

   python --version
   

   Expected Output: 

   Python 2.7.18

6. Check if all required files are correctly linked:

   ll /usr/bin/python
   ll /etc/alternatives/unversioned-python
   ll /usr/bin/python2
   ll /usr/bin/python2.7
   

   Expected Outputs:

   lrwxrwxrwx. 1 root root 36 Jul 19 06:23 /usr/bin/python -> /etc/alternatives/unversioned-python
   lrwxrwxrwx. 1 root root 16 Jul 19 06:23 /etc/alternatives/unversioned-python -> /usr/bin/python2
   lrwxrwxrwx. 1 root root 9 Nov 15  2023 /usr/bin/python2 -> python2.7
   -rwxr-xr-x. 1 root root 8096 Nov 15  2023 /usr/bin/python2.7

7. Check the version of libyara installed on your system:

   ll /usr/lib64/libyara*
   

   Expected Output:

   lrwxrwxrwx. 1 root root     16 Oct  8  2022 /usr/lib64/libyara.so.9 -> libyara.so.9.0.1
   -rwxr-xr-x. 1 root root 455696 Oct  8  2022 /usr/lib64/libyara.so.9.0.1

8. If libyara.so.3 is absent, upload the attached libyara.so.3.8.1 file to /usr/lib64/ directory and link it: 

   ln -s /usr/lib64/libyara.so.3.8.1 /usr/lib64/libyara.so.3

9. Verify the libyara files:

   ll /usr/lib64/libyara*
   

   Expected Output:

   lrwxrwxrwx. 1 root root     27 Jul 19 06:26 /usr/lib64/libyara.so.3 -> /usr/lib64/libyara.so.3.8.1
   -rw-r--r--. 1 root root 326296 Jul 19 06:26 /usr/lib64/libyara.so.3.8.1
   lrwxrwxrwx. 1 root root     16 Oct  8  2022 /usr/lib64/libyara.so.9 -> libyara.so.9.0.1
   -rwxr-xr-x. 1 root root 455696 Oct  8  2022 /usr/lib64/libyara.so.9.0.1

10. Move the Yara files to the watch directory:

    mv /var/netwitness/malware-analytics-server/spectrum/yara/error/rsa_mw_pdf_artifacts.yara.<timestamp> /var/netwitness/malware-analytics-server/spectrum/yara/error/rsa_mw_pdf_artifacts.yara
    mv /var/netwitness/malware-analytics-server/spectrum/yara/error/rsa_mw_pe_artifacts.yara.<timestamp> /var/netwitness/malware-analytics-server/spectrum/yara/error/rsa_mw_pe_artifacts.yara
    mv /var/netwitness/malware-analytics-server/spectrum/yara/error/rsa_mw_pe_packers.yara.<timestamp> /var/netwitness/malware-analytics-server/spectrum/yara/error/rsa_mw_pe_packers.yara
    mv /var/netwitness/malware-analytics-server/spectrum/yara/error/* /var/netwitness/malware-analytics-server/spectrum/yara/watch/

11. Verify the Yara job runs every minute and logs display all three Yara rules processed:

    tailf /var/netwitness/malware-analytics-server/spectrum/logs/spectrum.log
    

    Expected Log Output:

    2024-07-19 05:55:33,506 [jobExecutor-1(YaraFileWatchJob)] INFO  com.netwitness.malware.server.scheduler.jobs.YaraFileWatchJob - YaraFileWatch - Processing /var/netwitness/malware-analytics-server/spectrum/yara/watch/rsa_mw_pdf_artifacts.yara file...
    2024-07-19 05:57:36,165 [jobExecutor-1(YaraFileWatchJob)] INFO  com.netwitness.malware.core.carlos.handler.CarlosRequestHandler - Successfully consumed Yara content rsa_mw_pdf_artifacts.yara to /var/netwitness/malware-analytics-server/spectrum/yara/rsa_mw_pdf_artifacts.yara.
    2024-07-19 05:57:36,168 [jobExecutor-1(YaraFileWatchJob)] INFO  com.netwitness.malware.server.scheduler.jobs.YaraFileWatchJob - YaraFileWatch - The /var/netwitness/malware-analytics-server/spectrum/yara/watch/rsa_mw_pdf_artifacts.yara file has been processed successfully.
    2024-07-19 05:57:36,169 [jobExecutor-1(YaraFileWatchJob)] INFO  com.netwitness.malware.server.scheduler.jobs.YaraFileWatchJob - YaraFileWatch - Processing /var/netwitness/malware-analytics-server/spectrum/yara/watch/rsa_mw_pe_artifacts.yara file...
    2024-07-19 05:57:36,308 [jobExecutor-1(YaraFileWatchJob)] INFO  com.netwitness.malware.core.carlos.handler.CarlosRequestHandler - Successfully consumed Yara content rsa_mw_pe_artifacts.yara to /var/netwitness/malware-analytics-server/spectrum/yara/rsa_mw_pe_artifacts.yara.
    2024-07-19 05:57:36,310 [jobExecutor-1(YaraFileWatchJob)] INFO  com.netwitness.malware.server.scheduler.jobs.YaraFileWatchJob - YaraFileWatch - The /var/netwitness/malware-analytics-server/spectrum/yara/watch/rsa_mw_pe_artifacts.yara file has been processed successfully.
    2024-07-19 05:57:36,311 [jobExecutor-1(YaraFileWatchJob)] INFO  com.netwitness.malware.server.scheduler.jobs.YaraFileWatchJob - YaraFileWatch - Processing /var/netwitness/malware-analytics-server/spectrum/yara/watch/rsa_mw_pe_packers.yara file...
    2024-07-19 05:57:36,446 [jobExecutor-1(YaraFileWatchJob)] INFO  com.netwitness.malware.core.carlos.handler.CarlosRequestHandler - Successfully consumed Yara content rsa_mw_pe_packers.yara to /var/netwitness/malware-analytics-server/spectrum/yara/rsa_mw_pe_packers.yara.
    2024-07-19 05:57:36,448 [jobExecutor-1(YaraFileWatchJob)] INFO  com.netwitness.malware.server.scheduler.jobs.YaraFileWatchJob - YaraFileWatch - The /var/netwitness/malware-analytics-server/spectrum/yara/watch/rsa_mw_pe_packers.yara file has been processed successfully.