The "Log Parser Diff Tool" is a newly developed tool, by request of our customers, that performs a “difference” operation between any 2 log parsers and produces the result as an html report. The report shows the newly added, updated and deleted headers and message id’s in the parser.
The differences found could be in content definition, functions used in a message or content definition in a header. All of the differences in the value-map section of the parser is also listed.
A user can run the LogParserDiffTool on Microsoft Windows and Apple Mac systems. You must have a version of Python 3 running on your system in order to properly run the tool. You can download Python version 3 here: https://www.python.org
Please visit the following sites for more details about using Python in Windows or Apple environments:
Windows: https://docs.python.org/3/using/windows.html
Mac: https://docs.python.org/3/using/mac.html
Syntax for the Log Parser Diff Tool:
A user invokes the tool slightly different based on the OS where the Python script is being run.
For example:
On a Mac, the syntax is as follows:
$ python3 LogParserdiffTool.py <old> <new> [-all]
On Windows, the syntax is as follows:
c:\> LogParserdiffTool.py <old> <new> [-all]
Or
c:\> python3 LogParserdiffTool.py <old> <new> [-all]
<old>: Older version of the parser information. It can be a file or the envision content.
<new>: Newer version of the parser information. It can be a file or envision content.
Note: It is important to maintain the parameter expectation of old followed by new (LogParserdiffTool.py <old> <new>)
Usage:
<old> and <new> argument can be the following:
- Netwitness /etc folder in the Log Decoder
- Individual Parser files (in *.envision Format when downloaded from Live)
- Individual Parser files (in *msg.xml Format)
- Table-map xml files can also be compared.
To use the script, you need two of the same items from the above list, one to serve as a baseline (old), and one to serve as the updated version (new) that contains XML differences from the baseline version.
Without the -all argument, some devices which are mentioned below, will be excluded to reduce execution time and output file size. These devices are typically signature-based with large and frequent updates; hence their content updated will not be included in the output report unless the user uses the -all argument.
- Ciscoidsxml
- dragonids
- intrushield
- iss
- netscreenidp
- snort
- tippingpoint
You can include these event sources in the report by specifying the -all parameter when you run the script.
Examples of How to Use the Tool:
Example 1
This example assumes:
- You have two copies of ciscoasamsg.xml, from two different releases.
- You have placed the older copy (the one from the earlier release) into a folder named 83.
- You have placed the newer copy (the one from the later release) into a folder named 84.
In this situation, if you are using a PC, you open a Command Prompt and run the following:
c:\temp> $ LogParserdiffTool.py 83\ciscoasamsg.xml 84\ciscoasamsg.xml
Running this command generates a report that contains the differences (if any) for the ciscoasamsg parser.
Example 2
This example assumes:
- You have two copies of table-map.xml, from two different releases.
- You have placed the older copy (the one from the earlier release) into a folder named 83.
- You have placed the newer copy (the one from the later release) into a folder named 84.
In this situation, if you are using a PC, you open a Command Prompt and run the following:
c:\temp> $ LogParserdiffTool.py 83\table-map.xml 84\table-map.xml
Running this command generates a report that contains the differences (if any) between the two versions of the tablemap.xml file.
Example 3
This example assumes:
- You have two copies of ciscoasamsg.envision, from two different releases.
- You have placed the older copy (the one from the earlier release) into a folder named 83.
- You have placed the newer copy (the one from the later release) into a folder named 84.
In this situation, if you are using a PC, you open a Command Prompt and run the following:
c:\temp> $ LogParserdiffTool.py 83\ ciscoasamsg.envision 84\ ciscoasamsg.envision
Running this command generates a report that contains the differences (if any) between the two versions of the ciscoasamsg.envision parser.
Example 4
This example assumes:
- You have two copies of unzipped envision files in the Log Decoder Directory Structure (…/etc/devices), from two different releases or Log Decoders where there are multiple parsers
- You have placed the older copy (the one from the earlier release) into a folder named old
- You have placed the newer copy (the one from the later release) into a folder named new
In this situation, if you are using a PC, you open a Command Prompt and run the following:
c:\temp> $ LogParserdiffTool.py old\ new\
Running this command generates a report that contains the differences (if any) between the multiple parsers of different versions.
This new tool can come in very handy, as all the necessary changes made can be viewed in a single report, which are demonstrated in the images below.
Sample Reports:
Multiple Parsers report:
Overall Result:
Detailed Header Results:
Detailed Message Results:
Detailed ValueMap Results:
We hope you find this tool useful and welcome any feedback or suggestions for improvement. Please feel free to leave any constructive feedback in the comments below and if you have any questions, please reach out to parser_content_team@rsa.com.
Assisted by
