2014-12-14 04:50 AM
Hi,
I'm working with SA 10.3 and i have multiple IIS sites that i'm transferring the logs to SA using SFTP agent.
The problem i'm facing with is that most of the logs are not parsing in SA.
For example, the following log:
%IIS-4-: date="2014-12-14",time="06:17:09",s-ip="192.168.121.30",cs-method="GET",cs-uri-stem="/",cs-uri-query="-",s-port="444",cs-username="-",c-ip="192.168.121.252",cs(User-Agent)="-",sc-status="200",sc-substatus="0",sc-win32-status="0",time-taken="31",
SA will not parse the webpage and only show me the information inside the msg container:
sessionid=11000751318
time=2014-12-14T08:19:35.0
size=322
lc.cid="hostname***"
forward.ip=127.0.0.1
device.ip=192.168.121.30
medium=32
device.type="microsoftiis"
device.class="Web Logs"
header.id="0001"
ip.dst=192.168.121.30
ip.dstport=444
ip.src=192.168.121.252
result.code="200"
result="0"
event.time=2014-12-14 06:17:09.000
msg="date="2014-12-14",time="06:17:09",s-ip="192.168.121.30",cs-method="GET",cs-uri-stem="/",cs-uri-query="-",s-port="444",cs-username="-",c-ip="192.168.121.252",cs(User-Agent)="-",sc-status="200",sc-substatus="0",sc-win32-status="0",time-taken="31","
level=6
msg.id="GET"
event.cat.name="Content.Web Traffic"
I followed the documentation here: http://sadocs.emc.com/@api/deki/files/40485/Microsoft_IIS.pdf
Anyone knows how can i fix this?
2014-12-16 01:59 PM
Sorry, reread your initial post.
So the issue you are having is likely two fold.
First step: table-map.xml
By default it looks like below. You will notice the flags is set to transient. This essentially means that when the parser sees the web.page praser, it is going to ignore it and not write it to disk as meta. We want to change this.
<mapping envisionName="webpage" nwName="web.page" flags="Transient" envisionDisplayName="WebPage"/>
Below is the changes you should make, we are essentially just going to be switching the flag to say None.
<mapping envisionName="webpage" nwName="web.page" flags="None" envisionDisplayName="WebPage"/>
Now that we have done this we need to tell our decoders and concentrators that their is new data that you need to index. This is done via the gui.
Decoder:
Under Administration -> Services select your device and view the config section of that device.
Click the files tab and select index-logdecoder-custom.xml
With in that file, if nothing exists paste the following.
<?xml version="1.0" encoding="utf-8"?>
<language level="IndexNone" defaultAction="Auto">
<key description="Web Page" format="Text" level="IndexNone" name="web.page" />
</language>
If possible restart the services to ensure everything loads correctly.
Concentrator:
Under Administration -> Services select your device and view the config section of that device.
Click the files tab and select index-concentrator-custom.xml
With in that file, if nothing exists paste the following.
<?xml version="1.0" encoding="utf-8"?>
<language level="IndexNone" defaultAction="Auto">
<key description="Webpage" level="IndexValues" name="web.page" format="Text" valueMax="250000" defaultAction="Open"/>
</language>
Again, restart the services if possible. It is also a good idea at this point to restart the broker, and web service, this will force reloads.
Note, this will not allow you to look at the data historically. You should now notice new information populating for web.page.
2014-12-17 02:53 AM
Yes
It is working .. I can't tell you how happy i am that it is finally working.
I did as you told, i opened the table-map.xml file, and changed the webpage line from Transient to None.
Then i changed the index-logdecoder-custom.xml file (which was empty) , and the index-concentrator-custom.xml file (which wasn't empty so i just added the line: "<key description="Webpage" level="IndexValues" name="web.page" format="Text" valueMax="250000" defaultAction="Open"/>" to the right place)
After that all i had to do is restart the services:
restart nwlogdecoder
restart nwconcentrator
Then i waited for a few hours and when i checked the investigation tab i could see the webpage (also when making reports)
Thank you for all the help.
Best Regards,
Guy Miretzky
2014-12-17 07:08 AM
Glad it worked