Writing a Packet Parser to Extract Title Meta

[david_waugh]

Hello

Im stuck writing a packet parser that will extract some meta.

The session is an SMB print job.

In wireshark where I look at the packets I can see the %%Title: that equates to the line I want to capture.

When I look at the Hex i can see:

I dont seem to be able to get my packet parser to match though.

I have tried:

["%%%%Title:"] = SMBTitle.header,  --(% Needs to be escaped)

["Title:" ]= SMBTitle.header,

-- Hex: 25 25 54 69 74 6c 65 3a 20
-- Which is decimal
-- 37 37 84 105 116 108 101 58 32

So i have tried

["\37\37\84\105\116\108\101\58\32"] = SMBTitle.header

I have just tried

["SMB"]=SMBTitle.header then the parser works (eg the callback function fires)

Here is the actual parser

local parserName = "smbtitle"
local parserVersion = "2018-05-25"

local SMBTitle = nw.createParser(parserName, "smb title",139)

nw.logDebug(parserName .. " " .. parserVersion)

--[[
DESCRIPTION

Looks for lines such as
%%Title:
And then writes the remaining text into the filename meta key.
VERSION

1.0 First Version

DEPENDENCIES

none

CONFLICTS

STANDARD INDEX KEYS

CUSTOM INDEX KEYS

none

ALERT.ID

none

NOTES

none

OPTIONS

none

IMPLEMENTATION

Finds title meta from SMB Sessions

TOD

none

--]]

function SMBTitle:sessionBegin()
DebugFlag=True
myRandom=math.random(100000000)
nw.logInfo(myRandom .. " SMBTitle: Session Detected")
end

SMBTitle:setKeys({
nwlanguagekey.create("filename")
})

function SMBTitle:logger(mymessage, myvariable)
if True then
if mymessage and myvariable then
nw.logInfo(myRandom .. " SMBTitle: " .. mymessage .. myvariable)
elseif mymessage then
nw.logInfo(myRandom .. " SMBTitle: " .. mymessage .. "is nil")
else
nw.logInfo(myRandom .. " SMBTitle: Both Variables Nil")
end
end
end

function SMBTitle:header(token,first,last)
nw.logInfo(myRandom .. " SMBTitle: Title Matched")
local payload = nw.getPayload()

nw.logInfo("payload string:" .. "hello")
nw.logInfo("first" .. tostring(first))
nw.logInfo("last" .. tostring(last))

nw.createMeta(self.keys.filename,"found")
end

SMBTitle:setCallbacks({
[nwevents.OnSessionBegin] = SMBTitle.sessionBegin,
-- Look for SMBTitle in a session
-- We need to escape % with % so that % become %%
-- We are looking for %%Title:
-- Which is Hex: 25 25 54 69 74 6c 65 3a 20
-- Which is decimal
-- 37 37 84 105 116 108 101 58 3
 ["\37\37\84\105\116\108\101\58\32"] = SMBTitle.header,

})

Any Ideas what I am doing wrong here William Motley‌?

Once i know my call back is firing correctly I will extract the meta.

Ive attached my test pcap which contains the capture info.

[Anonymous]

David,

Thank you for attempting this parser.  I think it highlights several things.  

I wound up having to modify the parser initially to get it to work, however, there were some challenges that had to be overcome.  The default configuration for the decoder is to scan for tokens at a depth of 128 kB.  The PCAP and the token you were looking for was likely too deep in the session, meaning the parser would never fire.  The first way to solve this would be to change the default setting for "parse maximum bytes" from 128 KB to 0.  I do not recommend this.

In order to get the traffic we wanted, I had to create tokens that would ideally match earlier in the session.  This involved having to re-write the parser.  Furthermore, it meant having to grab all the session payload, which again, I don't usually recommend, but didn't see an alternative here.

****************************

local lua_smbtitle = nw.createParser("lua_smbtitle", "Find SMB Title", "139")

-- declare the meta keys we'll be registering meta with
lua_smbtitle:setKeys({
nwlanguagekey.create("filename")
})

function lua_smbtitle:sessionBegin(token,first,last)
protocol = nil
end

function lua_smbtitle:protocol(token, first, last)
protocol = 1
end

function lua_smbtitle:spool(token, first, last)
if protocol == 1 then
--nw.logInfo("*** FOUND SPOOL ***")
current_position = last + 1
local payload = nw.getPayload()
local startf, startl = payload:find("%%Title: ", 1, -1)
if startl then
--nw.logInfo("*** FOUND SMB TITLE ***")
local delim = payload:find("\13\10", startl + 1, -1)

if delim then
--nw.logInfo("*** FOUND SMB TITLE DELIM ***")
local foundtitle = payload:tostring(startl, delim -1)
if foundtitle then
nw.createMeta(self.keys["filename"], foundtitle)
--nw.logInfo("*** SMB TITLE: " .. foundtitle .. " ***")
end
end

end
end
end

lua_smbtitle:setCallbacks({
[nwevents.OnSessionBegin] = lua_smbtitle.sessionBegin,
["\255SMB"] = lua_smbtitle.protocol, -- 0xFF S M B
["\254SMB\64\00"] = lua_smbtitle.protocol, -- 0xFE S M B 0x40 0x00
["\83\0\112\0\111\0\111\0\108"] = lua_smbtitle.spool, -- S.p.o.o.l 83 0 112 0 111 0 111 0 108

})

****************************

This was my only sample, so I am not sure if this would match in all cases.

Hope this helps.

Chris

[david_waugh]

Thanks very much Christopher Ahearn‌ This has saved me hours of work!!

[david_waugh]

Ive hit a snag. I've attached another PCAP which shows me printing a test document.

I'm wondering if the information is just too deep inside the stream.

The information is in packet 9306.

The title of the document is 

%%Title: Microsoft Word - Test Print 12345.docx

[david_waugh]

I've been reading the parser book and it looks to be that it is the "assembler timeout packets" value that stops the session being read.

If I set it to 0 then my value is created.

If i set it to 60 then the value is not created.

I am going to experiment with this value.

[david_waugh]

It looks like the session lifetime is 1201 seconds in the pcap I uploaded.

Useful info here:

https://community.rsa.com/docs/DOC-80857 

[WilliamMotley1]

This worked for me, using the "testing and debugging" decoder tunings from the Parsers book:

local printTitle = nw.createParser("printTitle", "")
printTitle:setKeys({
 nwlanguagekey.create("title")
})
function printTitle:onTitle(token, first, last)
 local payload = nw.getPayload(last + 1, last + 255)
 if payload then
     local title = payload:tostring()
     title = string.match(title, "^%S-([^\010^\013]+)")
     if title and #title ~= 0 then
         nw.createMeta(self.keys.title, title)
     end
 end
end
printTitle:setCallbacks({
    ["%%%%Title: "] = printTitle.onTitle
})

[david_waugh]

Ive called a halt on this now.

It seems that with the setting above then only the first 60 seconds of a session are parsed. I could increase it but it would then affect all sessions.

I would then have to go from approximately 1 minute of meta for a session, to potentially over 20 minutes of meta for a session (if I set the timeout to 1200). The problem is that the SMB sessions are very long and persistent so even setting to 1200 minutes might not be long enough.

I looked at 

CIFS and SMB Timeouts in Windows – Microsoft Open Specifications Support Team Blog 

and a 15 Minute idle timeout was mentioned. 

[WilliamMotley1]

Keep in the mind the differences between testing with a pcap and the "real world".  Assembler timeouts and max don't matter that much because any remainder of a session isn't lost - rather it becomes (to decoder) a different session.  The ramification is that the parse.bytes thresholds are in effect reset for that "new" session.  As well, when testing a parser typically that is the only parser enabled.  Whereas in the real world many parsers are enabled, and the parse.bytes thresholds are for any enabled parser.  So long as any token from any parser matches within parse.bytes.min and parse.bytes.max, decoder will continue looking for tokens.

When testing a parser with successfully with non-default tunings you know that the parser will work if given the opportunity.  The example parser I posted above may work with that pcap with default tunings - I didn't try.  Even if not, it may work if you also enable a bunch of other parsers as well - again, I didn't try.