Parsers

[CristianoAlves]

H, 

I need understand, if exist the possible for crerate a new field for this logs :

{"User":"00006063180","Password":"34831123"}

we need create a field (meta) with 

User: 

Password

[DaveGlover]

Alves

You can create meta within the system.

Is this for logs or packets?

Thanks

Dave

[CristianoAlves]

Packet.

I did need konw, how create a field with regex.

Is possible ?

I don't know how create this field.

[DaveGlover]

Alves

This will have to be written in LUA for the packet parsing.

Does the value already parse? Do you want to do finer parsing than what is there?

Dave

[CristianoAlves]

I know nothing about LUA.
What i need do for beginning ?

Or can you help create this ?

[WilliamMotley1]

This is untested but syntactically correct.  It should get you at least close.

local userpass = nw.createParser("userpass", "custom for Cristiano Alves")
userpass:setKeys({
  nwlanguagekey.create("user"),
  nwlanguagekey.create("password"),
})
function userpass:streamBegin()
  self.streamVars = {}
end
function userpass:onUser(token, first, last)
  self.streamVars.userBegin = last + 1
end
function userpass:onPassword(token, first, last)
  if self.streamVars.userBegin then
    local payload = nw.getPayload(self.streamVars.userBegin, last + 64)
    local userEnd = first - self.streamVars.userBegin
    local passwordBegin = last - self.streamVars.userBegin + 2
    self.streamVars.userBegin = nil
    if payload then
      local passwordEnd = payload:find('"}', passwordBegin, -1)
      if passwordEnd then
        local userString = payload:tostring(1, userEnd)
        local passwordString = payload:tostring(passwordBegin, passwordEnd - 1)
        if user and password then
          nw.createMeta(self.keys["user"], userString)
          nw.createMeta(self.keys["password"], passwordString)
        end
      end
    end
  end
end
userpass:setCallbacks({
  [nwevents.OnStreamBegin] = userpass.streamBegin,
  ['{"User":"'] = userpass.onUser,
  ['","Password":"'] = userpass.onPassword,
})

[CristianoAlves]

Tnks Motley.

Can i save with .lua ?

[WilliamMotley1]

Yep.  Save it with a .lua extension, place it in /etc/netwitness/ng/parsers/ then reload parsers.

[CristianoAlves]

Hi, Motley.

We upload script.lua and reload parsers, but don't see events with parsers.

Exist one troubleshooting this ?

[WilliamMotley1]

Without a pcap to test against it's difficult to troubleshoot.  And since it would obviously contain sensitive information, I expect you'd be hesitant to share one.

Next best thing would be to add some lines to the parser that log what the parser is doing.  Caveat is that the log could easily be flooded depending on how many sessions the parser is seeing.

Load this, watch the log ('tail -f /var/log/messages | grep userpass'), and reload parsers.

local userpass = nw.createParser("userpass", "custom for Cristiano Alves")
nw.logInfo("userpass - parser loaded")
userpass:setKeys({
  nwlanguagekey.create("user"),
  nwlanguagekey.create("password"),
})
function userpass:streamBegin()
  self.streamVars = {}
end
function userpass:onUser(token, first, last)
  nw.logInfo("userpass - saw onUser")
  self.streamVars.userBegin = last + 1
end
function userpass:onPassword(token, first, last)
  nw.logInfo("userpass - saw onPassword")
  if self.streamVars.userBegin then
    nw.logInfo("userpass - have userBegin")
    local payload = nw.getPayload(self.streamVars.userBegin, last + 64)
    local userEnd = first - self.streamVars.userBegin
    local passwordBegin = last - self.streamVars.userBegin + 2
    self.streamVars.userBegin = nil
    if payload then
      nw.logInfo("userpass - have payload")
      local passwordEnd = payload:find('"}', passwordBegin, -1)
      if passwordEnd then
        nw.logInfo("userpass - found passwordEnd")
        local userString = payload:tostring(1, userEnd)
        local passwordString = payload:tostring(passwordBegin, passwordEnd - 1)
        if user and password then
          nw.logInfo("userpass - user '" .. user .. "', password '" .. password .. "'")
          nw.createMeta(self.keys["user"], userString)
          nw.createMeta(self.keys["password"], passwordString)
        end
      end
    end
  end
end
userpass:setCallbacks({
  [nwevents.OnStreamBegin] = userpass.streamBegin,
  ['{"User":"'] = userpass.onUser,
  ['","Password":"'] = userpass.onPassword,
})