Mail From email.src Custom Parser

[RSAAdmin]

Hi All,

I have been having some serious frustrations trying to populate the email.src meta field.

I have asked RSA directly how to populate this and they informed me that its used for Logs not packets. however the email parser that comes with LIVE content only populates 'email' making it almost impossible to drill based on a sender. (for instance only wanting outbound mail instead of everything an email address sent or received.

Anyhow, I was told I would need a custom parser to achieve my objective, and this is what I have come up with:

<parser Name="From Mail Parser" desc="Version 0.1">
<!-- Parser Created by David McClennon @ Integralis.com -->
<declaration>
<!-- Declare meta keys to be used in this parser-->
<meta format="Text" key="email.dst" name="recipient"/>
<meta format="Text" key="email.src" name="sender"/>
<!-- Mail from and To identifiers to Match -->
<token name="MailFrom" value="MAIL FROM:<" options="linestart"/>
<!-- Position Identifiers -->
<number name="vMailPosition" scope="stream" />
<number name="vMailStop" scope="steam" />
<!-- Variables for holding strings -->
 <string name="vEmailAddress" scope="stream" />
</declaration>
<!-- Evaluate From mail match -->
 <match name="MailFrom">
 <!-- Find ">" within 512 bytes -->
 <find name="vMailPosition" value="&#x3C;" length="512">
 <!-- when found, assign contents between current potistion and vmailPosition to vEmailAddress -->
 <read name="vEmailAddress" length="$vMailPosition">
 <register name="sender" value="$vEmailAddress" />
 </read>
 </find>
 </match>         
    </match>
</parser>

But for love nor money can I get it to work properly. I have tried a number of match combinations including encoding the "MAIL FROM:" in various formats, but cannot seem to get it to match or register the email address in the email.src register.

If anyone can see what I have done wrong, or have any suggestions, I would be most grateful.

Regards

Dave

[RSAAdmin]

I guess you might be able to do it with a parser, but as long as you can identify src and receiver, you can make a simple rule to determine directionality.  I typically use org.dst exists to detect traffic going to a public IP address.  If you have public IPs, then you can do org.dst exists && org.dst !=YOURORG 

Conversely, inbound would be org.src != YOURORG

[RSAAdmin]

run into a weird issue. after adding the parser and keys I am finding only "Email content, encoding and MessageID" are populating.

I removed a few parts of the parser that were either commented out, or potentially causing the pointer to missing the inital "MAIL FROM" but Im still baffled.

<parser name="advanced_email" desc="advanced_email">

    <declaration>

      <!--

****

META DECLARATIONS

****

-->

      <meta name="fromaddy" key="email.fromaddy" format="Text" />

      <meta name="toaddy" key="email.toaddy" format="Text" />

      <meta name="emailip" key="email.ip" format="Text" />

      <meta name="emaildomain" key="email.domain" format="Text" />

      <meta name="mailer" key="email.mailer" format="Text" />

      <meta name="content" key="email.content" format="Text" />

      <meta name="encoding" key="email.encoding" format="Text" />

      <meta name="messid" key="email.messageid" format="Text" />

<meta name="ioc" key="soc.ioc" format="Text" />

<meta name="misc" key="soc.misc" format="Text" />

      <!--

****

STRINGS

****

-->

      <string name="myString_Temp" />

      <string name="myString_Empty" scope="constant"/>

      <!--

****

NUMBERS

****

-->

      <number name="myNum_offset"/>

      <number name="myNum_offset2"/>

      <number name="myNum_offset3"/>

      <number name="myNum_FoundEHLO" scope="stream"/>

      <number name="myNum_FoundPopLogic" scope="stream"/>

      <number name="myNum_FoundLogin" scope="stream"/>

      <number name="myNum_TriggeredAlertAlready" scope="stream"/>

      <!--

****

TOKENS

****

-->

      <token name="myToken_EHLO" value="EHLO" options="linestart" />

      <token name="myToken_EHLO" value="HELO" options="linestart" />

      <token name="myToken_FROM" value="MAIL From: &#x3C;" options="linestart" />

      <token name="myToken_FROM" value="MAIL From:&#x3C;" options="linestart" />

<token name="myToken_FROM" value="&#x4D;&#x41;&#x49;&#x4C;&#x20;&#x46;&#x52;&#x4F;&#x4D;&#x3A;&#x3C" options="linestart"/>

<token name="myToken_FROM" value="&#x4D;&#x41;&#x49;&#x4C;&#x20;&#x46;&#x52;&#x4F;&#x4D;&#x3A;&#x20;&#x3C" options="linestart"/>

<token name="myToken_FROM" value="MAIL FROM:<" options="linestart"/>

<token name="myToken_FROM" value="MAIL FROM: <" options="linestart"/>

      <token name="myToken_TO" value="RCPT To: &#x3C;" options="linestart" />

      <token name="myToken_TO" value="RCPT To:&#x3C;" options="linestart" />

<token name="myToken_TO" value="MAIL TO: <" options="linestart"/>

<token name="myToken_TO" value="MAIL TO:<" options="linestart"/>

<token name="myToken_TO" value="&#x52;&#x43;&#x50;&#x54;&#x20;&#x54;&#x4F;&#x3A;&#x3C" options="linestart"/>

<token name="myToken_TO" value="&#x52;&#x43;&#x50;&#x54;&#x20;&#x54;&#x4F;&#x3A;&#x20;&#x3C" options="linestart"/>

      <token name="myToken_Mailer" value="X-mailer: " options="linestart" />

      <token name="myToken_Mailer" value="X-Mailer: " options="linestart" />

      <token name="myToken_Forefront" value="X-Forefront-Antispam-Report: CIP:" options="linestart" />

      <token name="myToken_MessageID" value="Message-ID:" options="linestart" />

      <token name="myToken_ContentType" value="Content-Type:" options="linestart" />

      <token name="myToken_Encoding" value="Content-Transfer-Encoding:" options="linestart" />

    </declaration>

    <!--

****

FILTER MATCH

****

-->

    <match name="myToken_EHLO">

      <assign name="myNum_FoundEHLO" value="1"/>

    </match>

    <!--

****

REMAINING MATCHES

****

-->

    <match name="myToken_FROM">

      <if name="myNum_FoundEHLO" notequal="1">

        <end/>

      </if>

      <find name="myNum_offset" value="&#x3E;" length="64">

        <read name="myString_Temp" length="$myNum_offset">

<if name="myString_Temp" notequal="$myString_Empty">

<register name="fromaddy" value="$myString_Temp" />

<assign name="myString_Temp" value="$myString_Empty"/>

</if>

        </read>

      </find>

    </match>

    <match name="myToken_TO">

      <if name="myNum_FoundEHLO" notequal="1">

        <end/>

      </if>

      <find name="myNum_offset" value="&#x3E;" length="64">

        <read name="myString_Temp" length="$myNum_offset">

<if name="myString_Temp" notequal="$myString_Empty">

<register name="toaddy" value="$myString_Temp" />

<assign name="myString_Temp" value="$myString_Empty"/>

</if>

        </read>

      </find>

    </match>

    <match name="myToken_Mailer">

      <if name="myNum_FoundEHLO" notequal="1">

        <end/>

      </if>

      <find name="myNum_offset" value="&#x0D;" length="64">

        <read name="myString_Temp" length="$myNum_offset">

<if name="myString_Temp" notequal="$myString_Empty">

<register name="mailer" value="$myString_Temp" />

<assign name="myString_Temp" value="$myString_Empty"/>

</if>

        </read>

      </find>

    </match>

    <match name="myToken_ContentType">

      <if name="myNum_FoundEHLO" notequal="1">

        <end/>

      </if>

      <find name="myNum_offset" value="&#x0D;" length="64">

        <read name="myString_Temp" length="$myNum_offset">

<if name="myString_Temp" notequal="$myString_Empty">

<register name="content" value="$myString_Temp" />

<assign name="myString_Temp" value="$myString_Empty"/>

</if>

        </read>

      </find>

    </match>

     <match name="myToken_Encoding">

      <if name="myNum_FoundEHLO" notequal="1">

        <end/>

      </if>

      <find name="myNum_offset" value="&#x0D;" length="64">

        <read name="myString_Temp" length="$myNum_offset">

<if name="myString_Temp" notequal="$myString_Empty">

<register name="encoding" value="$myString_Temp" />

<assign name="myString_Temp" value="$myString_Empty"/>

</if>

        </read>

      </find>

    </match>

    <match name="myToken_MessageID">

      <if name="myNum_FoundEHLO" notequal="1">

        <end/>

      </if>

      <find name="myNum_offset" value="&#x0D;" length="64">

        <read name="myString_Temp" length="$myNum_offset">

<if name="myString_Temp" notequal="$myString_Empty">

<register name="messid" value="$myString_Temp" />

<assign name="myString_Temp" value="$myString_Empty"/>

</if>

        </read>

      </find>

    </match>

  </parser>

No doubt my tampering broke it more, but the mail from and to where not populating in its native form as per what you posted Fielder.

I get the feeling Netwitness just don't want me to know who the email's are from

Regards

Dave

[RSAAdmin]

Make sure your pcaps have matching tokens being called in the parser.

[RSAAdmin]

I have deployed it on my decoder, and its seeing loads of email traffic, just for some reason its not pulling the email's out. I confirmed the packets have "MAIL FROM:<" in them, meaning this parser should be hitting the Match statement.

... Confused.

[RSAAdmin]

on a side note. I still have the Enhanced Mail parser deployed via live. is that likely to cause any conflicts?

Dave

[Anonymous]

i cant get this to work! i use a pcap that contains some smtp sessions but non of the new keys are populated very frustrating