Parse correct device.ip from forwarded logs

[KevinHraybi]

Hello,

I already took a look at How to override "device.ip" meta with the right one? 

We're using TCP Syslog for security and reglamentory reasons, we can't spoof the source ip.

We're using Balabit log forwarders to forward a huge quantity of logs into our VLC's. "Device.ip" is getting populated with the IP's of the log forwarders which is causing a lot of problems..

Has anybody found a solution for this ?

Thank you

[DavidWaugh1]

Hi Kevin,

Thank you for your post.

We have a document here that was written by Dave Glover

if you change the below to use tcp instead of udp does that work?

Can you verify with

tcpdump –A port 514

that the template is taking effect?

Here is the document:

Use at your own risk.  This is not blessed by engineering, however …….

Using the following templates does not require the spoofing settings to be used in the relays.

I prefer this method over the spoofing method as you will see below the relay server ip is not lost and can be quite useful in troubleshooting the flow of logs though the relays

Rsyslog format – Needs Rsyslog 6.x and above for the function %timestamp:::date-unixtimestamp% to work (if you are on version 5.x this will no longer work as a change was made in 10.3.3 to require the unix timestamps)

$EscapeControlCharactersOnReceive off  à this is needed for snare, but doesn’t hurt anything by being there

$template NWLDfmt,"[][][%FROMHOST-IP%][%timestamp:::date-unixtimestamp%][] %rawmsg%\n"

*.* @@10.12.236.228:514;NWLDfmt

If you are using Syslog-NG the settings are as follows:

Make sure of the following

keep_hostname(no)

use_dns (no)

destination d_syslog_RSA_udp { udp("1.1.1.1" port (514) template("[][][$HOST][$UNIXTIME][] $MSGHDR$MSG\n")); };

If you are relaying snare, it uses control characters in the header which mess with syslog-ng so you need to use the following template for NG

destination d_syslog_RSA_udp { udp("1.1.1.1" port (514) template("[][][$HOST][$UNIXTIME][] $DATE $TIME $HOST $MSGHDR$MSG\n")); };

To check what is being received by the log collector/VLC you can run:

tcpdump –A port 514

And you should see something like the following (this is coming from snare):

[][][10.12.236.206][1398440147][] <13>Apr 25 11:35:47 test-r2 MSWinEventLog,0,Security,87,Fri Apr 25 11:35:46 2014,4719,Microsoft-Windows-Security-Auditing,WORKGROUP\TEST-R2$,N/A,Success Audit,test-r2,Audit Policy Change,,System audit policy was changed.    Subject:   Security ID:  S-1-5-18   Account Name:  TEST-R2$   Account Domain:  WORKGROUP   Logon ID:  0x3e7    Audit Policy Change:   Category:  Account Logon   Subcategory:  Kerberos Service Ticket Operations   Subcategory GUID: {0CCE9240-69AE-11D9-BED3-505054503030}   Changes:  Success Added, Failure added,69

This format will populate the device.ip with the value from the third [].  What is great about using this format is the syslog forwarder system is not lost, it is called out in the forwarder meta key as shown below

Hope that helps

[DavidWaugh1]

In my syslog-ng.conf I have the following:

@version:3.2

# syslog-ng configuration file.
#
# This should behave pretty much like the original syslog on RedHat. But
# it could be configured a lot smarter.
#
# See syslog-ng(8) and syslog-ng.conf(5) for more information.
#

options {
flush_lines (0);
time_reopen (10);
log_fifo_size (1000);
long_hostnames (off);
use_dns (no);
use_fqdn (no);
create_dirs (no);
keep_hostname (yes);

};

source s_sys {
udp(ip(0.0.0.0) port(514) encoding("WINDOWS-1252"));
};

source s_syschinese {
udp(ip(0.0.0.0) port(5514) encoding("GB2312"));
};

source s_tcp_sys {
tcp(ip(0.0.0.0) port (514) keep_hostname(no) use_dns (no));
};

destination d_cons { file("/dev/console"); };
destination d_mesg { file("/var/log/messages"); };
destination d_auth { file("/var/log/secure"); };
destination d_mail { file("/var/log/maillog" flush_lines(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };
destination d_net {udp( "192.168.123.242");};
destination d_syslog_RSA_tcp{tcp("192.168.123.241" port (514) template("[][][$HOST][$UNIXTIME][] $DATE $TIME $HOST $MSGHDR
$MSG\n")); };
filter f_kernel { facility(kern); };
filter f_default { level(info..emerg) and
not (facility(mail)
or facility(authpriv)
or facility(cron)); };
filter f_auth { facility(authpriv); };
filter f_mail { facility(mail); };
filter f_emergency { level(emerg); };
filter f_news { facility(uucp) or
(facility(news)
and level(crit..emerg)); };
filter f_boot { facility(local7); };
filter f_cron { facility(cron); };

#log { source(s_sys); filter(f_kernel); destination(d_cons); };
log { source(s_sys); filter(f_kernel); destination(d_kern); };
log { source(s_sys); filter(f_default); destination(d_mesg); };
log { source(s_sys); filter(f_auth); destination(d_auth); };
log { source(s_sys); filter(f_mail); destination(d_mail); };
log { source(s_sys); filter(f_emergency); destination(d_mlal); };
log { source(s_sys); filter(f_news); destination(d_spol); };
log { source(s_sys); filter(f_boot); destination(d_boot); };
log { source(s_sys); filter(f_cron); destination(d_cron); };
log{ source(s_sys); destination(d_net); };
log{ source(s_syschinese); destination(d_net); };
log{ source(s_tcp_sys); destination(d_syslog_RSA_tcp);};
# vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:

The important lines are in red. Here i have set up a tcp listener on port 514 and forward it with my template to server 192.168.123.241

In tcpdump I can see the packets are rewritten:

...W...[][][192.168.123.177][1483982281][] Jan 9 17:18:01 192.168.123.177 systemd: Removed slice user-48.slice.
17:18:01.533425 IP 192.168.123.177.55148 > centos6.waugh.local.shell: Flags [P.], seq 577:638, ack 1, win 229, options [nop,nop,TS val 27740592 ecr 546941977], length 61

[KevinHraybi]

Hi David,

It works! 

We will be testing this configuration on our event sources. I will get back to you soon.

Thanks a lot

Kevin

[ThomasGaultier]

Hi David,

I would like to know what correspond the index value in the table ?

[Index1][Index2][Index3][index4[Index5]

We have made the test on Index3, if it is an IP address its goes in device.ip and if it is else it goes in device.host.

Is it possible to send both device.ip and device.host using the table above?

Also can we send other values using this table?

Regards,

Thomas (Kevin mate)

[AnujShrivastava]

David Waugh‌   

Hi David,

I am using Rsyslog v.7 and i am trying to forwared cisco asa logs from our syslog server to VLC.

VLC IP: 10.201.12.80

ASA IP: 10.201.3.103

-------------------------------is this correct ?------------------------

$template NWLDfmt,"[][][%10.201.3.103%][%timestamp:::date-unixtimestamp%][] %rawmsg%\n"

*.* @@10.201.12.80:514;NWLDfmt

-------------------------------Shall i use it as it is?------------------------

$template NWLDfmt,"[][][%FROMHOST-IP%][%timestamp:::date-unixtimestamp%][] %rawmsg%\n"

*.* @@10.201.12.80:514;NWLDfmt

i tried both configs but not able to see logs, 

please Help me.