How to send logs to RSA from Graylog?

[FredBlaise]

Hi,

We are running SA 10.5.

Currently, most of our logs are sent to Graylog, helping out DevOps teams on operational matters. Windows logs are sent up with NXlog, we have some syslog, app logs are sent with graylog-collector in GELF or plain format.

In order to support our security analysts as well, using SA, I'd like to send the raw logs that are coming to the Graylog right before they enter, for example with Logstash to split the flow.

My question is: Can I send a stream of logs directly to the VLC's rabbitMQ? What would be alternative ways to push logs out to the RSA chain otherwise? Straight to the decoder somehow? or ESA?

Thank you.

Best,

fred

[DavidWaugh1]

Hello

As long as you can get the logs into security analytics then we can parse them.

If the logs reside as files on the disk then you could upload them with the sftp agent.

If you can send them directly as syslog then that would be even easier.

Would any of these ways be possible?

Once you have the method of getting the logs into SA we can assist with the rest.

Sent from my iPhone

[DavidWaugh1]

Hi Can you configure

Graylog Marketplace

This has an output plugin that will send graylogs to Syslog which we can then ingest into SA.

The log messages look fairly structured for example:

<14>1 2016-03-31T19:31:46.358Z graylog unknown - nginx [all@0 request_verb="GET" remote_addr="192.168.1.37" response_status="404" from_nginx="true" level="6" connection_requests="1" http_version="1.1" response_bytes="1906" source="nginx" message="GET /test1/2 HTTP/1.1" gl2_source_input="566c96abe4b094dfbc2661a8" version="1.1" nginx_access="true" http_user_agent="Wget/1.15 (linux-gnu)" remote_user="-" connection_id="1755" http_referer="-" request_path="/test1/2" gl2_source_node="bebd092c-85d7-49a3-8188-f7af734747fb" _id="34cb0f40-f777-11e5-b30c-0800276c97db" millis="0.002" facility="runit-service" timestamp="2016-03-31T19:31:46.000Z"] GET /test1/2 HTTP/1.1

So it would be possible to write a parser for this.

Even better - it looks like it may be possible to send output from graylog using a CEF output. We have a CEF parser in Security Analytics so you wouldnt need to write your own parser.

[FredBlaise]

Hi David,

Yes, I know all this. I was just wondering if there was an alternative way that could be used, instead of the "oldies" sftp and other plain syslog.

Since you're using rabbitMQ, it seems a bit odd that we can't use it as such to input message to the VLC for example. I would even expect some Kafka for high loads.

When used to the recent tools, RSA looks like a dinosaur; I wish that could change?

Thanks,

fred

[FredBlaise]

Hi again,

Thanks, yes I know about this plugin. I was just wondering about something else, as expressed above.

However, in the "structured" mode you copy-pasted above, I don't think the out-of-the-box parser of RSA would work right? The goal is to have the decoder stuff just work, so no extra has to be added, just the raw stuff (which I obviously can get working there since I have the source code, and even the "plain" mode may work.

Thanks about your hint about CEF.

Cheers,

fred

[DavidWaugh1]

Hi I think the CEF format will be the best way to go in your case then.