2018-12-07 06:58 PM
In our main data centre, we have a decoder each for logs and packets, and also a concentrator each for logs and packets.
I'm curious if I should keep this configuration or go for a single concentrator for both logs and packets.
What would be the pros and cons of each config.
One issue I'm finding with separate concentrators is correlating the same events between logs and packets, which made me think that if there was a single concentrator for both I'd be able to do it.
Thanks.
2018-12-11 12:36 PM
Jeremy,
The general rule that RSA goes by is that each decoder should have its own concentrator. This is to provide the customer with the best possible performance. However I will attempt to provide the pros and cons of what you are looking for.
Below is the pros and cons of using a single concentrator for a log decoder and packet decoder.
Pros:
- Less maintenance as you only require one concentrator server.
- Only require one concentrator license
- Only need to adjust one XML indexing file if adding additional meta or changing meta indexing.
- Single location for an ESA to connect to for retrieving data from a log and packet decoder.
- Only needing to maintain storage for one Concentrator.
Cons:
- Potential to cause high latency during Investigations, report generation, and ESA calls due to a single concentrator being busy consuming and indexing from two decoders. (based on decoder Gb/s and EPS)
- If the single concentrator goes offline for any reason you loose access to both the packet decoder and log decoder meta data/raw data until it comes back online.
- Limited ability to increase the existing load on the decoders. Appliance log decoders are rated at 30,000 EPS and packet decoders are rated at 10 GB/s. If you use a single concentrator you will have serious performance issues at these max rates for the decoder. If you use a single concentrator my guess would be that you could be restricted to about under 500 Mb/s for the packet decoder and under 1000 EPS for the log decoder. These are just guesses as it would depend on the amount of indexing required for your environment. See first point above.
- Depending on the rate of ingestion on the two decoders and the current amount of storage on the concentrator you could find that concentrator retention drops significantly. Lets say you have currently 3 DACs on the concentrators and retention is about 6 months. If you move both decoders to a single concentrator with the same 3 DACs on it you will reduce retention by at least half. If you then add the 3 DACs from the other concentrator you may be able to get back to the amount of retention as before but now you have an extra concentrator with no DACs.
One of your main issues with the two concentrators is stitching the packet and log data together. Pulling them into the same concentrator will not address that issue. The best way to do this is to use information that would be common to logs and packets, such as source/destination IP addresses, hostnames, and a few others. If you have known items such as you are watching for suspicious activity on known hosts you can create application rules and/or feeds to help better merge the data from the packets and logs by creating additional context (new meta) to Investigate on.
i hope the above helps with your decision making process.
2018-12-07 07:16 PM
You can maintain your current setup then use a Broker service that aggregates from both Concentrators to achieve your cross-correlation / queries.
Sent from my iPhone
2018-12-11 12:36 PM
Jeremy,
The general rule that RSA goes by is that each decoder should have its own concentrator. This is to provide the customer with the best possible performance. However I will attempt to provide the pros and cons of what you are looking for.
Below is the pros and cons of using a single concentrator for a log decoder and packet decoder.
Pros:
- Less maintenance as you only require one concentrator server.
- Only require one concentrator license
- Only need to adjust one XML indexing file if adding additional meta or changing meta indexing.
- Single location for an ESA to connect to for retrieving data from a log and packet decoder.
- Only needing to maintain storage for one Concentrator.
Cons:
- Potential to cause high latency during Investigations, report generation, and ESA calls due to a single concentrator being busy consuming and indexing from two decoders. (based on decoder Gb/s and EPS)
- If the single concentrator goes offline for any reason you loose access to both the packet decoder and log decoder meta data/raw data until it comes back online.
- Limited ability to increase the existing load on the decoders. Appliance log decoders are rated at 30,000 EPS and packet decoders are rated at 10 GB/s. If you use a single concentrator you will have serious performance issues at these max rates for the decoder. If you use a single concentrator my guess would be that you could be restricted to about under 500 Mb/s for the packet decoder and under 1000 EPS for the log decoder. These are just guesses as it would depend on the amount of indexing required for your environment. See first point above.
- Depending on the rate of ingestion on the two decoders and the current amount of storage on the concentrator you could find that concentrator retention drops significantly. Lets say you have currently 3 DACs on the concentrators and retention is about 6 months. If you move both decoders to a single concentrator with the same 3 DACs on it you will reduce retention by at least half. If you then add the 3 DACs from the other concentrator you may be able to get back to the amount of retention as before but now you have an extra concentrator with no DACs.
One of your main issues with the two concentrators is stitching the packet and log data together. Pulling them into the same concentrator will not address that issue. The best way to do this is to use information that would be common to logs and packets, such as source/destination IP addresses, hostnames, and a few others. If you have known items such as you are watching for suspicious activity on known hosts you can create application rules and/or feeds to help better merge the data from the packets and logs by creating additional context (new meta) to Investigate on.
i hope the above helps with your decision making process.
2018-12-11 06:55 PM
Hi John,
Thanks for your detailed replied, that helps me make the decision to leave our current configuration of a concentrator per decoder type the way it is.
Cheers,
J.