2018-09-21 06:06 AM
Hi All,
We are continuously facing issues related to ESA lag. THE ESA falls behind on specific concentrators and there is delay in alerting. We have a RSA case open for this as well and we are being helped there but also thought of asking this in the forum for any further ideas.
We have reviewed ESA rules and disabled the rules taking maximum memory (which was around 60 MB). A service restart of the ESA works as a quickfix but the lag builds up again on these specific concentrators. There is no lag though between the decoder and the concetrator at any point of time.
Any pointers on what would be causing this? Thanks in advance
2018-09-21 10:35 AM
is your ESA still on SD cards? depending on when it was installed and if it has been reimaged/SD cards disabled you might want to check.
I thought H&W had that stat somewhere but i cant find it at the moment, try this from the CLI
lsblk --ascii --output NAME,KNAME,MODEL,RM,RO,SCHED,TYPE,SIZE,STATE,MOUNTPOINT
look for Internal Dual SD and the partitions that are running from it.
If the card is there and running partitions consider reimaging the appliance to get off them (dual sd cards) to improve performance.
2018-09-21 11:10 AM
By default ESA service aggregates 10,000 sessions per aggregation cycle, I found at one customer this was not enough to keep up with the session rate of some concentrators.
In Explore View of the ESA service:
Workflow -> Source -> nextgenAggregationSource
increase the value of "MaxSessions" to a point where the aggregation is maintained at near realtime. in the case of my customer, I had to increase this to 50000 and also increase the "MaxEpsExpectedPerSource" to 25000 to get it to keep up. Haven't seen any ill effects on the operation of the ESA service after the change, and alerts are not falling behind.
2018-09-24 05:16 AM
Hi Eric,
Please find the output below:
NAME KNAME MODEL RM RO SCHED TYPE SIZE STATE MOUNTPOINT
sdb sdb PERC H710P 0 0 cfq disk 7.3T running
`-sdb1 sdb1 0 0 cfq part 6.6T
|-VolGroup00-root (dm-0) dm-0 0 0 lvm 8G running /
|-VolGroup00-swap00 (dm-1) dm-1 0 0 lvm 8G running [SWAP]
|-VolGroup00-usrhome (dm-2) dm-2 0 0 lvm 4G running /home
|-VolGroup00-var (dm-3) dm-3 0 0 lvm 8G running /var
|-VolGroup00-rabmq (dm-4) dm-4 0 0 lvm 20G running /var/lib/rabbitmq
|-VolGroup00-vartmp (dm-5) dm-5 0 0 lvm 4G running /var/tmp
|-VolGroup00-varlog (dm-6) dm-6 0 0 lvm 10G running /var/log
|-VolGroup00-tmp (dm-7) dm-7 0 0 lvm 100G running /tmp
|-VolGroup00-rsaroot (dm-8) dm-8 0 0 lvm 10G running /opt/rsa
`-VolGroup00-apps (dm-9) dm-9 0 0 lvm 6.4T running /opt/rsa/database
sda sda PERC H710P 0 0 cfq disk 256M running
`-sda1 sda1 0 0 cfq part 255M /boot
Could you please check once as I could not find internal sd here.
2018-09-24 05:18 AM
Hi John,
I have made the changes. Awaiting results. For now, there is no immediate impact as we are still facing lag.
2019-03-01 09:33 AM
Hi All,
There is no significant change in the ESA lag yet. I modified the parameters as suggested.
2019-03-06 04:31 PM
provide more information about your architecture as there are many factors that can impact aggregation outside NW that have not been mentioned above.
2019-03-11 05:19 AM
2019-03-21 01:39 AM
Hi , Is you issue got resolved , we are also facing same issue. And engineer told us to increase resources on Decoder and concentration.
2019-03-21 01:49 AM
Hi Eric,
My ESA output , How to check SD in it
NAME KNAME MODEL RM RO SCHED TYPE SIZE STATE MOUNTPOINT
sda sda PERC H730P Mini 0 0 deadline disk 931G running
|-sda1 sda1 0 0 deadline part 1M
|-sda2 sda2 0 0 deadline part 256G
| |-netwitness_vg00-root dm-0 0 0 lvm 29.3G running /
| |-netwitness_vg00-swap dm-1 0 0 lvm 4G running [SWAP]
| |-netwitness_vg00-varlog dm-3 0 0 lvm 10G running /var/log
| `-netwitness_vg00-usrhome dm-4 0 0 lvm 10G running /home
`-sda3 sda3 0 0 deadline part 519M /boot
sdb sdb PERC H730P Mini 0 0 deadline disk 3.7T running
`-sdb1 sdb1 0 0 deadline part 3.3T
`-netwitness_vg00-rsaapps dm-2 0 0 lvm 3.3T running /var/netwitness