Throttle Remote Collector to Local Collector Bandwidth

To improve performance, you can throttle the bandwidth to control the rate that the Remote Collector sends event data to Local Collector or between Message Brokers. To do this, you configure the Linux kernel's filtering and IPTable functionality.

This works for both push and pull Remote Collector configurations. The set-shovel-transfer-limit.sh shell script located on the /opt/netwitness/bin automates the configuration of the kernel filter and iptables related to this port.

This topic describes how to throttle Remote Collector to Local Collector bandwidth using the set-shovel-transfer-limit.sh shell script. It contains the following sections:

  • The set-shovel-transfer-limit.sh shell script command line help.

    Note: The filter value that you need to set depends on the rate at which remote log collector is sending events to the Local Collector.

  • An example that sets the Filter to 4096 kilobits per second.

Command Line Help for Set Shovel Transfer Limit Script

Issue the –h command to display help for set-shovel-transfer-limit.sh shell script.

cd /opt/netwitness/bin
./set-shovel-transfer-limit.sh

Usage:

set-shovel-transfer-limit.sh -s|-c|-d|[-i interface] [-r rate]

where:

  • -c = clear existing
  • -d = display filter
  • -s = set new values
  • -i = interface is the name of the network interface. Default value is eth0
  • -r = rate is the bandwidth rate. Default value is 256kbps

Bandwidths and rates can be specified in:

  • nolimit: disables throttling
  • kbit: Kilobits per second
  • mbit: Megabits per second
  • kbps: Kilobytes per second
  • mbps: Megabytes per second
  • bps: Bytes per second

Set the Filter to 4096 Kilobits per Second

This example sets the Filter to 4096 kilobits per second.

[root@<hostname> bin]#./set-shovel-transfer-limit.sh -s -r 4096kbit

RATE=4096kbit
PORTNUMBER=5671
DEVICE_INTERACE=eth0

iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ]

Current/new values...

iptables -t mangle -n -v -L
Chain PREROUTING (policy ACCEPT 2 packets, 161 bytes)
pkts bytes target prot opt in out source destination

Chain INPUT (policy ACCEPT 2 packets, 161 bytes)
pkts bytes target prot opt in out source destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 2 packets, 248 bytes)
pkts bytes target prot opt in out source destination
0 0 MARK tcp -- * eth0 0.0.0.0/0 0.0.0.0/0 multiport dports 5671 MARK set 0xa
0 0 MARK tcp -- * eth0 0.0.0.0/0 0.0.0.0/0 multiport sports 5671 MARK set 0xa

Chain POSTROUTING (policy ACCEPT 2 packets, 248 bytes)
pkts bytes target prot opt in out source destination

tc -s -d class show dev eth0
class htb 1:1 root rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 7
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 20000 ctokens: 20000

class htb 1:2 parent 1:1 prio 0 quantum 51200 rate 4096Kbit ceil 4096Kbit burst 1599b/8 mpu 0b overhead 0b cburst 1599b/8 mpu 0b overhead 0b level 0
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 48828 ctokens: 48828