Cloudera Docs

Kernel and OS tuning

The Cloudera Runtime platform depends on a tuned underlying host operating system (OS) for optimal performance. Cloudera suggests setting the vm.swappiness and transparent hugepage compaction kernel parameters.

For additional background information and suggested settings, see Optimizing Performance in Cloudera Runtime.

Entropy

Cryptographic operations require entropy to ensure randomness. The Cloudera Security guide explains how to check available entropy and how to ensure sufficient entropy is available: Entropy Requirements.

Networking parameters

The Transmit and Receive ring buffer sizes on the ethernet interfaces of every node of the cluster should be adjusted to ensure higher throughput rates. Check existing ring buffer sizes by running the following command:
ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX:		4096
RX Mini:	0
RX Jumbo:	0
TX:		4096
Current hardware settings:
RX:		256
RX Mini:	0
RX Jumbo:	0
TX:		256
After checking the preset maximum values and the current hardware settings, run the following commands to resize the ring buffers:
ethtool –G [***INTERFACE***] rx [***NEWSIZE***]
or 
ethtool –G [***INTERFACE***] tx [***NEWSIZE***]

Most modern enterprise-grade network adapters have several performance optimization features, such as offload capabilities and large segmentation offload, which reduce load on host CPU by handling these functions at the network interface level which can be explored as part of performance optimization initiatives. An iterative approach is recommended while applying a standard load generator such as the Terasuite benchmarks, to test the effect of enabling said features. Performance optimization parameters should not be applied indiscriminately, without thorough testing, and should only be applied based on genuine need.

The following parameters should be added to /etc/sysctl.conf to optimize various network behaviors:

  • Disable TCP timestamps to improve CPU utilization (this is an optional parameter and depends on your NIC vendor):
    net.ipv4.tcp_timestamps=0
  • Enable TCP sacks to improve throughput:
    net.ipv4.tcp_sack=1
  • Increase the maximum length of processor input queues:
    net.core.netdev_max_backlog=250000
  • Increase the TCP max and default buffer sizes using setsockopt():
    net.core.rmem_max=4194304
net.core.wmem_max=4194304
net.core.rmem_default=4194304
net.core.wmem_default=4194304
net.core.optmem_max=4194304
  • Increase memory thresholds to prevent packet dropping:
    net.ipv4.tcp_rmem=4096 87380 4194304
net.ipv4.tcp_wmem=4096 65536 4194304