Networking parameters

An overview on Linux kernel parameters related to networking to fine tune for better Kafka performance.

Kafka is designed to handle a huge amount of network traffic. By default, the Linux kernel is not tuned for this scenario. The following kernel settings may need to be tuned based on use case or specific Kafka workload:

• net.core.wmem_default: Default send socket buffer size.
• net.core.rmem_default: Default receive socket buffer size.
• net.core.wmem_max: Maximum send socket buffer size.
• net.core.rmem_max: Maximum receive socket buffer size.
• net.ipv4.tcp_wmem: Memory reserved for TCP send buffers.
• net.ipv4.tcp_rmem: Memory reserved for TCP receive buffers.
• net.ipv4.tcp_window_scaling: TCP Window Scaling option.
• net.ipv4.tcp_max_syn_backlog: Maximum number of outstanding TCP SYN requests (connection requests).
• net.core.netdev_max_backlog: Maximum number of queued packets on the kernel input side (useful to deal with spike of network requests).

Use the Cloudera Enterprise Reference Architecture for Bare Metal Deploymentsas a guideline when specifying values for the parameters.