Adjusting Heartbeat TCP Timeout Interval

Using the TCP flag, you can prevent the Statestore from waiting indefinitely for a response from the subscribers that fail to respond to the heartbeat RPC within the set period.

This flag statestore_heartbeat_tcp_timeout_seconds defines the time that may elapse before a heartbeat RPC connection request from a Statestore server to an Impalad or a Catalog server (subscribers) should be considered dead.

You can increase the flag value if you see intermittent heartbeat RPC timeouts listed in the statestore's log. You may find the max value of "statestore.priority-topic-update-durations" metric on the statestore to get an idea of a reasonable value to be used in this flag.

  1. In Cloudera Manager, navigate to Impala service > Configuration.
  2. In the Impala Daemon Command Line Argument Advanced Configuration Snippet (Safety Valve) field, add the flag statestore_heartbeat_tcp_timeout_seconds with an appropriate value.
  3. You can also control the maximum number of consecutive heartbeat messages an impalad can miss before being declared failed by the statestore by adding this flag statestore_max_missed_heartbeats. Typically, you will not have to change this value.
  4. Click Save Changes and restart Impala.
  5. Log in to the CDP web interface and navigate to the Data Warehouse service.
  6. In the Data Warehouse service, click Virtual Warehouses in the left navigation panel.
  7. Select the Impala Virtual Warehouse, click options for the warehouse you want to adjust the Heartbeat TCP Timeout.
  8. Click Edit and navigate to Impala Statestoredunder Configurationstab.
  9. Using the + sign, specify the following if the options are not already added.
  10. In the Add Customs Configuration field, add the flag statestore_heartbeat_tcp_timeout_seconds with an appropriate value.
  11. You can also control the maximum number of consecutive heartbeat messages an impalad can miss before being declared failed by the statestore by adding this flag statestore_max_missed_heartbeats. Typically, you will not have to change this value.
  12. Click Apply and restart Impala.