Streaming Analytics cluster layout

Describes the layout and capacity of the Streaming Analytics Light Duty cluster definitions.

Streaming Analytics: Light Duty cluster layout

You can use a Streaming Analytics: Light Duty cluster definition in development and testing scenarios. The Light Duty cluster definition can also be used in production for stateless Flink jobs or for Flink jobs with minimal state. The Light Duty cluster has the following specifications:
  • Flink, HDFS, YARN, and Zookeeper are co-located on all instances
  • For each node hosting Flink, HDFS, YARN, and Zookeeper
    • AWS: m5.2xlarge
    • Azure: Standard_D8_v3

For more information, see your cloud provider-specific information about instance types and storage information.

Streaming Analytics: Heavy Duty cluster layout

You can use a Streaming Analytics: Heavy Duty cluster definition in production for Flink jobs with large state with RockDB as state backend. The Heavy Duty cluster has the following specifications:
  • Flink, HDFS, YARN, and Zookeeper are co-located on all instances
  • For each node hosting Flink, HDFS, YARN, and Zookeeper
    • AWS: m5.2xlarge
    • Azure: Standard_D8_v3
  • For worker nodes:
    • Storage type: SSD
    • Volume size: 1000 GB

For more information, see your cloud provider-specific information about instance types and storage information.