Streaming Analytics cluster layout
The Light and Heavy Duty layouts differ from each other based on the volume of storage. You need to choose between the two options based on your operational objectives and application specifications.
Streaming Analytics: Light Duty cluster layout
You can use a Streaming Analytics: Light Duty cluster definition in development and testing
scenarios. The Light Duty cluster definition can also be used in production for stateless
Flink jobs or for Flink jobs with minimal state. The Light Duty cluster has the following
specifications:
- Flink, HDFS, YARN, and Zookeeper are co-located on all instances
- For each node hosting Flink, HDFS, YARN, and Zookeeper
- AWS: m5.2xlarge
- Azure: Standard_D8_v3
For more information, see your cloud provider-specific information about instance types and storage information.
Streaming Analytics: Heavy Duty cluster layout
You can use a Streaming Analytics: Heavy Duty cluster definition in production for Flink
jobs with large state with RockDB as state backend. The Heavy Duty cluster has the following
specifications:
- Flink, HDFS, YARN, and Zookeeper are co-located on all instances
- For each node hosting Flink, HDFS, YARN, and Zookeeper
- AWS: m5.2xlarge
- Azure: Standard_D8_v3
- For worker nodes:
- Storage type: SSD
- Volume size: 1000 GB
For more information, see your cloud provider-specific information about instance types and storage information.
