Cloudera Docs

Creating your cluster

Once you have met the prerequisites, you are ready to create a managed and secured Streams Messaging cluster in Cloudera on cloud by using one of the prescriptive Streams Messaging cluster definitions.

  1. Log into the Cloudera web interface.
  2. Navigate to Management Console > Environments, and select the environment where you would like to create a cluster.
  3. Click Create Data Hub.
    The following page is displayed:
  4. Select Cluster Definition.
  5. Select the appropriate Streams Messaging cluster definition from the Cluster Definition dropdown depending on your operational objectives.
    The list of services is automatically shown below the selected cluster definition name. For more information on templates, see Cloudera Data Hub cluster definitions and Streams Messaging cluster layout.
  6. Give the cluster a name and add any tags you might need.
    You can define tags that will be applied to your cluster-related resources on your cloud provider account. For more information about tags, see Tags.
  7. If using the Streams Messaging High Availability definitions, ensure that you select multiple subnets.
    Multiple subnets can be selected by navigating to Advanced Options > Network And Availability.
  8. Optional: Add Kafka KRaft to your cluster.

    KRaft nodes are not provisioned by default in any of the Streams Messaging cluster definitions. To run Kafka in KRaft mode, set the Instance Count of the Kraft Nodes host group to an odd number (Cloudera recommends a minimum of three nodes). Cluster provisioning fails if you specify an even number. KRaft nodes cannot be scaled after cluster provisioning. ZooKeeper service instances are still deployed in KRaft mode and can be optionally deleted after the cluster is deployed.

    1. Go to Advanced Options > Hardware and Storage.
    2. Locate the Kraft Nodes host group and click to edit host group details.
    3. Set the Instance Count to at an odd number
    4. Click to save your changes.
  9. Optional: Add Streams Replication Manager or Kafka Connect to Heavy Duty clusters.
    Streams Replication Manager and Kafka Connect nodes are not provisioned by default when using the Streams Messaging Heavy Duty definition. If you are using this definition and want to have Streams Replication Manager, Kafka Connect, or both deployed in the cluster, set the Instance Count of the Srm Nodes or Connect Nodes host group to at least one.
    1. Go to Advanced Options > Hardware and Storage.
    2. Locate the appropriate host group and click to edit host group details.
    3. Set the Instance Count to at least 1.
    4. Click to save your changes.
  10. Optional: Use the Advanced Options section to customize the infrastructure settings.
  11. Click Provision Cluster.
You will be redirected to the Cloudera Data Hub cluster dashboard, and a new tile representing your cluster will appear at the top of the page.
What steps you take next depends on how you configured your cluster
  • If you have provisioned KRaft nodes in your cluster, you can optionally choose to remove ZooKeeper from your cluster. For more information, see Deleting ZooKeeper from Streams Messaging clusters.
  • If you configured Attached Volume per Instances and the volume count is not identical for the Broker and Core_brokers host groups, continue with Configuring data directories for clusters with custom disk configurations.
  • If you did not customize Attached Volume per Instances or do not want to remove ZooKeeper, continue with Give users access to your cluster.