Creating your cluster

Once you have met the prerequisites, you are ready to create a managed and secured Streams Messaging cluster in CDP Public Cloud by using one of the prescriptive streams messaging cluster definitions. For this, you must select the cluster definition that matches your cloud provider for the environment and further choose from the light and heavy duty options available.

  1. Log into the CDP web interface.
  2. Navigate to Management Console > Environments, and select the environment where you would like to create a cluster.
  3. Click Create Data Hub.
    The following page is displayed:
  4. Select Cluster Definition.
  5. Select the appropriate Streams Messaging cluster definition from the Cluster Definition dropdown depending on your operational objectives.

    You have the following options:

    • Streams Messaging Heavy Duty for AWS
    • Streams Messaging Light Duty for AWS
    • Streams Messaging High Availability for AWS
    • Streams Messaging Heavy Duty for Azure
    • Streams Messaging Light Duty for Azure
    • Streams Messaging High Availability for Azure (Technical Preview)
    • Streams Messaging Heavy Duty for GCP
    • Streams Messaging Light Duty for GCP
    • Streams Messaging High Availability for GCP (Technical Preview)

    For more information on templates, see Planning your Streams Messaging deployment.

    The list of services is automatically shown below the selected cluster definition name.

  6. Give the cluster a name and add any tags you might need.
    You can define tags that will be applied to your cluster-related resources on your cloud provider account. For more information about tags, see Tags.
  7. If using the Streams Messaging High Availability definitions, ensure that you select multiple subnets.
    Multiple subnets can be selected by navigating to Advanced Options > Network And Availability.
  8. Optional: Add SRM or Kafka Connect to Heavy Duty clusters.
    SRM and Kafka Connect are not provisioned by default when using the Streams Messaging Heavy Duty definition. If you are using this definition and want to have SRM, Kafka Connect, or both deployed in the cluster, set the Instance Count of the Srm Nodes or Connect Nodes host group to at least one.
    1. Go to Advanced Options > Hardware and Storage.
    2. Locate the appropriate host group and click to edit host group details.
    3. Set the Instance Count to at least 1.
    4. Click to save your changes.
  9. Optional: Use the Advanced Options section to customize the infrastructure settings.
  10. Click Provision Cluster.
You will be redirected to the Data Hub cluster dashboard, and a new tile representing your cluster will appear at the top of the page.
What steps you take next depends on whether you have customized Attached Volume per Instances .
  • If you configured Attached Volume per Instances and the volume count is not identical for the Broker and Core_brokers host groups, continue with Configure data directories for clusters with custom disk configurations.
  • If you did not customize Attached Volume per Instances or customized it in a way that the volume count remained identical for the Brokers and Core_brokers host groups, continue with Give users access to your cluster.