Cloudbreak utilizes cloud provider security resources such as virtual networks, security groups, and identity and access management:
Network isolation is achieved via user-configured virtual networks and subnets.
Network security is achieved via out-of-the-box security group settings.
Controlled use of cloud resources using IAM roles (AWS, GCP) or Active Directory (in case of Azure).
Cloud providers use virtual networks which resemble traditional networks. Depending on the options that you selected during deployment, your Cloudbreak instance and clusters are launched into new or existing cloud provider networking infrastructure (virtual networks and subnets). For more information about virtual networks, refer to the cloud-provider documentation:
|Cloud provider||External documentation link|
|AWS||Amazon Virtual Private Cloud (Amazon VPC)|
|Azure||Microsoft Azure Virtual Network|
|Google Cloud Platform||Virtual Private Cloud (VPC) network|
Security groups are set up to control network traffic to the instances in the system.
Cloudbreak uses public IP addresses when communicating with cluster nodes. On AWS, you can configure it to use private IPs instead. For instructions, refer to Configure communication via private IPs on AWS.
Configure communication via private IPs on AWS
Cloudbreak instance security group
The following table lists the minimum security group port configuration required for the Cloudbreak instance:
|22||SSH access to the Cloudbreak VM.|
|80||HTTP access to the Cloudbreak UI. This is automatically redirected to the HTTPS (443) port.|
|443||HTTPS access to the Cloudbreak UI.|
Default cluster security groups
The following section describes the network requirements and options. By default, when creating a cluster, a new network, subnet, and security groups will be created automatically.
By default, when creating a cluster, a new network, subnet, and security groups are created automatically. The default experience of creating network resources such as network, subnet and security group automatically is provided for convenience. We strongly recommend that you review these options and for production cluster deployments leverage your existing network resources that you have defined and validated to meet your enterprise requirements.
Depending on the cluster components that you are planning to use, you may need to open additional ports required by these components.
|*||All cluster hosts||TCP||22||
In addition to the ports described above, Cloudbreak uses certain ports for internal communication within the subnet. By default, Cloudbreak opens ports 0-65535 to the subnet's internal CIDR (such as 10.0.0.0/16). Use the following table to limit this CIDR:
|Salt-bootstrap||Gateway instance (Ambari server instance)||TCP||7070||Salt-bootstrap service launches and configures Saltstack.|
|Salt-master||All hosts in the cluster||TCP||4505,4506||Salt-minions connect to the Salt-master(s).|
|Consul server||All hosts in the cluster||TCP,UDP||8300,8301||Consul agents connect to the Consul server.|
|Consul agent (all hosts in the cluster)||All hosts in the cluster||TCP,UDP||8300,8301||Consul agents connect to other Consul agents (Gossip protocol).|
|Prometheus node exporter||Gateway instance (Ambari server instance)||TCP||9100||Prometheus server scrapes metrics from the node exporters.|
|Ambari server||All hosts in the cluster||Refer to Ambari documentation.||Refer to Ambari documentation.||Ambari agents connect to the Ambari server.|
When creating data lakes and their attached clusters,, you must also open the following internal port:
|Data lake cluster||Clusters attached to the data lake||TCP||6080||Used for communication between the data lake and attached clusters.|
To securely control access to cloud resources, cloud providers use identity management services such as IAM roles (AWS and GCP) and Active Directory (Azure).
|Cloud provider||External documentation link|
|AWS||AWS Identity and Access Management (IAM)|
|Azure||Azure Active Directory ((Azure AD))|
|Google Cloud Identity and Access Management (IAM)|
Cloudbreak utilizes cloud provider’s identity management services via Cloudbreak credential. After launching Cloudbreak on your chosen cloud provider, you must create a Cloudbreak credential, which allows Cloudbreak to authenticate with your cloud provider identity management service. Only after you have completed this step, Cloudbreak can create resources on your behalf.
Authentication with AWS
When launching Cloudbreak on AWS, you must select a way for Cloudbreak to authenticate with your AWS account and create resources on your behalf. While key-based authentication uses your AWS access key and secret key, role-based authentication uses IAM roles.
If you are using role-based authentication for Cloudbreak on AWS, you must create two IAM roles: one to grant Cloudbreak access to allow Cloudbreak to assume AWS roles (using the "AssumeRole" policy) and the second one to provide Cloudbreak with the capabilities required for cluster creation (using the "cb-policy" policy).
The following table provides contextual information about the two roles required:
|Role||Purpose||Overview of steps||Configuration|
|CloudbreakRole||Allows Cloudbreak to assume other IAM roles - specifically the CredentialRole.||Create a role called "CloudbreakRole" and attach the "AssumeRole" policy. The "AssumeRole" policy definition and steps for creating the CloudbreakRole are provided below.||
When launching Cloudbreak, you will attach the "CloudbreakRole" IAM role to the VM.
If you are using hosted Cloudbreak, you do not need to perform this step.
|CredentialRole||Allows Cloudbreak to create AWS resources required for clusters.||
Create a new IAM role called "CredentialRole" and attach the "cb-policy" policy to it. The "cb-policy" policy definition and steps for creating the CredentialRole are provided below.
When creating this role using the AWS Console, make sure that that it is a role for cross-account access and that the trust-relation is set up as follows: 'Account ID' is your own 12-digit AWS account ID and 'External ID' is “provision-ambari”. See steps below.
|Once you log in to the Cloudbreak UI and are ready to create clusters, you will use this role to create the Cloudbreak credential.|
Authentication with Azure
After launching Cloudbreak on Azure, you are required to create a Cloudbreak credential, which allows Cloudbreak to authenticate with your Azure Active Directory.
You have two options:
Interactive: The app and service principal creation and role assignment are fully automated, so the only input that you need to provide to Cloudbreak is your Subscription ID and Directory ID.
App-based: The app and service principal creation and role assignment are not automated You must create an Azure Active Directory application registration and then provide its parameters to Cloudbreak, in addition to providing your Subscription ID and Directory ID.
Create Cloudbreak credential
Authentication with GCP
After launching Cloudbreak on GCP, you are required to register a service account in Cloudbreak via creating a Cloudbreak credential. Cloudbreak uses this account to authenticate with the GCP identity management service.
Authentication with OpenStack
After launching Cloudbreak on OpenStack, you are required to create a Cloudbreak credential, which allows Cloudbreak to authenticate with keystone.
Create Cloudbreak credential