Launch on GCP - Cloudbreak

Prerequisites on GCP

Before launching Cloudbreak on GCP, you must meet the following prerequisites.

GCP Account

In order to launch Cloudbreak on GCP, you must log in to your GCP account. If you don't have an account, you can create one at https://console.cloud.google.com.

Once you log in to your GCP account, you must either create a project or use an existing project.

Service Account

In order to launch clusters on GCP via Cloudbreak, you must have a service account that Cloudbreak can use to create resources. In addition, you must also have a JSON key associated with the account.

The service account must have the following roles are enabled:

Compute Engine > Compute Image User
Compute Engine > Compute Instance Admin (v1)
Compute Engine > Compute Network Admin
Compute Engine > Compute Security Admin
Storage > Storage Admin

A user with an "Owner" role can assign roles to new and existing service accounts from IAM & admin > Service accounts, as presented in the following screenshots:

For more information on creating a service account and generating a JSON key, refer to GCP documentation.

Related links
Service account credentials (External)

SSH Key Pair

Generate a new SSH key pair or use an existing SSH key pair. You will be required to provide it when launching the VM.

Virtual Network

You must have a virtual network configured on your cloud provider.

Security Group

Ports 22 (SSH), 80 (HTTPS), and 443 (HTTPS) must be open on the security group

Region and Zone

Decide in which region and zone you would like to launch Cloudbreak. You can launch Cloudbreak and provision your clusters in all regions supported by GCP.

Clusters created via Cloudbreak can be in the same or different region as Cloudbreak; when you launch a cluster, you select the region in which to launch it.

Related links
Regions and zones (External)

Launching Cloudbreak on GCP

These steps describe how to launch Cloudbreak on GCP for production. Before launching Cloudbreak on Google Cloud, review and meet the prerequisites. Next, follow the steps below.

VM Requirements

To launch the Cloudbreak deployer and install the Cloudbreak application, you must have an existing VM.

System Requirements

Your system must meet the following requirements:

Minimum VM requirements: 16GB RAM, 40GB disk, 4 cores
Supported operating systems: RHEL, CentOS, and Oracle Linux 7 (64-bit)

You can install Cloudbreak on Mac OS X for evaluation purposes only. Mac OS X is not supported for a production deployment of Cloudbreak.

Root Access

Every command must be executed as root. In order to get root privileges execute:

sudo -i

System Updates

Ensure that your system is up-to-date by executing:

yum -y update

Reboot it if necessary.

Install iptables

Perform these steps to install and configure iptables.

Steps

Install iptables-services:

yum -y install net-tools ntp wget lsof unzip tar iptables-services
systemctl enable ntpd && systemctl start ntpd
systemctl disable firewalld && systemctl stop firewalld

Without iptables-services installed the iptables save command will not be available.

Configure permissive iptables on your machine:

iptables --flush INPUT && \
iptables --flush FORWARD && \
service iptables save

Disable SELINUX

Perform these steps to disable SELINUX.

Steps

Disable SELINUX:

setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' 
/etc/selinux/config

Run the following command to ensure that SELinux is not turned on afterwards:
```
getenforce
```
The command should return "Disabled".

Install Docker

Perform these steps to install Docker. The minimum Docker version is 1.13.1. If you are using an older image that comes with an older Docker version, upgrade Docker to 1.13.1 or newer.

Steps

Install Docker service:

yum install -y docker
systemctl start docker
systemctl enable docker

Check the Docker Logging Driver configuration:
```
docker info | grep "Logging Driver"
```
If it is set to Logging Driver: journald, you must set it to "json-file" instead. To do that:
1. Open the docker file for editing:
```
vi /etc/sysconfig/docker
```
2. Edit the following part of the file so that it looks like below (showing log-driver=json-file):
```
# Modify these options if you want to change the way the docker daemon runs
OPTIONS='--selinux-enabled --log-driver=json-file --signature-verification=false'
```
3. Restart Docker:
```
systemctl restart docker
systemctl status docker
```

Install Cloudbreak on a VM

Install Cloudbreak using the following steps.

Steps

Install the Cloudbreak deployer and unzip the platform-specific single binary to your PATH. For example:
```
yum -y install unzip tar
curl -Ls public-repo-1.hortonworks.com/HDP/cloudbreak/cloudbreak-deployer_2.4.3_$(uname)_x86_64.tgz | sudo tar -xz -C /bin cbd
cbd --version
```
Once the Cloudbreak deployer is installed, you can set up the Cloudbreak application.
Create a Cloudbreak deployment directory and navigate to it:
```
mkdir cloudbreak-deployment
cd cloudbreak-deployment
```
In the directory, create a file called Profile with the following content:
```
export UAA_DEFAULT_SECRET=MY-SECRET
export UAA_DEFAULT_USER_PW=MY-PASSWORD
export UAA_DEFAULT_USER_EMAIL=MY-EMAIL
export PUBLIC_IP=MY_VM_IP
```
For example:
```
export UAA_DEFAULT_SECRET=MySecret123
export UAA_DEFAULT_USER_PW=MySecurePassword123
export UAA_DEFAULT_USER_EMAIL=dbialek@hortonworks.com
export PUBLIC_IP=172.26.231.100
```
You will need to provide the email and password when logging in to the Cloudbreak web UI and when using the Cloudbreak CLI. The secret will be used by Cloudbreak for authentication.

You should set the CLOUDBREAK_SMTP_SENDER_USERNAME variable to the username you use to authenticate to your SMTP server. You should set the CLOUDBREAK_SMTP_SENDER_PASSWORD variable to the password you use to authenticate to your SMTP server.
Generate configurations by executing:
```
rm *.yml
cbd generate
```
The cbd start command includes the cbd generate command which applies the following steps:
- Creates the docker-compose.yml file, which describes the configuration of all the Docker containers required for the Cloudbreak deployment.
- Creates the uaa.yml file, which holds the configuration of the identity server used to authenticate users with Cloudbreak.
Start the Cloudbreak application by using the following commands:
```
cbd pull-parallel
cbd start
```
This will start the Docker containers and initialize the application. The first time you start the Cloudbreak app, the process will take longer than usual due to the download of all the necessary docker images.

If you encounter errors during cbd start, refer to Toubleshooting.
Next, check Cloudbreak application logs:
```
cbd logs cloudbreak
```
You should see a message like this in the log: Started CloudbreakApplication in 36.823 seconds. Cloudbreak normally takes less than a minute to start.

Related links
Toubleshooting

Access Cloudbreak web UI

Steps

You can log into the Cloudbreak application at https://IP_Address. For example https://34.212.141.253. You may use cbd start to obtain the login information. Alternatively, you can obtain the VM's IP address from your cloud provider console.
Confirm the security exception to proceed to the Cloudbreak web UI.

The first time you access Cloudbreak UI, Cloudbreak will automatically generate a self-signed certificate, due to which your browser will warn you about an untrusted connection and will ask you to confirm a security exception.

Browser Steps

Firefox Click Advanced > Click Add Exception... > Click Confirm Security Exception

Safari Click Continue

Chrome Click Advanced > Click Proceed...
The login page is displayed:
Log in to the Cloudbreak web UI using the credentials that you configured in your Profile file:
- The username is the UAA_DEFAULT_USER_EMAIL
- The password is the UAA_DEFAULT_USER_PW
Upon a successful login, you are redirected to the dashboard:

Browser	Steps
Firefox	Click Advanced > Click Add Exception... > Click Confirm Security Exception
Safari	Click Continue
Chrome	Click Advanced > Click Proceed...

Configure external Cloudbreak database

By default, Cloudbreak uses an embedded PostgreSQL database to persist data related to Cloudbreak configuration, setup, and so on. For a production Cloudbreak deployment, you must configure an external database.

Related links
Configure an external database

Create Cloudbreak Credential

Cloudbreak works by connecting your GCP account through this credential, and then uses it to create resources on your behalf. Before you can start provisioning cluster using Cloudbreak, you must create a Cloudbreak credential.

Prerequisites

In order to launch clusters on GCP via Cloudbreak, you must have a service account that Cloudbreak can use to create resources. If you do not have one, refer to Prerequisites: Service account.

Steps

In the Cloudbreak web UI, select Credentials from the navigation pane.
Click Create Credential.
Under Cloud provider, select "Google Cloud Platform":

Provide the following information:

Parameter	Description
Key type	Select JSON or P12. Since activating service accounts with P12 private keys has been deprecated in the Cloud SDK, we recommend using JSON.
Name	Enter a name for your credential.
Description	(Optional) Enter a description.
Project Id	(Only required for P12 key type) Enter the project ID. You can obtain it from your GCP account by clicking on the name of your project at the top of the page and copying the ID.
Service Account Email Address	(Only required for P12 key type) "Service account ID" value for your service account created in prerequisites. You can find it on GCP at IAM & Admin > Service accounts.
Service Account Private Key	Upload the key that you created in the prerequisites when creating a service account.

Click Create.
Your credential should now be displayed in the Credentials pane.

Congratulations! You have successfully launched Cloudbreak and created a Cloudbreak credential. Now you can use Cloudbreak to create clusters.

Next: Create a Cluster