Deploy CDP on AWS using Terraform
This guide demonstrates how to deploy CDP on AWS by using one of the CDP deployment templates.
The templates use Terraform, an open source Infrastructure as Code (IaC) software tool for defining and managing cloud or data center infrastructure. You interface the templates via a simple configuration file residing in a GitHub repository.
Prerequisites
Prior to deploying CDP, you should make sure that your AWS account meets the basic requirements and you need to install a few prerequisites.
Next, you can follow the instructions below for deploying CDP.
Deploy CDP
Setting up a CDP deployment involves cloning a GitHub repository, editing the configuration, and running Terraform commands.
Step 1: Clone the repository
The cdp-tf-quickstarts repository contains Terraform resource files to quickly deploy CDP Public Cloud and associated prerequisite cloud provider resources via CDP Terraform modules.
git clone https://github.com/cloudera-labs/cdp-tf-quickstarts.git
cd cdp-tf-quickstarts
Step 2: Edit the configuration file for the required cloud provider
In the cloned repository, change to the required cloud provider directory. Currently only AWS is available.
cd aws
vi terraform.tfvars
# ------- Global settings -------
env_prefix = "<ENTER_VALUE>" # Required name prefix for cloud and CDP resources, e.g. cldr1
aws_region = "<ENTER_VALUE>" # Change this to specify Cloud Provider region, e.g. eu-west-1
aws_key_pair= "<ENTER_VALUE>" # Change this with the name of a pre-existing AWS keypair (Example: my-keypair)
deployment_template = "<ENTER_VALUE>" # Specify the deployment pattern below. Options are public, semi-private or private
# ------- Network Resources -------
# **NOTE: If required change the values below any additional CIDRs to add the the AWS Security Groups**
ingress_extra_cidrs_and_ports = {
cidrs = ["<ENTER_IP_VALUE>/32", "<ENTER_IP_VALUE>/32"],
ports = [443, 22]
}
# ------- Global settings -------
env_prefix = "test-env" # Required name prefix for cloud and CDP resources, e.g. cldr1
aws_region = "eu-west-1" # Change this to specify Cloud Provider region, e.g. eu-west-1
aws_key_pair= "test-public-key" # Change this with the name of a pre-existing AWS keypair (Example: my-keypair)
deployment_template = "public" # Specify the deployment pattern below. Options are public, semi-private or private
# ------- Network Resources -------
# **NOTE: If required change the values below any additional CIDRs to add the the AWS Security Groups**
ingress_extra_cidrs_and_ports = {
cidrs = ["137.171.0/32"],
ports = [443, 22]
}
The following tables explain the mandatory and optional inputs that need to be provided in the configuration file. While the mandatory inputs are present in the configuration file and only their values need to be provided, the optional inputs should be added to the configuration file manually.
Table 1: Mandatory inputs
Input | Description | Default value |
aws_region |
The AWS region in which the cloud prerequisites and CDP will be deployed. For example, eu-west-1. For a list of supported AWS regions, see Supported AWS regions. | Not set |
env_prefix |
A string prefix that will be used to name the cloud provider and CDP resources created. | Not set |
aws_key_pair |
The name of an AWS keypair that exists in your account in the selected region. | Not set |
deployment_template |
The selected deployment pattern. Values allowed:
|
public |
ingress_extra_cidrs_and_ports |
Inbound access to the UI and API endpoints of your deployment will be allowed
from the Enter your machine’s public IP here, with ports 443 and 22. If unsure, you can check your public IP address here. |
CIDRs are not set. Ports are set to 443, 22 by default. |
The following table explains the optional inputs. The optional inputs can optionally be added to the configuration file.
Table 2: Optional inputs
Input | Description | Default value |
aws_profile |
AWS configuration profile | default |
cdp_profile |
CDP configuration profile | default |
deploy_cdp |
Can be true or false . If set to true, the
CDP stack is deployed on the cloud prerequisites that have been created. If set to false,
then only the AWS prerequisite resources are created and no CDP stack is deployed. |
true |
Step 3: Launch the deployment
terraform init
terraform apply
Terraform will show a plan with the list of cloud provider and CDP resources that will be created.
When you are prompted, type yes
to tell Terraform to perform the
deployment. Typically, this will take about 60 minutes. Once the deployment is complete, CDP
will print output similar to the following:
Apply complete! Resources: 48 added, 0 changed, 0 destroyed.
You can navigate to the CDP web interface at https://cdp.cloudera.com/ and see your deployment progressing. Once the deployment completes, you can create Data Hubs and data services.
Clean up the CDP environment and infrastructure
If you no longer need the infrastructure that’s provisioned by Terraform, run the following command to remove the deployment infrastructure and terminate all resources:
terraform destroy