Using a New AWS Region in Cloudera Director
For more information about AWS regions, see Regions and Availability Zones in the AWS documentation.
Entering the Region Code
When using its web interface, Cloudera Director asks you which region to use when you define a new AWS environment. You can select the region for EC2, where instances hosting Cloudera Manager and cluster components run, and for RDS, where an external database server can house databases for Cloudera Manager and services like Hive and Oozie.
The region selection widgets are ordinary drop-down menus, but the menus are also editable. To use a region that isn't listed, just type in its region code.
- EC2 region: in the provider section, as the region field
- RDS region: in the provider section, as the rdsRegion field. If the region is not specified, it defaults to the EC2 region
In most cases, Cloudera Director can figure out the AWS endpoints for the different services in a region, so just naming the new region is enough to get things moving. If you receive errors that an AWS service could not be reached, you may need to specify some endpoints, as described below for RDS, IAM, and KMS.
For general information about region endpoints in AWS, see AWS Regions and Endpoints in the AWS documentation.
Using the web UI interface, specify the endpoint URL directly when you define your environment. In the web interface, expand the Advanced Options section under RDS (Relational Database Service) and enter the endpoint URL for RDS region endpoint. In a configuration file, give the URL as the value for the rdsRegionEndpoint field in the provider section. Here is what an endpoint URL looks like:
- Rather than specifying the RDS endpoint URL with each environment you create, you can supply it in a configuration file that is read by Cloudera Director's AWS plugin, so it will be
used for all environments created with that instance of Cloudera Director. The configuration file is named rds.endpoints.properties and, by default, resides in the
directory /var/lib/cloudera-director-plugins/aws-provider-version/etc/. The version number for the aws-provider
part of the path changes with most Cloudera Director releases, as the plugin changes version. For example, aws-provider-1.4.1 matches with Cloudera Director 2.4. So the
path and file name with Cloudera Director 2.4 would be as follows:
/var/lib/cloudera-director-plugins/aws-provider-1.4.1/etc/rds.endpoints.propertiesCloudera Director ships with an example of the file that you can use as a template: rds.endpoints.properties.example. Copy this file to a new rds.endpoints.properties file in that directory, and add a line for the RDS endpoint URL, for example:
After adding a new endpoint, restart Cloudera Director if it is running.
A new AWS region usually does not support the full range of services and features that are available in older, established regions. It's important to understand what services and features your chosen region lacks, so that you do not request them through Cloudera Director. Cloudera Director does not retain knowledge on which regions have which services available.
- AMIs - common "stock" AMIs may not exist for new regions
- instance types - deprecated instance types are often left out of new regions
- dedicated instances (tenancy)
- Spot blocks
- RDS instance encryption