Using a New AWS Region in Altus Director

Altus Director's AWS support, embodied in a plugin, ships with a predefined, known set of AWS regions. Cloudera adds support for additional regions when possible in new Altus Director releases. But, because you might want to use a new region before it has been added in a new release, Altus Director makes it possible to add the new region yourself.

For more information about AWS regions, see Regions and Availability Zones in the AWS documentation.

Entering the Region Code

When using its web interface, Altus Director asks you which region to use when you define a new AWS environment. You can select the region for EC2, where instances hosting Cloudera Manager and cluster components run, and for RDS, where an external database server can house databases for Cloudera Manager and services like Hive and Oozie.

The region selection widgets are ordinary drop-down menus, but the menus are also editable. To use a region that isn't listed, just type in its region code.

When you use Altus Director's configuration file support for defining new deployments and clusters, you don't have any widgets. Simply supply the region code for EC2 and RDS in the expected locations.
  • EC2 region: in the provider section, as the region field
  • RDS region: in the provider section, as the rdsRegion field. If the region is not specified, it defaults to the EC2 region

Region Endpoints

In most cases, Altus Director can figure out the AWS endpoints for the different services in a region, so just naming the new region is enough to get things moving. If you receive errors that an AWS service could not be reached, you might need to specify some endpoints, as described below for RDS, IAM, and KMS.

For general information about region endpoints in AWS, see AWS Regions and Endpoints in the AWS documentation.

RDS

If you plan on using RDS, you must supply the RDS endpoint for your chosen region. There are two ways to do this.

  • Using the web UI interface, specify the endpoint URL directly when you define your environment. In the web interface, expand the Advanced Options section under RDS (Relational Database Service) and enter the endpoint URL for RDS region endpoint. In a configuration file, give the URL as the value for the rdsRegionEndpoint field in the provider section. Here is what an endpoint URL looks like:

    rdsRegionEndpoint: https://rds.xy-east-1.amazonaws.com
  • Rather than specifying the RDS endpoint URL with each environment you create, you can supply it in a configuration file that is read by Altus Director's AWS plugin, so it will be used for all environments created with that instance of Altus Director. The configuration file is named rds.endpoints.properties and, by default, resides in the directory /var/lib/cloudera-director-plugins/aws-provider-version/etc/. The version number for the aws-provider part of the path changes with most Altus Director releases, as the plugin changes version. For example, aws-provider-1.4.1 matches with Altus Director 2.4. So the path and file name with Altus Director 2.4 would be as follows: 
    /var/lib/cloudera-director-plugins/aws-provider-1.4.1/etc/rds.endpoints.properties
    Altus Director ships with an example of the file that you can use as a template: rds.endpoints.properties.example. Copy this file to a new rds.endpoints.properties file in that directory, and add a line for the RDS endpoint URL, for example: 
    xy-east-1=https://rds.xy-east-1.amazonaws.com

After adding a new endpoint, restart Altus Director if it is running.

IAM

The IAM service is normally accessed using a single, global endpoint that works across all AWS regions. Some regions, however, have their own IAM endpoint. If you are using such a region, supply its custom IAM endpoint. When using the web interface, expand the Advanced Options section under EC2 (Elastic Compute Cloud) on the environment page, and enter the endpoint URL for IAM endpoint. In a configuration file, specify it in the field iamEndpoint in the provider section. 
iamEndpoint: https://iam.xy-east-1.amazonaws.com

KMS

Altus Director normally computes the expected KMS endpoint for your chosen region. If that process fails, then you can provide the endpoint URL yourself. In the web interface, expand the Advanced Options section under EC2 (Elastic Compute Cloud) on the environment page, and enter the endpoint URL for KMS region endpoint. In a configuration file, specify it in the field kmsEndpoint in the provider section. 
kmsEndpoint: https://kms.xy-east-1.amazonaws.com

Other Considerations

A new AWS region usually does not support the full range of services and features that are available in older, established regions. It's important to understand what services and features your chosen region lacks, so that you do not request them through Altus Director. Altus Director does not retain knowledge on which regions have which services available.

Here are some examples of items that can work in older regions but not fully, or at all, in newer ones.
  • AMIs - common "stock" AMIs might not exist for new regions
  • instance types - deprecated instance types are often left out of new regions
  • dedicated instances (tenancy)
  • Spot blocks
  • RDS instance encryption