Configuring Cloudera Director for a New AWS Instance Type
Amazon Web Services occasionally introduces new instance types with improved specifications. Cloudera Director ships with the functionality needed to support all of the instance types available at the time of release, but customers can augment that to allow it to support new types that are introduced after release.
Updated Virtualization Mappings
Each Linux Amazon Machine Image (AMI) uses one of two types of virtualization, paravirtual or HVM. Cloudera Director ensures that the instance type of an instance that is to host an AMI supports the AMI’s virtualization type. The knowledge of which instance types support which virtualizations resides in a virtualization mappings file.
The AWS plugin included with Cloudera Director ships with an internal mappings file for all instance types that are available at the time of release. You can add new mappings, or override existing mappings, by creating another custom mappings file. Only new or changed mappings need to be included in the custom mappings file.
virtualizationMappings { customMappingsPath: ec2.customvirtualizationmappings.properties }
If the property is a relative path, it is based on the etc directory under the AWS plugin directory.
hvm=m3.medium,\ m3.large,\ m3.xlarge,\ m3.2xlarge,\ ... d2.xlarge,\ d2.2xlarge,\ d2.4xlarge,\ d2.8xlargeTo learn more about virtualization types, see Linux AMI Virtualization Types in the AWS documentation.
Updated Ephemeral Device Mappings
Each AWS instance type provides zero or more instance store volumes, also known as ephemeral storage. These volumes are distinct from EBS-backed storage volumes; some instance types include no ephemeral storage. Cloudera Director specifies naming for each ephemeral volume, and keeps a list of the number of such volumes supported per instance type in an ephemeral device mappings file.
The AWS plugin included with Cloudera Director ships with an internal mappings file for all instance types that are available at the time of release. You can add new mappings, or override existing mappings, by creating another custom mappings file. Only new or changed mappings need to be included in the custom mappings file.
ephemeralDeviceMappings { customMappingsPath: ec2.customephemeraldevicemappings.properties }
If the property is a relative path, it is based on the etc directory under the AWS plugin directory.
d2.xlarge=3 d2.2xlarge=6 d2.4xlarge=12 d2.8xlarge=24To learn more about ephemeral storage, including the counts for each instance type, see Instance Stores Available on Instance Types in the AWS documentation.
Using the New Mappings
Once the custom mappings files have been created, restart the Cloudera Director server so that they are detected and overlaid on the built-in mappings.
New instance types do not automatically appear in drop-down menus in the Cloudera Director web interface. However, the selected values for these menus can be edited by hand to specify a new instance type.