Using EBS Volumes for Cloudera Manager and CDH
EBS Volume Types
|EBS volume type||Minimum and Maximum Size||Usage|
|gp2||1 GiB - 16 TiB||General-purpose SSD (solid state drive) volume that balances price and performance for a wide variety of transactional workloads.|
|st1||500 GiB - 16 TiB||Low-cost HDD (hard disk drive) volume designed for frequently accessed, throughput-intensive workloads.|
|sc1||500 GiB - 16 TiB||Lowest-cost HDD (hard disk drive) volume designed for less frequently accessed workloads.|
For more information, see Amazon EBS Volume Types.
Amazon EC2 Instance Stores
If an instance type has instance store volumes and you do not specify EBS volumes, Cloudera Director automatically mounts all the instance store volumes that are available. If you do specify EBS volumes, Cloudera Director does not mount instance store volumes.
For more information on EC2 instance stores, see Amazon EC2 Instance Stores in the AWS documentation.
Configuring EBS Volumes
- Number of EBS volumes you want
- Type of the EBS volumes (gp2, st1, or sc1). All EBS volumes for an instance must be of the same type.
- Size of the volumes. Specifying a size outside the ranges defined in the table above causes cluster deployment to fail.
- Whether or not to encrypt data in the EBS volume
- Whether to use the default KMS key for the EBS service or use a custom KMS key
EBS volumes for a Cloudera Manager or CDH cluster instance have the same lifecycle as the instance. EBS volumes are terminated when the instance is terminated. Repair of an instance does not result in the remounting of an existing EBS volume; a new volume is used.
EBS Volume Encryption
- enableEbsEncryption: Labeled Enable EBS Encryption in the web UI. Set to true or false. If this value is set to true, the data on EBS volumes created with this instance template will be encrypted.
- ebsKmsKeyId: Labeled EBS KMS Key ID in the web UI. The key used to encrypt data in the EBS volumes. KMS includes a
default master key for each service that supports encryption, including EBS. If you leave this field empty, Cloudera Director configures the EBS volumes to use the KMS default master key for EBS.
Alternatively, you can import a custom master key from your own key management infrastructure into KMS and specify it here to be used for the EBS service. To specify a custom master key, enter the
full Amazon Resource Name (ARN) of the custom master key that you have stored in
KMS: arn:aws:kms:your_key_name. For example:
Configuring an EBS Volume with the Web UI
To configure EBS volumes in the web UI, provide the required values in the Advanced Options section of the instance template:
Configuring EBS Volumes with the Configuration File
# # EBS Volumes # # Director can create and attach additional EBS volumes to the instance. These volumes # will be automatically deleted when the associated instance is terminated. These # properties don't apply to the root volume. # # See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumes.html # # ebsVolumeCount : 0 # ebsVolumeType: st1 # specify either st1, sc1 or gp2 volume type # ebsVolumeSizeGiB: 500 # # EBS Volume Encryption # # Encryption can be enabled on the additional EBS volumes. An optional CMK can # be specified for volume encryption. Not setting a CMK means the default CMK # for EBS will be used. The encryption here does not apply to the root volume. # # See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html # # enableEbsEncryption: false # ebsKmsKeyId: arn:aws:kms:REPLACE-ME # full ARN of the KMS CMK