Adding a customer managed encryption key to a CDP environment running on AWS
By default, Data Lake and FreeIPA's Amazon Elastic Block Store (EBS) volumes and Relational Database Service (RDS) are encrypted using a default key from Amazon’s KMS, but you can optionally configure encryption using Customer Managed Keys (CMK). Data Hubs inherit environment's encryption key by default but you have an option to specify a different CMK during Data Hub creation.
Amazon offers the option to encrypt EBS volumes and RDS instances using a default key from Amazon's Key Management System (KMS) or using an external customer-managed KMS. By default, Data Lake and FreeIPA are encrypted using the default key from Amazon’s KMS present in the region where the environment is running, but you can provide a customer-managed KMS key instead of the default key.
By default, Data Hubs use the same default key from Amazon’s KMS or CMK as the parent environment but you have an option to pass a different CMK during Data Hub creation.
Encryption is configured for block devices and root devices. When encryption is configured for a given cluster, it is automatically applied to all the disk devices of any new VM instances added as a result of cluster scaling or repair.
Environment and Data Hub encryption options
To learn about encryption options that CDP offers for Data Lake, FreeIPA, and Data Hubs, refer to Environment and Data Hub encryption options.
AWS prerequisites for using a CMK
You can use your existing AWS CMK or create a new AWS CMK.
For detailed requirements, refer to AWS Requirements: Customer managed encryption keys.
Register an AWS environment with a CMK
You can specify an existing customer managed key (CMK) during AWS environment registration and the encryption key will be used to encrypt the EBS volumes and RDS instances running in the environment.
Under Customer-Managed Keys, click Enable Customer-Managed Keys.
In the same section, select the CMK:
You can use your usual CDP CLI command to create an environment with a CMK, just add the --encryption-key-arn parameter and provide the encryption key created earlier.
<ENVIRONMENT-NAME>should be replaced with an actual name. The parameter important for this feature are highlighted:
cdp environments create-aws-environment \ --environment-name <ENVIRONMENT-NAME> \ --credential-name <EXISTING_CREDENTIAL> \ --region "<REGION>" \ --security-access cidr=<CIDR> \ --authentication publicKeyId="<SSH_KEY>" \ --log-storage storageLocationBase=<BUCKET_URL>,instanceProfile=<IDBROKER_IP> \ --vpc-id <VPC_ID> \ --subnet-ids <SUBNETS \ --encryption-key-arn <ENCRYPTION_KEY_ARN>
The ARN of the encryption key created earlier should be passed in the parameter --encryption-key-arn
If the customer-managed encryption key ARN is not passed, then the AWS region-specific default encryption key is used for encrypting EBS volumes and RDS instances.
You can obtain more complete commands using the instructions in Obtain CLI commands for registering an environment.