Configuring Transparent Data Encryption for Ozone
Transparent Data Encryption (TDE) allows data on the disks to be encrypted-at-rest and automatically decrypted during access. For Ozone, you can enable TDE at the bucket-level.
- The Key Management Server must be installed and running. Ozone uses the same Key Management Server as HDFS.
- To use TDE, you (admin) must
setup a Key Management Server and provide that URI to Ozone/HDFS. You can
provide this configuration via hdfs-site.xml.
Property Value hadoop.security.key.provider.path KMS uri.e.g. kms://http@kms-host:9600/kms
- If you are using Ranger Key Management Server, then you must ensure that the key
management admin user has assigned the
omservice user with the Get Metadata and Generate EEK permissions.
Further, you must ensure that any user attempting to create an encryption key has the required permissions in Ranger.
Create a bucket encryption key.
hadoop key create enck1
This command creates an encryption key for the bucket you want to protect. After the key is created, Ozone can use that key when you are reading and writing data into a bucket.
Assign the encryption key to a bucket.
The following example shows how you can assign the key
enck1to the bucket
ozone sh bucket create -k enck1 /vol/encbucket1After you run this command, all data written to
encbucket1will be encrypted using
enck1. During the read process, the client applications interact with the Key Management Server to read the key and decrypt it.
The encryption of data is completely transparent to users and client applications.
Using Transparent Data Encryption from S3G
There are two ways to create an encrypted bucket that can be accessed via S3 Gateway.
Create a bucket using shell under “/s3v” volume.
ozone sh bucket create -k enck1 /s3v/encbucket1
Create a link to an encrypted bucket under “/s3v” volume.
ozone sh bucket create -k enck1 /vol/encbucket1 ozone sh bucket link /vol/encbucket1 /s3v/linkencryptedbucket
In non-secure mode, the user running the S3Gateway daemon process is the proxy user, while in secure mode the S3Gateway Kerberos principal (ozone.s3g.kerberos.principal) is the proxy user. The S3Gateway proxies all the users accessing the encrypted buckets to decrypt the key. Therefore, on security enabled cluster, during S3Gateway server startup logins using configured ozone.s3g.kerberos.keytab.file and ozone.s3g.kerberos.principal.You must add these two configurations to the kms-site.xml to allow the S3Gateway principal to act as a proxy for other users.In this example, “ozone.s3g.kerberos.principal” is assumed to be “s3g”.
<property> <name>hadoop.kms.proxyuser.s3g.users</name> <value>user1,user2,user3</value> <description> Here the value can be all the S3G accesskey ids accessing Ozone S3 or set to '*' to allow all the accesskey ids. </description> </property>
<property> <name>hadoop.kms.proxyuser.s3g.hosts</name> <value>s3g-host1.com</value> <description> This is the host where the S3Gateway is running. Set this to '*' to allow requests from any hosts to be proxied. </description> </property>