Configuring Transparent Data Encryption for Ozone

Transparent Data Encryption (TDE) allows data on the disks to be encrypted-at-rest and automatically decrypted during access. For Ozone, you can enable TDE at the bucket-level.

  • The Key Management Server must be installed and running. Ozone uses the same Key Management Server as HDFS.
  • To use TDE, you (admin) must setup a Key Management Server and provide that URI to Ozone/HDFS. You can provide this configuration via hdfs-site.xml.
    Property Value KMS uri.e.g. kms://http@kms-host:9600/kms
  • If you are using Ranger Key Management Server, then you must ensure that the key management admin user has assigned the om service user with the Get Metadata and Generate EEK permissions.

    Further, you must ensure that any user attempting to create an encryption key has the required permissions in Ranger.

  1. Create a bucket encryption key.
    hadoop key create enck1

    This command creates an encryption key for the bucket you want to protect. After the key is created, Ozone can use that key when you are reading and writing data into a bucket.

  2. Assign the encryption key to a bucket.
    The following example shows how you can assign the key enck1 to the bucket encbucket1:
    ozone sh bucket create -k enck1 /vol/encbucket1
    After you run this command, all data written to encbucket1 will be encrypted using enck1. During the read process, the client applications interact with the Key Management Server to read the key and decrypt it.

    The encryption of data is completely transparent to users and client applications.

Using Transparent Data Encryption from S3G

There are two ways to create an encrypted bucket that can be accessed via S3 Gateway.

  1. Create a bucket using shell under “/s3v” volume.
    ozone sh bucket create -k enck1 /s3v/encbucket1
  2. Create a link to an encrypted bucket under “/s3v” volume.
    ozone sh bucket create -k enck1 /vol/encbucket1
    ozone sh bucket link  /vol/encbucket1 /s3v/linkencryptedbucket

    In non-secure mode, the user running the S3Gateway daemon process is the proxy user, while in secure mode the S3Gateway Kerberos principal (ozone.s3g.kerberos.principal) is the proxy user. The S3Gateway proxies all the users accessing the encrypted buckets to decrypt the key. Therefore, on security enabled cluster, during S3Gateway server startup logins using configured ozone.s3g.kerberos.keytab.file and ozone.s3g.kerberos.principal.

    You must add these two configurations to the kms-site.xml to allow the S3Gateway principal to act as a proxy for other users.In this example, “ozone.s3g.kerberos.principal” is assumed to be “s3g”.
            Here the value can be all the S3G accesskey ids accessing Ozone S3 
            or set to '*' to allow all the accesskey ids.
             This is the host where the S3Gateway is running. Set this to '*' to allow
             requests from any hosts to be proxied.