Using EncryptTool for Flink properties
Cloudera Streaming Analytics offers EncryptTool to further protect your user information and configurations when communicating with Flink using the command line. After generating a master key to the user, you need to manually encrypt the parameters and Flink automatically decrypts the protected values. You also must enable EncryptTool protection in the configuration file for Flink.
In addition to the functionality provided by the vanilla version of Apache Flink, CSA includes a solution to protect for sensitive properties in the configuration file and the dynamic properties. This way passwords in clear text to Flink can be avoided.
flink-encrypt-tool
command line
client to use the EncryptTool:- generate-key: generating master key per user. The master key is saved to an arbitrary filesystem location specified by the user, by default to the HDFS home folder of the user. It is the responsibility to protect the privileges of the key, so that it is only accessible by them. EncryptTool assumes that all sensitive properties are protected using the same key in a single configuration file.
- encrypt: encrypting configuration property. The configuration properties have to be manually encrypted and updated in the configuration file or supplied in encrypted format via dynamic properties.
Flink automatically decrypts the values based on the configuration object during runtime with the privileges of the user that has submitted the Flink job, so the visibility of the key has to be set up accordingly.
security.encrypt-tool.key.location:
hdfs:///user/alice/myencryptionkey
export
FLINK_CONF_DIR=/path/to/modified-flink-conf-dir
- Generate master key using generate-key action.
- Define a secure location for the master key.
- Use encrypt action to get the encrypted value for each sensitive key.
- Update the configuration.
security.encrypt-tool.enabled: true