Configuring Apache HDFS Encryption
Also available as:
PDF
loading table of contents...

Encrypting Other Hive Directories

Reference information on encrypting other Hive directories.

  • LOCALSCRATCHDIR : The MapJoin optimization in Hive writes HDFS tables to a local directory and then uploads them to distributed cache. To enable encryption, either disable MapJoin (set hive.auto.convert.join to false) or encrypt the local Hive Scratch directory (hive.exec.local.scratchdir). Performance note: disabling MapJoin will result in slower join performance.
  • DOWNLOADED_RESOURCES_DIR : Jars that are added to a user session and stored in HDFS are downloaded to hive.downloaded.resources.dir. If you want these Jar files to be encrypted, configure hive.downloaded.resources.dir to be part of an encryption zone. This directory needs to be accessible to the HiveServer2.
  • NodeManager Local Directory List: Hive stores Jars and MapJoin files in the distributed cache, so if you'd like to use MapJoin or encrypt Jars and other resource files, the YARN configuration property NodeManager Local Directory List (yarn.nodemanager.local-dirs) must be configured to a set of encrypted local directories on all nodes.

    Alternatively, to disable MapJoin, set hive.auto.convert.join to false.