Cloudera Docs

Unable to insert data due to noexec flag on /tmp directory

Condition

If you are running an environment in which the /tmp directory is mounted with the noexec option, Hive table insertion fails stating that the Tez compiler could not initialize class org.xerial.snappy.Snappy.

Cause

While inserting data into a Hive table through a Beeline query, the insert operation can fail with the following error:
Error: Error while compiling statement: FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Could not initialize class org.xerial.snappy.Snappy (state=08S01,code=-101)

The error occurs when the /tmp directory is mounted with the noexec option. The noexec mount option is used as an enhanced security measure to prevent running of binaries from the /tmp directory. However, Hive refers to the /tmp library path to access the Snappy JARs and since the noexec mount option is set on /tmp, the compiler fails to initialize.

Remedy

  1. Log in to Cloudera Manager as an administrator.
  2. Go to Clusters > Hive on Tez > Configuration and search for the hiveserver2_java_opts property.
  3. Specify a temporary directory other than /tmp, for example, /opt/tmp by appending the following in hiveserver2_java_opts. 
    -Djava.io.tmpdir=/opt/tmp -Dorg.xerial.snappy.tempdir=/opt/tmp

    Java configuration options to specify temp directory
  4. Search for the hive.exec.local.scratchdir property and set the value to /opt/tmp.

    Configuration option to set hive local scratch directory
  5. Click Save Changes
  6. Before you restart the Hive on Tez service, ensure that the temporary directory (/opt/tmp) that you specified is present in all the hosts and the Hive user has access to this directory.
  7. Restart the Hive on Tez service for the changes to apply.