Configuring Services to Use the GPL Extras Parcel

After you install the GPL Extras parcel, reconfigure and restart services that need to use LZO functionality. Any service that does not require the use of LZO need not be configured.

HDFS

  1. Go to the HDFS service.
  2. Click the Configuration tab.
  3. Search for the io.compression.codecs property.
  4. In the Compression Codecs property, click in the field, then click the + sign to open a new value field.
  5. Add the following two codecs:
    • com.hadoop.compression.lzo.LzoCodec
    • com.hadoop.compression.lzo.LzopCodec
  6. Save your configuration changes.
  7. Restart HDFS.
  8. Redeploy the HDFS client configuration.

Oozie

  1. Go to /var/lib/oozie on each Oozie server and even if the LZO JAR is present, symlink the Hadoop LZO JAR:
    • CDH 5 - /opt/cloudera/parcels/GPLEXTRAS/lib/hadoop/lib/hadoop-lzo.jar
    • CDH 4 - /opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/hadoop-lzo.jar
  2. Restart Oozie.

HBase

Restart HBase.

Impala

Restart Impala.

Hive

Restart the Hive server.

Sqoop 1

  1. Add the following entries to the Sqoop 1 Client Client Advanced Configuration Snippet (Safety Valve)
    • HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/cloudera/parcels/GPLEXTRAS/lib/hadoop/lib/
    • JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:/opt/cloudera/parcels/GPLEXTRAS/lib/hadoop/lib/native
  2. Re-deploy the client configuration.

Sqoop 2

  1. Add the following entries to the Sqoop 2 Service Environment Advanced Configuration Snippet:
    • HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/cloudera/parcels/GPLEXTRAS/lib/hadoop/lib/*
    • JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:/opt/cloudera/parcels/GPLEXTRAS/lib/hadoop/lib/native
  2. Restart the Sqoop service.