Set up the Sharelib
Run the following command as oozie user:
cd /tmp tar xzf /usr/hdp/${hdp_version}/oozie/oozie-sharelib.tar.gz hadoop fs -put share /user/oozieAdd the two zip files from spark-client python to spark share lib:
hdfs dfs -put /usr/hdp/current/spark-client/python/lib/py4j-0.9-src.zip /user/oozie/share/lib/lib_<timestamp>/spark
hdfs dfs -put /usr/hdp/current/spark-client/python/lib/pyspark.zip /user/oozie/share/lib/lib_<timestamp>/spark
Update the oozie share lib:
oozie admin –oozie http://oozie_server:11000/oozie -sharelibupdate