Cloudera Docs

Oozie Sidecar migration to CDP

To migrate Oozie to CDP, you must manually migrate workloads and configurations to the destination cluster.

The Oozie database on the source cluster is automatically configured to purge data that is older than 30 days. If you need to retain this data, Cloudera recommends that you back up the data and make it available for running simple SQL queries. See Back Up the Databases.

Ensure that you have set up a new Oozie database on the destination cluster. (You may have already done this when you installed ​​Cloudera Manager on the destination cluster.)

Most Oozie workloads and configurations will need to be migrated manually to the destination cluster. Although you can copy the XML and property files from the source cluster, these files will require extensive modifications before they can be used in the destination cluster. Cloudera Professional Services can assist you with these migrations.

In general, you will need to ensure that the following settings are correct for the destination cluster:

  • Name-node settings for each workflow.
  • Resource-manager settings for each workflow.
  • Paths, if the locations are being changed
  • JDBC URLs, including Hive must point to the new database,
  • DFS paths may need to be replaced with S3, Azure, as appropriate
  • New cluster-specific configurations for the Workflow (for example, Amazon S3 credentials.)
  • Coordinator Jobs that have future sdf runs scheduled. Recreate these jobs to match what was included in HDP. For such Jobs, the Workflow and its configuration might be stored in Hue.
  • Copy any custom sharelib jars from the source to the destination cluster. The jars are stored here: /user/oozie/share/lib/lib_{TIMESTAMP}/{COMPONENT}
  • If you have custom application jars used by any of your workflows, please make sure you recompile them with the 3rd-party libraries used by the latest Cloudera runtime. For example on older versions of HDP, Oozie was using Guava 11, but in CDP 7 it is using Guava 28 or higher. If you compiled your application with Guava 11 you must also upgrade Guava.
  • On the destination cluster, update the sharelib using the following command: 
    oozie admin -oozie {URL} -sharelibupdate