Fixed Issues in Apache Oozie

Review the list of Oozie issues that are resolved in Cloudera Runtime 7.1.7.

CDPD-22161: When the Yarn remote log dir was set to s3 or abfs, log aggregation for Oozie actions was not working by default. The workaround for this was to extend the IdBroker mapping and add the oozie user there, or to add an explicit file-system credential (which pointed to the Yarn remote log dir) to the Oozie Workflow. These workarounds are no longer required as from now on a delegation token for the remote log dir will be obtained from IdBroker in the name of the user who is running the Workflow.

This issue is resolved.

OPSAPS-59215: Oozie will now use the Hadoop native libraries when JNI group mapping is enabled.

This issue is resolved.

CDPD-19684: Oozie will now automatically pick-up the hive-site.xml and add it to the Yarn container of a Spark action so from now on it's not necessary to put a hive-site.xml manually onto Oozie Spark's sharelib.

This issue is resolved.

CDPD-23141: Sqoop logs were not present in the aggregated Yarn logs

This issue is resolved.

CDPD-25982: Fixed a regression introduced in CDPD-21870 where Oozie always used the unix user and not the Kerberos user. However this fix did not take the Kerberos name rules into consideration which is now fixed in scope of this ticket.

This issue is resolved.

CDPD-26692: The Oozie purge process for bundles is creating orphan coordinators. When purging bundle jobs and bundle actions, it does not always purge coordinator jobs. This causes orphaned coordinators, meaning neither they nor their children will ever be purged due to the purge logic. Hence the purge logic was modified to only purge bundles and coordinators if all of their children (Workflows) can be purged.

This issue is resolved.

CDPD-27541: In case Yarn resource manager is in HA and the resource manager for an Oozie action was specified with an alias then Oozie failed to obtain the actual Yarn RM principal because it could not resolve the hostname for the alias. This did not cause an issue if there were no auth-to-local rules as Oozie did a fallback to use the short name from the principal template. For example it extracted "yarn" from "yarn/_HOST@ROOT.HWX.SITE". However when there are auth-to-local rules then Yarn could not map the short name. This was fixed by Oozie handling the case when Yarn is in HA and/or a resource manager address alias is used.

This issue is resolved.

CDPD-18703: Oozie version returning incorect values

This issue is resolved.

CDPD-20002: When killed, SSH action must stop the spawned processes on target Host if specified.

When the SSH action is killed the child processes launched by the actions are not killed. The default behaviour is still these not getting killed but found 2 ways: 1. Use the new 0.3 schema version for your SSH action in your workflow.xml and add the "terminate-subprocesses" XML element with value "true". For example, :<terminate-subprocesses>true</terminate-subprocesses> 2. You can set this globally by adding the following oozie-site.xml safety-valve in Cloudera Manager with value "true" : "oozie.action.ssh.action.terminate.subprocesses" If both are set then the value set in the workflow.xml takes precedence. This issue is resolved.

CDPD-11965: Cookie without HttpOnly and Secure flag set.

The Secure and HttpOnly attributes are now set on all Cookies returned by Oozie as per recommendations. This issue is resolved.

CDPD-19281: Oozie - Missing CSP, X-XSS-Protection, HSTS Headers.

Oozie is enhanced with extra HTTP Headers to make it more secure. In scope of these enhancements the following HTTP Headers are now returned by Oozie: X-XSS-Protection with value "1; mode=block" ; Content-Security-Policy with value "default-src 'self'" ; Strict-Transport-Security with value "max-age=31536000; includeSubDomains". You can remove these Headers by adding an oozie-site.xml safety-valve with an empty value - must be a space - in Cloudera Manager with the "oozie.servlets.response.header." prefix. For example, "oozie.servlets.response.header.Strict-Transport-Security= " You can also modify the value of these Header the same way through a safety-valve. For example, "oozie.servlets.response.header.Strict-Transport-Security=max-age=604800; includeSubDomains" Using the same prefix you can also make Oozie return custom HTTP Headers. For example, "oozie.servlets.response.header.MyHeader=MyValue". This issue is now resolved.

CDPD-17648: Oozie HWC (hive-warehouse-connector) jar conflicts in CDP DC

The Hive Warehouse Connector is now compatible with Oozie.

CDPD-9174: Missing atlas-application.properties in Oozie ShareLib

This issue is now resolved.

When upgrading from CDH5 / HDP 2.6.5, if you have oozie.service.CallbackService.base.url defined as a safety-valve, you need to remove it as it will be configured by Cloudera Manager.

CDPD-18931: "No appropriate protocol" error with email action(disable TLS1.0/1.1)

This issue is now resolved.

When upgrading from CDH5 / HDP 2.6.5, if you have oozie.service.CallbackService.base.url defined as a safety-valve, you need to remove it as it will be configured by Cloudera Manager.

CDPD-21870: A bug in the Oozie CLI launched the workflow in the name of the current Unix user even if Kerberos authentication is used with a ticket for a different user.

This issue is now resolved.

CDPD-20984: The falcon-oozie-el-extension library written for HDP 2.6 is not compatible with CDP 7.x Oozie. Introduced a change in Oozie to make that library forward compatible with Oozie in CDP 7. You must note that the rest of the HDP 2.6 Falcon library is still not compatible with CDP but only falcon-oozie-el-extension is.

This issue is now resolved.

CDPD-20649: Revise yarn.app and mapreduce property overrides in Oozie.

When upgrading from CDH 5 or HDP 2 or HDP 3, Oozie is able to handle the following map-reduce related properties: * oozie.launcher.mapreduce.map.memory.mb * oozie.launcher.mapreduce.map.cpu.vcores * oozie.launcher.mapreduce.map.java.opts These were originally decommissioned when Oozie was rebased from 4.x to 5.x, but to reduce the migration effort for users these are supported again. This issue is resolved.

CDPD-19473: Security vulnerability in Oozie's sharelib CLI created a temporary directory.

This issue is now resolved.