Oozie
This release provides Oozie 4.2.0 and the following Apache patches:
OOZIE-1402: Increase retry interval for non-progressing coordinator action with fix value.
OOZIE-1837: LauncherMainHadoopUtils sensitive to clock skew.
OOZIE-1922: MemoryLocksService fails if lock is acquired multiple times in same thread and released.
OOZIE-1976: Specifying coordinator input datasets in more logical ways.
OOZIE-2030: Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml.
OOZIE-2159: 'oozie validate' command should be moved server-side.
OOZIE-2165: Job log fetching can fail in Oozie HA mode when using doAs impersonation.
OOZIE-2168: Oozie flow and action names have 50 char limit.
OOZIE-2173: DISCLAIMER.txt still refers to Apache Incubator.
OOZIE-2178: fix javadoc to compile on JDK8.
OOZIE-2185: Make oozie cli source conf/oozie-client-env.sh.
OOZIE-2187: Add a way to specify a default JT/RM and NN.
OOZIE-2215: Support glob in FS EL function.
OOZIE-2222: Oozie UI parent job should be clickable.
OOZIE-2228: Statustransit service doesn't pick bundle with suspend status.
OOZIE-2245: Service to periodically check database schema.
OOZIE-2246: CoordinatorInputCheckCommand does not behave properly when har file is one of data dependency and doesn't exist.
OOZIE-2250: Show log for WAITING and TIMEDOUT coord actions.
OOZIE-2251: Expose instrumental matrices in Realtime Graphing tool.
OOZIE-2257: BundleStatusTransitXCommand doesn't update last modified time.
OOZIE-2260: Only set "java.io.tmpdir" to "tmp" for the AM.
OOZIE-2262: Fix log streaming from other server with start/end filter.
OOZIE-2264: Fix coord:offset(n,"DAY") to resolve correct data set.
OOZIE-2266: Fix 'total' actions returned in coordinator job info.
OOZIE-2268: Update ActiveMQ version for security and other fixes.
OOZIE-2270: Audit log doesn't get loaded when "audit log" tab is clicked for coord job.
OOZIE-2271: Upgrade Tomcat to 6.0.44.
OOZIE-2272: Use Hadoop's CredentialProvider for passwords in oozie-site.
OOZIE-2277: Honor oozie.action.sharelib.for.spark in Spark jobs.
OOZIE-2283: Documentation should not say that System.exit is not allowed in Java Action.
OOZIE-2284: HBaseCredentials should only add hbase-default.xml and hbase-site.xml to actionConf.
OOZIE-2285: Change in concurrency should trigger coord action ready command.
OOZIE-2286: Update Log4j and Log4j-extras to latest 1.2.x release.
OOZIE-2287: Add support for deleting hcat partitions in fs action delete.
OOZIE-2289: hive-jdbc dependency in core/pom.xml should be compile.
OOZIE-2291: Hive2 workflow.xml.security should have "cred" in action tag instead of "hive2" tag.
OOZIE-2293: Oozie 4.0.1 build failed while building Catalog.
OOZIE-2300: TestAuthFilterAuthOozieClient.testClientAuthTokenCache fails with Hadoop 2.7.0 and later.
OOZIE-2303: Typo in documentation.
OOZIE-2305: Compile Oozie with Hive-1.2.0.
OOZIE-2308: Add support for bundle:conf() function.
OOZIE-2309: Enable the coord:dateOffset() function in /coordinator-app/datasets/dataset/@initial-instance.
OOZIE-2312: oozie doesn't purge audit and error log.
OOZIE-2314: Unable to kill old instance child job by workflow or coord rerun by Launcher.
OOZIE-2315: TestOozieCLI.testshareLibUpdate_withSecurity fails with Hadoop 2.
OOZIE-2318: Provide better solution for specifying SSL truststore to Oozie Client.
OOZIE-2320: TestZKXLogStreamingService.testStreamingWithMultipleOozieServers_coordActionList is failing.
OOZIE-2322: Oozie Web UI doesn't work with Kerberos in Internet Explorer 10 or 11 and curl.
OOZIE-2324: A syntax error in the kill node causes the workflow to get stuck and other problems.
OOZIE-2325: Shell action fails if user overrides oozie.launcher.mapreduce.map.env.
OOZIE-2328: Coordinator endtime change should check if the last action is in database.
OOZIE-2329: Make handling yarn restarts configurable.
OOZIE-2330: Spark action should take the global jobTracker and nameNode configs by default and allow file and archive elements.
OOZIE-2332: Add ability to provide Hive and Hive 2 Action queries inline in workflows.
OOZIE-2343: Shell Action should take Oozie Action config and setup HADOOP_CONF_DIR.
OOZIE-2344: Enabling 'oozie.action.jobinfo.enable' doesn't inject the job information into the map/reduce job's configuration..
OOZIE-2345: Parallel job submission for forked actions.
OOZIE-2346: Add sub-workflow information like the super-parent id and workflow depth into the 'oozie.job.info' property.
OOZIE-2347: Remove unnecessary new Configuration()/new jobConf() calls from oozie.
OOZIE-2348: Recovery service keeps on recovering coord action of suspended jobs.
OOZIE-2350: Package changes for release.
OOZIE-2355: Hive2 Action doesn't pass along oozie configs to jobconf.
OOZIE-2356: Add a way to enable/disable credentials in a workflow.
OOZIE-2358: Coord rerun cleanup should reuse hcat connections.
OOZIE-2360: Spark Action fails due to missing mesos jar.
OOZIE-2362: SQL injection in BulkJPAExecutor.
OOZIE-2365: oozie fail to start when smtp password not set.
OOZIE-2367: fs delete should support skipTrash option.
OOZIE-2368: coord:dateOffset and coord:dateTzOffset can't be used for coord initial-instance.
OOZIE-2369: coord:offset doesn't resolve correctly.
OOZIE-2370: Docs for Coordinator Action Status Notification has wrong property name.
OOZIE-2376: Default action configs not honored if no <configuration> section in workflow.
OOZIE-2377: Hive2 Action should not propagate oozie.hive2.* properties to Beeline.
OOZIE-2378: org.apache.oozie.client.TestOozieCLI.testAdminInstrumentation is flakey.
OOZIE-2379: org.apache.oozie.command.coord.TestCoordPushDependencyCheckXCommand.testRequeueOnException is flakey.
OOZIE-2380: Oozie Hive action failed with wrong tmp path.
OOZIE-2382: org.apache.oozie.action.hadoop.TestPigMain.testPig_withNullExternalID is flakey.
OOZIE-2385: org.apache.oozie.TestCoordinatorEngineStreamLog.testCoordLogStreaming is flakey.
OOZIE-2386: org.apache.oozie.util.TestXConfiguration.testSubstituteVar is flakey.
OOZIE-2391: spark-opts value in workflow.xml is not parsed properly.
OOZIE-2394: Oozie can execute command without holding lock.
OOZIE-2397: LAST_ONLY and NONE don't properly handle READY actions.
OOZIE-2400: Workflow xml configuration parser cannot deal with namespace prefix.
OOZIE-2401: Typo in twiki docs with FilesAchives instead of FilesArchives.
OOZIE-2402: oozie-setup.sh sharelib create takes a long time on large clusters.
OOZIE-2407: AbandonedService should not send mail if there is no abandoned coord.
OOZIE-2411: Add BCC to oozie email action.
OOZIE-2413: Kerberos credentials can expire if the KDC is slow to respond.
OOZIE-2419: HBase credentials are not correctly proxied.
OOZIE-2428: TestSLAService, TestSLAEventGeneration flakey tests.
OOZIE-2429: TestEventGeneration test is flakey.
OOZIE-2430: Add root logger for hive,sqoop action.
OOZIE-2431: Support hive.metastore.uris as alias for hcat.metastore.uri.
OOZIE-2432: TestPurgeXCommand fails.
OOZIE-2434: inconsistent coord action status and workflow job status.
OOZIE-2435: TestCoordChangeXCommand is flakey.
OOZIE-2436: Fork/join workflow fails with "oozie.action.yarn.tag must not be null".
OOZIE-2437: Exclude additional Hadoop dependencies from Spark Sharelib.
OOZIE-2438: Oozie client "jobs -filter" diagnostic message clarification.
OOZIE-2439: FS Action no longer uses name-node from global section or default NN.
OOZIE-2441: SubWorkflow action with propagate-configuration but no global section throws NPE on submit.
OOZIE-2444: Need conditional logic in bundles.
OOZIE-2445: Doc for - Specifying coordinator input datasets in more logical ways.
OOZIE-2446: Job does not fail during submission if non existent credential is specified.
OOZIE-2447: Illegal character 0x0 oozie client.
OOZIE-2452: Coordinator Functional Specification - EL Constants Typo.
OOZIE-2466: Repeated failure of TestMetricsInstrumentation.testSamplers.
OOZIE-2467: Oozie can shutdown itself on long GC pause.
OOZIE-2470: Remove infinite socket timeouts in the Oozie email action.
OOZIE-2471: Show child job url tab for distcp.
OOZIE-2474: <job-xml> is not being applied to the launcher job.
OOZIE-2475: Oozie does not cleanup action dir of killed actions.
OOZIE-2476: When one of the action from fork fails with transient error, WF never joins.
OOZIE-2481: Add YARN_CONF_DIR in the Shell action.
OOZIE-2482: Pyspark job fails with Oozie.
OOZIE-2485: Oozie client keeps trying to use expired auth token.
OOZIE-2486: TestSLAEventsGetForFilterJPAExecutor is flakey.
OOZIE-2489: XML parsing is vulnerable.
OOZIE-2490: Oozie can't set hadoop.security.token.service.use_ip.
OOZIE-2492: JSON security issue in js code.
OOZIE-2503: show ChildJobURLs to spark action.
OOZIE-2504: Create a log4j.properties under HADOOP_CONF_DIR in Shell Action.
OOZIE-2509: SLA job status can stuck in running state.
OOZIE-2510: TestLogStreamer.testStreamLog() fails in other timezones.
OOZIE-2511: SubWorkflow missing variable set from option if config-default is present in parent workflow.
OOZIE-2512: ShareLibservice returns incorrect path for jar.
OOZIE-2522: There can be multiple coord submit from bundle in case of ZK glitch.
OOZIE-2529: Support adding secret keys to Credentials of Launcher.
OOZIE-2531: Prevent Spark trying for token which is already available.
OOZIE-2532: patch apply does not handle binary files.
OOZIE-2535: User can't disable uber mode.
OOZIE-2537: SqoopMain does not set up log4j properly.
OOZIE-2541: Possible resource leak in Hive2Credentials.
OOZIE-2542: Option to disable OpenJPA BrokerImpl finalization.
OOZIE-2546: Improperly closed resources in OozieDBCLI.
OOZIE-2547: Add mapreduce.job.cache.files to spark action.
OOZIE-2548: Flaky test TestZKLocksService.testLockReaper.
OOZIE-2550: Flaky tests in TestZKUUIDService.java.
OOZIE-2551: Feature request: epoch timestamp generation.
OOZIE-2553: Cred tag is required for all actions in the workflow even if an action does not require it.
OOZIE-2556: TestAbandonedCoordChecker.testCatchupJob is flaky.
OOZIE-2563: Pass spark-defaults.conf to spark action.
OOZIE-2567: HCat connection is not closed while getting hcat cred.
OOZIE-2570: remove -PtestPatchCompile from patch testing as there is no such profile.
OOZIE-2577: Flaky tests TestCoordActionInputCheckXCommand.testTimeout and testTimeoutWithException.
OOZIE-2578: Oozie example distcp job fails to run within an encrypted zone with checksum match error.