Fixed Issues in Apache YARN and YARN Queue Manager

Review the list of YARN and YARN Queue Manager issues that are resolved in Cloudera Runtime 7.2.18.

COMPX-15361: Backport YARN-11590 to cdpd-master: RM process stuck after calling confStore.format() when ZK SSL/TLS is enabled, as netty thread waits indefinitely
Fixed the issue when SSL/TLS was enabled for YARN with Curator leader-elector, a Netty thread rendered RM unresponsive.
COMPX-15323: YARN Client unit tests are failing because of "NoClassDefFoundError: org/junit/jupiter/api/TestInfo"
Unit test dependency added to the pom.
COMPX-15308: YARN-11578 Fix performance issue of permission check in verifyAndCreateRemoteLogDir
Performance issue fix in LogAggregation's verifyAndCreateRemoteLogDir.
COMPX-15299: YARN-9954 Configurable max application tags and max tag length
Configurable max application tags and length.
COMPX-15289: Make t9000 compatible with mixed resource mode allocation
Now the fine plugin is disabled in mixed mode
COMPX-15288: [7.2.18] QM Database migration breaks for clusters where H2 database prior to migration had duplicate config_sets for a namespace, version pair
FIx tracked under COMPX-15216
COMPX-15272: Yarn - Queue Manager cannot load correctly on 7.2.18
Changes done in https://jira.cloudera.com/browse/COMPX-15432 to make embedded H2 db as the default db again removing the requirement on a postgres db for QM in 7.2.18
COMPX-15111: Backport HADOOP-18870: CURATOR-599 change broke functionality introduced in HADOOP-18139 and HADOOP-18709
Hadoop Common can be configured now to communicate via SSL/TLS with ZK.
COMPX-14929: Backport YARN-11468: Zookeeper SSL/TLS support
Zookeeper SSL/TLS support enabled for YARN.
COMPX-14928: Backport HADOOP-18709:Add curator-based ZooKeeper communication support over SSL/TLS into the common library
Hadoop Common can be configured now to communicate via SSL/TLS with ZK.
COMPX-14855: Backporting YARN-11535 (Remove jackson-dataformat-yaml dependency)
Removed Jackson-dataformat-yaml to fix CVEs
COMPX-14759: Queue Manager - Upgrade commons-configuration2 to 2.9.0 due to CVEs
Updated commons-configuration2 to 2.9.0 for 7.1.9 CHF2
COMPX-14340: YARN-11490 JMX QueueMetrics breaks after mutable config validation in CS
Fix: JMX metrics broke after 2 or more configuration validation.
COMPX-14064: Default ULF for dynamic queue template should be -1 in Weight mode
Set Default ULF for dynamic queue template to be -1
COMPX-13056: QM - Upgrade Grizzly-http to 4.0.0 due to CVE-2017-1000028
Removed grizzly dependency
COMPX-9022: YARN Scaling metrics API should return queue statistics
The API was extended with new debug infos
CDPD-64617: [7.2.18][terminate-flow] yarn cmd -getServiceState trying to connect to an intentionally down host in ranger-post
Due to multi-comp execution on the same cluster some services are in stopped state. When recreated the scenario with one master node down and all services running this issue is gone and tests started executing.
CDPD-57948: [7.1.9 ZDU Simulation] Hive Query is failing when YARN is into rolling restart
YARN-side fix is implemented and backported to cdpd-master and 7.1.9.x

Apache patch information

  • MAPREDUCE-7468
  • MAPREDUCE-7446
  • MAPREDUCE-7432
  • YARN-11578
  • YARN-9954
  • YARN-11520
  • YARN-11551
  • YARN-11535
  • YARN-11545
  • YARN-11533
  • YARN-11490
  • YARN-11621