Fixed Issues in Apache YARN and YARN Queue Manager
Review the list of YARN and YARN Queue Manager issues that are resolved in Cloudera Runtime 7.2.18.
- COMPX-15361: Backport YARN-11590 to cdpd-master: RM process stuck after calling confStore.format() when ZK SSL/TLS is enabled, as netty thread waits indefinitely
- Fixed the issue when SSL/TLS was enabled for YARN with Curator leader-elector, a Netty thread rendered RM unresponsive.
- COMPX-15323: YARN Client unit tests are failing because of "NoClassDefFoundError: org/junit/jupiter/api/TestInfo"
- Unit test dependency added to the pom.
- COMPX-15308: YARN-11578 Fix performance issue of permission check in verifyAndCreateRemoteLogDir
- Performance issue fix in LogAggregation's verifyAndCreateRemoteLogDir.
- COMPX-15299: YARN-9954 Configurable max application tags and max tag length
- Configurable max application tags and length.
- COMPX-15289: Make t9000 compatible with mixed resource mode allocation
- Now the fine plugin is disabled in mixed mode
- COMPX-15288: [7.2.18] QM Database migration breaks for clusters where H2 database prior to migration had duplicate config_sets for a namespace, version pair
- FIx tracked under COMPX-15216
- COMPX-15272: Yarn - Queue Manager cannot load correctly on 7.2.18
- Changes done in https://jira.cloudera.com/browse/COMPX-15432 to make embedded H2 db as the default db again removing the requirement on a postgres db for QM in 7.2.18
- COMPX-15111: Backport HADOOP-18870: CURATOR-599 change broke functionality introduced in HADOOP-18139 and HADOOP-18709
- Hadoop Common can be configured now to communicate via SSL/TLS with ZK.
- COMPX-14929: Backport YARN-11468: Zookeeper SSL/TLS support
- Zookeeper SSL/TLS support enabled for YARN.
- COMPX-14928: Backport HADOOP-18709:Add curator-based ZooKeeper communication support over SSL/TLS into the common library
- Hadoop Common can be configured now to communicate via SSL/TLS with ZK.
- COMPX-14855: Backporting YARN-11535 (Remove jackson-dataformat-yaml dependency)
- Removed Jackson-dataformat-yaml to fix CVEs
- COMPX-14759: Queue Manager - Upgrade commons-configuration2 to 2.9.0 due to CVEs
- Updated commons-configuration2 to 2.9.0 for 7.1.9 CHF2
- COMPX-14340: YARN-11490 JMX QueueMetrics breaks after mutable config validation in CS
- Fix: JMX metrics broke after 2 or more configuration validation.
- COMPX-14064: Default ULF for dynamic queue template should be -1 in Weight mode
- Set Default ULF for dynamic queue template to be -1
- COMPX-13056: QM - Upgrade Grizzly-http to 4.0.0 due to CVE-2017-1000028
- Removed grizzly dependency
- COMPX-9022: YARN Scaling metrics API should return queue statistics
- The API was extended with new debug infos
- CDPD-64617: [7.2.18][terminate-flow] yarn cmd -getServiceState trying to connect to an intentionally down host in ranger-post
- Due to multi-comp execution on the same cluster some services are in stopped state. When recreated the scenario with one master node down and all services running this issue is gone and tests started executing.
- CDPD-57948: [7.1.9 ZDU Simulation] Hive Query is failing when YARN is into rolling restart
- YARN-side fix is implemented and backported to cdpd-master and 7.1.9.x
Apache patch information
- MAPREDUCE-7468
- MAPREDUCE-7446
- MAPREDUCE-7432
- YARN-11578
- YARN-9954
- YARN-11520
- YARN-11551
- YARN-11535
- YARN-11545
- YARN-11533
- YARN-11490
- YARN-11621
