Fixed Issues in Apache YARN and YARN Queue Manager

Review the list of YARN and YARN Queue Manager issues that are resolved in Cloudera Runtime 7.2.17.

CDPD-2936: Application logs are not accessible in WebUI2 or Cloudera Manager
Fixed the issue so logs are now visible.
COMPX-1445: Queue Manager operations are failing when Queue Manager is installed separately from YARN
Fixed issue.
COMPX-14340: YARN-11490 JMX QueueMetrics breaks after mutable config validation in CS
Fix: JMX metrics broke after 2 or more configuration validation.
COMPX-14147: YARN-11312 [UI2] Refresh buttons don't work after EmberJS upgrade
Fixed the issue of EmberJs variable change from targetObject to _targetObject
COMPX-14120: Backport YARN-11463: Node Labels root directory creation doesn't have a retry logic
Retry logic is implemented and backported for root directory creation during RM node label store inititalization.
COMPX-13773: YARN-11461 NPE in determineMissingParents when the queue is invalid
Fix NPE log warning when submitting to invalid queue.
COMPX-13423: MAPREDUCE-7433 Remove unused mapred/LoggingHttpResponseEncoder.java
Unused code.
COMPX-13272: HADOOP-18602 Remove netty3 dependency
netty3 is removed from the dependencies
COMPX-12661: YARN-11075 Explicitly declare serialVersionUID in LogMutation class
The serialVersionUID field is explicitly set for the LogMutation class.
COMPX-12645: Backport YARN-10946
Changes are submitted to cdpd-master.
COMPX-12560: Fix JSON formatting in the RM tests
The previous backports are extended in cdpd-master, CDH-7.1.7.2000, CDH-7.2.16.x and CDH-7.1.8.x branches.
COMPX-12524: Backport YARN-10005
Code improvements in MutableCSConfigurationProvider
COMPX-11487: RM service experiences failure at recovery when auto created queue has issues
Fix NPE log warning when submitting to invalid queue.
COMPX-11384: Backport YARN-11063
Fixed Issue Text
COMPX-10909: Investigate if placement rules are working fine if username contains dot, and default queue is set to that queue
Usernames with dot now will work well with CS placement rules
COMPX-7460: YARN-10965. Introduce universal capacity resource vector
New feature added to CS, not yet wired in so it won't change any existing CS config or behaviour.
CDPD-50485: [7.2.17][azure] yarn applicaitons stuck in accepted state
Root cause fixed in https://jira.cloudera.com/browse/OPSAPS-66526.
COMPX-13719: Queue creation is failing with 500 errors in cdp_cloud_7_2_17
Fixed as part of OPSAPS-66613
COMPX-13567: MAPREDUCE-7434 Fix ShuffleHandler tests
Unit test fix.
COMPX-13422: MAPREDUCE-7268 Fix TestMapreduceConfigFields
Test fix.
COMPX-13421: MAPREDUCE-7237 Supports config the shuffle's path cache related parameters
ShuffleHandler's cache related parameters are configurable.
COMPX-12964: MAPREDUCE-7431 ShuffleHandler is not working correctly in SSL mode after the Netty 4 upgrade
Bugfix.
COMPX-12922: QueueManager should allow configuring absolute capacities beyond current cluster capacity
n/a
COMPX-12872: [Config-Service]- Queue Manager - Upgrade OkHttp to 3.14.9/4.10.0 due to medium CVEs
Upgrade OkHttp to 4.10.0
COMPX-12803: QM 7.1.8 CHF4 - Upgrade Apache Commons Text to 1.10.0 due to CVE-2022-42889
n/a
COMPX-12487: Queue Manager 7.2.17 - Upgrade jackson-databind to 2.13.3 due to high CVEs
n/a
COMPX-12374: [CR-7.2.16.0] Notice file update - config-service, cpx
n/a
COMPX-12341: CPX [QM]Upgrade snakeyaml to 1.33 due to high CVEs
Upgrade snakeyaml to 1.33
COMPX-12340: CPX [Config-Store]Upgrade snakeyaml to 1.33 due to high CVEs
Upgrade snakeyaml to 1.33
COMPX-12246: Config Service - Upgrade Apache Commons Text to 1.10.0 due to CVE-2022-42889
Upgrade commons-text to 1.10.0
COMPX-12228: QM - Upgrade jersey to 2.35 / 3.0.2 due to CVE-2021-28168
Upgrade jersey to 2.37 and jackson to 2.13.4
COMPX-11479: [CPX] Queue Manager - Upgrade OkHttp to 3.14.9/4.10.0 due to medium CVEs
Removed unused OkHttp dependency
COMPX-11112: Queue Manager - Upgrade Bouncy Castle to 1.70 due to medium CVEs
n/a

Technical Service Bulletins

TSB 2023-641: InvalidClassException while editing queue configurations in YARN Queue Manager UI

For the latest update on this issue see the corresponding Knowledge article: TSB 2022-641: InvalidClassException while editing queue configurations in YARN Queue Manager UI.

Apache patch information

  • YARN-11490
  • YARN-11312
  • YARN-11463
  • YARN-11461
  • YARN-11063
  • YARN-11190
  • MAPREDUCE-7433
  • MAPREDUCE-7434
  • MAPREDUCE-7268
  • MAPREDUCE-7237
  • MAPREDUCE-7431