Cloudera Docs

What's New in Apache Hadoop YARN and YARN Queue Manager

Learn about the new features of YARN and YARN Queue Manager in Cloudera Runtime 7.1.8

Apache Hadoop YARN

There are no new features for Apache Hadoop YARN in this release of Cloudera Runtime.

YARN Queue Manager

Dynamic Queue Scheduling

Dynamic Queue Scheduling is now generally available and can be used in production environments. This is the result of multiple changes, improvements, and new features such as Dynamic Configuration revalidation and execution logs.

For more information, see Dynamic Queue Scheduling.

Queue priority

Setting queue priorities is now supported by the YARN Queue Manager UI. By setting queue priorities you can ensure that applications can access cluster resources. This is especially important in the case of Hive LLAP, long-running applications, and applications that require large containers. For more information about the Dynamic Queue Scheduling feature, see Setting queue priorities.

Setting Maximum Parallel Application Limits

You can set the maximum number of applications limits for all queues, all users, and at the user level. The maximum parallel application limit is inherited from the “root” queue level and is lowered down in the queue hierarchy. The limit is checked in the queue hierarchy and the lowest value is applied as the limit.

For more information, see Setting Maximum Parallel Application.

Editing placement rules

Support to edit previously created placement rules was added.

For more information, see Editing placement rules.

Refresh queues option in Queue Manager UI

A Refresh button was added to the Overview tab in the YARN Queue Manager UI which provides the functionality to refresh the queues on demand.

Configuring the the capacity and max capacity of root queue in absolute mode

Support to configure memory/vcores and maximum memory/vcores for the root queue in absolute resource allocation mode is added. They can be set using the YARN Queue Manager UI.

For more information, see Configuring the resource capacity of root queue in absolute mode.

New YARN Queue Manager Overview Page
The new YARN Queue Manager Overview page has a new improved User Interface (UI) with the following new features:
  • Minimap: The Overview page now has a minimap of the queue structure. It shows the whole queue structure even if you zoom in to a specific part of it.
  • Refresh: You can click the Refresh icon for in-screen refresh of the page.
  • Zoom and Panning : You can use the mouse to zoom in and zoom out on the screen to view the queue structure. You can also drag the queue structure to see different parts of the structure.
  • Tool Tip: You can hover on the queue name for information like queue name and its queue path, queue status, and capacity. Previously, only the queue name and its path was displayed.