What's New in Apache Hadoop YARN and YARN Queue Manager
Learn about the new features of YARN and YARN Queue Manager in Cloudera Runtime 7.1.8
Apache Hadoop YARN
There are no new features for Apache Hadoop YARN in this release of Cloudera Runtime.
YARN Queue Manager
- Dynamic Queue Scheduling
Dynamic Queue Scheduling is now generally available and can be used in production environments. This is the result of multiple changes, improvements, and new features such as Dynamic Configuration revalidation and execution logs.
For more information, see Dynamic Queue Scheduling.
- Queue priority
Setting queue priorities is now supported by the YARN Queue Manager UI. By setting queue priorities you can ensure that applications can access cluster resources. This is especially important in the case of Hive LLAP, long-running applications, and applications that require large containers. For more information about the Dynamic Queue Scheduling feature, see Setting queue priorities.
- Setting Maximum Parallel Application Limits
You can set the maximum number of applications limits for all queues, all users, and at the user level. The maximum parallel application limit is inherited from the “root” queue level and is lowered down in the queue hierarchy. The limit is checked in the queue hierarchy and the lowest value is applied as the limit.
For more information, see Setting Maximum Parallel Application.
- Editing placement rules
Support to edit previously created placement rules was added.
For more information, see Editing placement rules.
- Refresh queues option in Queue Manager UI
A Refresh button was added to the Overview tab in the YARN Queue Manager UI which provides the functionality to refresh the queues on demand.
- Configuring the the capacity and max capacity of root queue in absolute mode
Support to configure memory/vcores and maximum memory/vcores for the root queue in absolute resource allocation mode is added. They can be set using the YARN Queue Manager UI.
For more information, see Configuring the resource capacity of root queue in absolute mode.
- New YARN Queue Manager Overview Page
The new YARN Queue Manager Overview page has a new improved User Interface (UI) with the following new features:
- Minimap: The Overview page now has a minimap of the queue structure. It shows the whole queue structure even if you zoom in to a specific part of it.
- Refresh: You can click the Refresh icon for in-screen refresh of the page.
- Zoom and Panning : You can use the mouse to zoom in and zoom out on the screen to view the queue structure. You can also drag the queue structure to see different parts of the structure.
- Tool Tip: You can hover on the queue name for information like queue name and its queue path, queue status, and capacity. Previously, only the queue name and its path was displayed.