Cloudera Docs

Known Issues in YARN, YARN Queue Manager and MapReduce

Known issues and technical limitations for YARN and YARN Queue Manager are addressed in Cloudera Runtime 7.3.2, its service packs, and cumulative hotfixes.

Known issues identified in Cloudera Runtime 7.3.2

There are no new known issues identified in this release.

Known issues identified before Cloudera Runtime 7.3.2

Known issues identified before Cloudera Runtime 7.3.2 include only unresolved issues from previous releases that continue to affect the Cloudera Runtime 7.3.2 base release.

CDPD-75652: Reverse DNS lookup fails for YARN but works for HDFS
7.3.2, 7.3.1 and its SPs and CHFs
When submitting a YARN application from a host without the correct DNS configuration (reverse DNS does not work for the YARN ResourceManager's host) Kerberos principal validation error, such as Server has invalid Kerberos principal. This is because the reverse DNS configuration does not work for YARN ResourceManager's host.
Add the following to YARN Service Advanced Configuration Snippet (Safety Valve) for yarn-site.xml: 
<property> 
<name>yarn.resourcemanager.principal.pattern</name> 
<value>*</value> 
</property>
COMPX-14682: Fix health check after Queue Manager restart
7.3.2, 7.3.1 and its SPs and CHFs, 7.1.9 SP1 and its CHFs
In some cases, after the QM webapp and config-service are restarted Cloudera Manager reports a healthy status after some time. However, the configuration service might not be ready and usable yet when this status is reported. Therefore, QM (webapp) is not ready and usable yet.
Wait for a minute or two after restart and then try QM.
COMPX-14820: Delete Queue and its Children throws "Queue capacity was reduced to zero, but failed to delete queue."
7.3.2, 7.3.1 and its SPs and CHFs, 7.1.9 SP1 and its CHFs
When trying to perform the operation "Delete Queue and its Children" on a queue that has one or more siblings, the operation fails as YARN has some constraints. If the queue performing the operation "Delete Queue and its Children" is a leaf node, then the operations succeeds.
None.
A fresh install of 7.3.1 or its SPs or CHFs does not allow user to bypass the Setup Database screen for YARN Queue Manager
7.3.2, 7.3.1 and its SPs and CHFs
YARN Queue Manager in Cloudera Base on premises 7.3.1 does not require you to install a PostGres database, therefore users should not see the Setup Database screen and should be able to skip the Setup Database screen. With this known issue, users who are conducting a fresh install of 7.3.1 or its SPs or CHFs are not able to bypass the Setup Database screen as expected.
  1. When conducting a fresh install of YARN Queue Manager in 7.3.1 or its SPs or CHFs, you must ensure that you have both Cloudera and Cloudera Manager upgraded to 7.3.1.
  2. When you reach the Setup Database screen in the Cloudera Manager installation wizard for Queue Manager, enter any dummy values for the following fields:
    1. Database name: configstore
    2. Database Username: dbuser
    3. Database Password: dbpassword
    4. Database Hostname: localhost
    YARN Queue Manager will not connect to PostGres with the above details and will fall back to the embedded database.
  3. Run the following script command in a browser console to enable the Continue button:

    document.querySelector('.btn.next').removeAttribute('disabled');

  4. Click Continue and proceed with the YARN Queue Manager installation.
  5. After installation is complete, SSH into the host that has Queue Manager installed, and run this command: sed -i 's/migrationCompleted=true/migrationCompleted=false/' /var/lib/hadoop-yarn/migration.properties
  6. Restart YARN Queue Manager.
20202 Database migration after enabling opt-in migration
7.3.2, 7.3.1 and its SPs and CHFs
When migrating from an H2 database to a PostgreSQL database in YARN Queue Manager after installation or upgrade, you might encounter an issue only when you have followed the following specific scenario:
  • New install or upgrade to Cloudera 7.1.9, forcing migration from H2 to PostgreSQL database.
  • Upgrade to Cloudera 7.1.9 CHF2, moving back to H2 database.
  • Upgrade to Cloudera 7.1.9 SP1 with valid PostgreSQL connection details in Queue Manager configurations.
To avoid any issues during the upgrade to version Cloudera 7.1.9 SP1, ensure that PostgreSQL connection details are removed from the YARN database configuration if you prefer to continue using the H2 database.
Queue Manager does not open on using a custom user with a default Kerberos principal
7.3.2, 7.3.1 and its SPs and CHFs
If a custom user is used with the default Kerberos principal, the Queue Manager web UI displays an HTTP ERROR 400 error.
Ensure that the Queue Manager process_username property matches the YARN process_username property.
Third-party applications do not launch if MapReduce framework path is not included in the client configuration
7.3.2, 7.3.1 and its SPs and CHFs
MapReduce application framework is loaded from HDFS instead of being present on the NodeManagers. By default the mapreduce.application.framework.path property is set to the appropriate value, but third-party applications with their own configurations does not launch.
Set the mapreduce.application.framework.path property to the appropriate configuration for third-party applications.
JobHistory URL mismatch after server relocation
7.3.2, 7.3.1 and its SPs and CHFs
After moving the JobHistory Server to a new host, the URLs listed for the JobHistory Server on the ResourceManager web UI still point to the old JobHistory Server. This affects existing jobs only. New jobs started after the move are not affected.
For any existing jobs that have the incorrect JobHistory Server URL, there is no option other than to allow the jobs to roll off the history over time. For new jobs, make sure that all clients have the updated the mapred-site.xml file that references the correct JobHistory Server.
YARN cannot start if Kerberos principal name is changed
7.3.2, 7.3.1 and its SPs and CHFs
If the Kerberos principal name is changed in Cloudera Manager after launch, YARN does not start. In such cases, the keytabs can be correctly generated but YARN cannot access ZooKeeper with the new Kerberos principal name and old ACLs.
There are two possible workarounds:
  • Delete the znode and restart the YARN service.
  • Use the reset ZK ACLs command. This also sets the znodes below /rmstore/ZKRMStateRoot to world:anyone:cdrwa which is less secure.
YARN Resource Manager UI does not display logs
7.3.2, 7.3.1 and its SPs and CHFs
When the parameter mapreduce.cluster.acl.enabled is set to true, the Yarn RM UI does display and logs and the Logs are not available message is displayed.
Set mapreduce.cluster.acl.enabled to false.