6.5. Known Issues for Oozie

  • Problem: Oozie fails smoke tests in secured cluster.

    Workaround:

    1. Download the following files attached to, https://issues.apache.org/jira/browse/AMBARI-2879:

      • check_oozie_status.sh

      • oozieSmoke.sh

    2. Replace /var/lib/ambari-agent/puppet/modules/hdp-nagios/files/check_oozie_status.sh with the downloaded file.

    3. On the Nagios Server host machine, restart Nagios using the following command:

      service nagios start

    4. Replace /var/lib/ambari-agent/puppet/modules/hdp-oozie/files/oozieSmoke.sh with the downloaded file all the hosts in your cluster.

    5. Restart Oozie on the Oozie Server host machine using the following command:

       sudo su -l $OOZIE_USER -c "cd $OOZIE_LOG_DIR/log; /usr/lib/oozie/bin/oozie-start.sh" 

      where:

      • $OOZIE_USER is the Oozie Service user. For example, oozie

      • $OOZIE_LOG_DIR is the directory where Oozie log files are stored (for example: /var/log/oozie).

  • Problem: TestBundleJobsFilter test fails on RHEL v6.3, Oracle v6.3, and SUSE clusters with PostgreSQL.

    This issue is caused due to the strict typing of PostgreSQL which restricts the auto casting of string integer to an integer. The issue is reported when string representation of integer values is substituted into a query for PostgreSQL on the JPA layer.

  • Problem: Delegation Token renewal exception in JobTracker logs.

    The following exception is reported in the JobTracker log file when executing a long running job on Oozie in secure mode:

    ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:jt/hor1n22.gq1.ygridcore.net@HORTON.YGRIDCORE.NET cause:org.apache.hadoop.security.AccessContr
    olException: org.apache.hadoop.security.AccessControlException: Client mapred tries to renew a token with renewer specified as jt
    2013-04-25 15:09:41,543 ERROR org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: Exception renewing tokenIdent: 00 06 68 72 74 5f 71 61 02 6a 74 34 6f 6f 7a 69 65 2f 68 6f 72 31 6e 32 34 2e 67 71 31 2e 79 67 72 69 64 63 6f 72 65 2e 6e 65 74 40 48 4f 52 54 4f 4e 2e 59 47 52 49 44 43 4f 52 45 2e 4e 45 54 8a 01 3e 41 b9 67 b8 8a 01 3e 65 c5 eb b8 8f 88 8f 9c, Kind: HDFS_DELEGATION_TOKEN, Service: 68.142.244.41:8020. Not rescheduled
    org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Client mapred tries to renew a token with renewer specified as jt
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
            at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
            at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
            at org.apache.hadoop.hdfs.DFSClient$Renewer.renew(DFSClient.java:678)
            at org.apache.hadoop.security.token.Token.renew(Token.java:309)
            at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:221)
            at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:217)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:396)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1195)
            at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask.run(DelegationTokenRenewal.java:216)
            at java.util.TimerThread.mainLoop(Timer.java:512)
            at java.util.TimerThread.run(Timer.java:462)
    Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.security.AccessControlException: Client mapred tries to renew a token with renewer specified as jt
            at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:267)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renewDelegationToken(FSNamesystem.java:6280)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.renewDelegationToken(NameNode.java:652)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
            at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1405)
            at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1401)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:396)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1195)
            at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1399)
    
            at org.apache.hadoop.ipc.Client.call(Client.java:1118)
            at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
            at $Proxy7.renewDelegationToken(Unknown Source)
            at org.apache.hadoop.hdfs.DFSClient$Renewer.renew(DFSClient.java:676)
            ... 9 more

    Workaround: Any new job job on secure cluster that runs longer than the validity of the Kerberos ticket (typically 24 hours) will fail as the delegation token will not be renewed.


loading table of contents...