Problem: Oozie fails smoke tests in secured cluster.
Workaround:
Download the following files attached to https://issues.apache.org/jira/browse/AMBARI-2879:
check_oozie_status.sh
oozieSmoke.sh
Replace
/var/lib/ambari-agent/puppet/modules/hdp-nagios/files/check_oozie_status.sh
with the downloaded file.On the Nagios Server host machine, restart Nagios using the following command:
service nagios start
Replace
/var/lib/ambari-agent/puppet/modules/hdp-oozie/files/oozieSmoke.sh
with the downloaded file all the hosts in your cluster.Restart Oozie on the Oozie Server host machine using the following command:
sudo su -l $OOZIE_USER -c "cd $OOZIE_LOG_DIR/log; /usr/lib/oozie/bin/oozie-start.sh"
where:
$OOZIE_USER
is the Oozie Service user. For example,oozie
$OOZIE_LOG_DIR
is the directory where Oozie log files are stored (for example:/var/log/oozie
).
Problem: The
ambari-server
command displays invalid options for setting up Ganglia and Nagios.On the Ambari server host machine, when you execute the following command:
ambari-server
You see the following output:
Using python /usr/bin/python2.6 Usage: /usr/sbin/ambari-server {start|stop|restart|setup|upgrade|status|upgradestack|setup-ldap|setup-https|setup-ganglia_https|setup-nagios_https|encrypt-passwords} [options]
Workaround: The
setup-ganglia_https
andsetup-nagios_https
are not valid options.Use
setup-ganglia-https
andsetup-nagios-https
with theambari-server
command to set up Ganglia and Nagios.Problem:
ntpd
service warning might be displayed as part of host check at the bootstrap stage.Workaround: Verify that
ntpd
is running on all nodes. Execute the following command on all the nodes:For RHEL/CentOS:
service ntpd status
For SLES:
service ntp status
Problem: Selecting Use local software repository option causes Ambari to deploy
default
stack version. Thedefault
stack version for HDP v1.3.3 isHDP-1.3.3
Workaround: To install previous version of HDP with local repository option, complete the following instructions:
SSH into the Ambari Server host machine and execute the following commands:
cd /usr/lib/ambari-server/web/javascripts
rm app.js
gunzip app.js.gz
vi app.js
Change the value of
App.defaultLocalStackVersion
parameter inapp.js
file to the expected value of HDP release,For example, to install HDP 1.3.0, change the
App.defaultLocalStackVersion
parameter as shown below:App.defaultLocalStackVersion ='HDPLocal-1.3.0';
Execute the following command:
gzip app.js
Clear the browser cache and log in to Ambari Web.
Problem: Ganglia RRD database requires a large amount of disk space. Ganglia collects Hadoop and System metrics for the hosts in the cluster. These metrics are stored in the RRD database. Based on the number of Services you install, and the number of hosts in your clusters, the RRD database could become quite large.
Workaround: During cluster install, on the Customize Services page, select the Misc tab and set the base directory where RRD stores the collected metrics. Choose a directory location that has a minimum of 16 GB disk space available.
Workaround: You can also minimize the space used by Ganglia.
To reduce the Ganglia metrics collection granularity and reduce the overall disk space used by Ganglia, perform these steps after successfully completing your cluster install:
Download the following utility script configs.sh.
From Ambari, stop the Ganglia service and wait for it to stop completely.
Get the existing directory path for Ganglia RRD files (the "rrds" folder) using the
configs.sh
script:./configs.sh get $my.ambari.server $clustername global | grep "rrdcached_base_dir"
where,
$my.ambari.server
is the Ambari Server host and$clustername
is the name of the cluster.Log into the Ganglia Server host.
Backup the content of the
rrds
folder and then clean the folder.Edit the
gmetadLib.sh
filevi /var/lib/ambari-agent/puppet/modules/hdp-ganglia/files/gmetadLib.sh
Comment out the existing RRAs entry and enter the following:
RRAs "RRA:AVERAGE:0.5:1:244" "RRA:AVERAGE:0.5:24:244" "RRA:AVERAGE:0.5:168:244" "RRA:AVERAGE:0.5:672:244" "RRA:AVERAGE:0.5:5760:374
From Ambari, start the Ganglia service.
To confirm your change is applied, on the Ganglia Server host, you should see the line from above in the
gmetad.conf
file:more /etc/ganglia/hdp/gmetad.conf
Note You may need to wait for 5-10 minutes to see the metrics populate.