Modify HDFS Configurations
You may need to modify your hdfs-site
configuration and/or your
core-site
configuration.
Prerequisites
Check whether you need to modify your hdfs-site
configuration, by
executing the following command on the Ambari Server host:
/var/lib/ambari-server/resources/scripts/configs.sh -u
<AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> get localhost
<CLUSTER_NAME> hdfs-site
If you see any of the following properties, you must delete them from your configuration.
dfs.nameservices
dfs.client.failover.proxy.provider.<NAMESERVICE_ID>
dfs.ha.namenodes.<NAMESERVICE_ID>
dfs.ha.fencing.methods
dfs.ha.automatic-failover.enabled
dfs.namenode.http-address.<NAMESERVICE_ID>.nn1
dfs.namenode.http-address.<NAMESERVICE_ID>.nn2
dfs.namenode.rpc-address.<NAMESERVICE_ID>.nn1
dfs.namenode.rpc-address.<NAMESERVICE_ID>.nn2
dfs.namenode.shared.edits.dir
dfs.journalnode.edits.dir
dfs.journalnode.http-address
dfs.journalnode.kerberos.internal.spnego.principal
dfs.journalnode.kerberos.principal
dfs.journalnode.keytab.file
Where
<NAMESERVICE_ID>
is the NameService ID you created when you ran the Enable NameNode HA wizard.
To modify your hdfs-site
configuration:
Steps
On the Ambari Server host, execute the following for each property you found:
/var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> delete localhost <CLUSTER_NAME> hdfs-site property_name
Replace
property_name
with the name of each of the properties to be deleted.Verify that all of the properties have been deleted:
/var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> get localhost <CLUSTER_NAME> hdfs-site
None of the properties listed above should be present.
Determine whether you must modify your
core-site
configuration:/var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> get localhost <CLUSTER_NAME> core-site
If you see the property
ha.zookeeper.quorum
, delete it:/var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> delete localhost <CLUSTER_NAME> core-site ha.zookeeper.quorum
If the property
fs.defaultFS
is set to the NameService ID, revert it to its non-HA value:"fs.defaultFS":"hdfs://<name-service-id>" The property fs.defaultFS needs to be modified as it points to a NameService ID "fs.defaultFS":"hdfs://<nn01.mycompany.com>"
You need not change the property
fs.defaultFS
, because it points to a specific NameNode, not to a NameService ID.Revert the property
fs.defaultFS
to the NameNode host value:/var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> set localhost <CLUSTER_NAME> core-site fs.defaultFS hdfs://<NAMENODE_HOSTNAME>
Verify that the
core-site
properties are now properly set:/var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> get localhost <CLUSTER_NAME> core-site
The property
fs.defaultFS
should be the NameNode host and the propertyha.zookeeper.quorum
should not appear.