DocsHortonworks Data Platform

 

 1.2.7. Modify HDFS Configurations

You may need to modify your hdfs-site configuration and/or your core-site configuration.

  1. To check if you need to modify your hdfs-site configuration, on the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> get localhost <CLUSTER_NAME> hdfs-site

    If you see any of the following properties, you must delete them from your configuration.

    • dfs.nameservices

    • dfs.client.failover.proxy.provider.<NAMESERVICE_ID>

    • dfs.ha.namenodes.<NAMESERVICE_ID>

    • dfs.ha.fencing.methods

    • dfs.ha.automatic-failover.enabled

    • dfs.namenode.http-address.<NAMESERVICE_ID>.nn1

    • dfs.namenode.http-address.<NAMESERVICE_ID>.nn2

    • dfs.namenode.rpc-address.<NAMESERVICE_ID>.nn1

    • dfs.namenode.rpc-address.<NAMESERVICE_ID>.nn2

    • dfs.namenode.shared.edits.dir

    • dfs.journalnode.edits.dir

    • dfs.journalnode.http-address

    • dfs.journalnode.kerberos.internal.spnego.principal

    • dfs.journalnode.kerberos.principal

    • dfs.journalnode.keytab.file

      Where <NAMESERVICE_ID> is the NameService ID you created when you ran the Enable NameNode HA wizard.

  2. To delete these properties, execute the following for each property you found. On the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> delete localhost <CLUSTER_NAME> hdfs-site property_name

    Where you replace property_name with the name of each of the properties to be deleted.

  3. Verify that all of the properties have been deleted. On the Ambari Server host: /var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> get localhost <CLUSTER_NAME> hdfs-site

    None of the properties listed above should be present.

  4. To check if you need to modify your core-site configuration, on the Ambari Server host: /var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> get localhost <CLUSTER_NAME> core-site

  5. If you see the property ha.zookeeper.quorum, it must be deleted. On the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> delete localhost <CLUSTER_NAME> core-site ha.zookeeper.quorum

  6. If the property fs.defaultFS is set to the NameService ID, it must be reverted back to its non-HA value. For example:

    "fs.defaultFS":"hdfs://<name-service-id>" The property fs.defaultFS needs to be modified as it points to a NameService ID "fs.defaultFS":"hdfs://<nn01.mycompany.com>" The property fs.defaultFS does not need to be changed as it points to a specific NameNode, not to a NameService ID

  7. To revert the property fs.defaultFS to the NameNode host value, on the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> set localhost <CLUSTER_NAME> core-site fs.defaultFS hdfs://<NAMENODE_HOSTNAME>

  8. Verify that the core-site properties are now properly set. On the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u <AMBARI_USER> -p <AMBARI_PW> -port <AMBARI_PORT> get localhost <CLUSTER_NAME> core-site

    The property fs.defaultFS should be set to point to the NameNode host and the property ha.zookeeper.quorum should not be there.

Legal notices
loading table of contents...