Checkpoint HDFS

You must perform additional steps before upgrading your HDP cluster if you have configured NameNode HA to create a checkpoint for HDFS.

If you are configured for NameNode HA, then locate the Active NameNode from Ambari Web > Services > HDFS in the Summary area.
Check the NameNode directory to ensure that there is no snapshot of any prior HDFS upgrade. Specifically, using Ambari Web, browse to Services > HDFS > Configs, and examine the dfs.namenode.name.dir in the NameNode Directories property. Make sure that only a /current directory exists and remove /previous directory on the NameNode host.
Create the following log and other files. The commands below back up additional state from the file system in addition to what fsImage contains. This information can be used to validate the filesystem state after the upgrade by comparing the output before or after the upgrade. This can help but is not required for troubleshooting if issues arise during the upgrade.

note
The fsck or ls command may take a significant amount of time if the number of blocks or files in HDFS is in ten to a hundred million range.

Authenticate as the HDFS user su -l [HDFS_USER]
note
If the cluster is Kerberized, then you must kinit as the service principal. If the cluster is not Kerberized, then you must run as a cluster administrator user or log in as the service user.
Run the following (where [HDFS_USER] is the HDFS Service user, for example, hdfs):
- Run fsck with the following flags and send the results to a log. The resulting file contains a complete block map of the file system. You use this log later to confirm the upgrade.
  hdfs fsck / -files -blocks -locations > dfs-old-fsck-1.log
- Create a list of all the DataNodes in the cluster.
  hdfs dfsadmin -report > dfs-old-report-1.log
- Capture the complete namespace of the file system. The following command does a recursive listing of the root file system:
  hdfs dfs -ls -R / > dfs-old-lsr-1.log
- Take a backup of the HDFS data to your local file system or the backup instance of your HDFS.
Save the namespace. As the HDFS user, " su -l [HDFS_USER] ", you must put the cluster in Safe Mode.
- hdfs dfsadmin -safemode enter
- hdfs dfsadmin -saveNamespace
note
In a highly-available NameNode configuration, the command hdfs dfsadmin -saveNamespace sets a checkpoint in the first NameNode specified in the configuration, in dfs.ha.namenodes.[nameserviceID]. You can also use the dfsadmin -fs .
Copy the checkpoint files located in $[dfs.namenode.name.dir]/current into a backup directory.
Store the layout version for the NameNode located at $[dfs.namenode.name.dir]/current/VERSION into a backup directory where $[dfs.namenode.name.dir] is the value of the config parameter NameNode directories.
This file will be used later to verify that the layout version is upgraded. As the HDFS user, su -l [HDFS_USER], take the NameNode out of Safe Mode. hdfs dfsadmin -safemode leave