Upgrading HDP Manually
Also available as:
PDF
loading table of contents...

Verify HDFS Filesystem Health

Analyze if the filesystem is healthy.

  1. Run the fsck command on namenode as $HDFS_USER:

    su - $HDFS_USER

    hdfs fsck / -files -blocks -locations > dfs-new-fsck-1.log

  2. Run hdfs namespace and report.

    1. List directories.

      su -l <HDFS_USER>

      hdfs dfs -ls -R / > dfs-new-lsr-1.log

    2. Run report command to create a list of DataNodes in the cluster.

      su -l <HDFS_USER>

      hdfs dfsadmin -report > dfs-new-report-1.log

    3. Compare the namespace report before the upgrade and after the upgrade. Verify that user files exist after upgrade.

      su - $HDFS_USER

      dfs-old-fsck-1.log < -- > dfs-new-fsck-1.log dfs-old-lsr-1.log < -- > dfs-new-lsr-1.log

      [Note]Note

      You must do this comparison manually to catch all errors.

    4. From the Namenode WebUI, see if all DataNodes are up and running.

      http://<namenode>:50070