13. Validating Your Data

Verify that your data is intact by comparing the HDFS data directory tree wtih the HDP 1.3 tree.

  1. Run an lsr report on your upgraded system. Execute the following command from the Hadoop command line:

    hadoop fs -lsr / > dfs-new-lsr-1.log
                    
  2. Compare the directory listing to the old directory listing from HDP 1.3. All old directories, files and timestamps should match. There will be some new entries in the HDP 2.0 directory listing:

    • /apps/hbase is only in HDP 2.0 and is used by HBase

    • /mapred/system/jobtracker will have a new timestamp

  3. Run a fsck report on your upgraded system. Execute the following command from the Hadoop Command Line:

    hadoop fsck / -blocks -locations -files fsck-new-report-1.log
  4. Compare this fsck report to the prior to upgrade report to check the validity of your current HDFS data.


loading table of contents...