Verify that your data is intact by comparing the HDFS data directory tree with the HDP 1.3 or HDP 2.0 tree.
Run the following command to open a command prompt with the Hadoop user in the Hadoop Home directory:
runas /user:hadoop "cmd /K cd %HADOOP_HOME%\bin"
Run an
lsr
report on your upgraded system. Execute the following command from the Hadoop command line:hadoop fs -lsr / > dfs-new-lsr-1.log
Compare the directory listing to the older HDP directories. All old directories, files and timestamps should match. There will be some new entries in the HDP directory listing:
/apps/hbase
is only in HDP and is used by HBase (new when upgrading from 1.3 to 2.1)/mapred/system/jobtracker
will have a new timestamp
Run a
fsck
report on your upgraded system. Execute the following command from the Hadoop Command Line:hadoop fsck / -blocks -locations -files fsck-new-report-1.log
Compare this fsck report to the prior to upgrade report to check the validity of your current HDFS data.