18. Validating Your Data

Verify that your data is intact by comparing the HDFS data directory tree with your previous directory tree.

Run the following command as the hadoop user, from the Hadoop home directory:
runas /user:hadoop "cmd /K cd %HDFS_HOME%\bin"
Run an lsr report on your upgraded system. Enter the following at the Hadoop command line:
hdfs fs -ls R / > fs-new-lsr-1.log
Compare the directory listing to the older HDP directories. All old directories, files and timestamps should match. There will also be some new entries in the HDP directory listing:
- /apps/hbase is used by HBase (new when upgrading from 1.3 to 2.2)
- /mapred/system/jobtracker should have a new timestamp
Run an fsck report on your upgraded system. From the Hadoop command line:
hdfs fsck / -blocks -locations -files > fsck-new-report-1.log
To check the validity of your current HDFS data, compare this fsck report to the report generated before the upgrade.