Solution
Solution for the use case.
Recovery Steps
You can get the list of corrupt blocks using hdfs fsck /. After identifying the corrupt blocks, recover the corrupt blocks:
- The Namenode will be in safemode at this point, We can force exit out of safemode to perform the following operations hdfs dfsadmin -safemode forceExit.
- At this point the Namenode is out of safemode. You must copy the corrupt file data to a backup file hdfs dfs -cp <corrupt_file> <backup_file>.
- Truncate the corrupt file to 0 size hdfs dfs -truncate 0 <corrupt_file>.
- Append the backup file to the original corrupt file hdfs dfs -cat <backup_file> | hdfs dfs -appendToFile - <corrupt_file>.