Cloudera Docs

Step 3: Recover open files in corrupt state

This section provides a scenario and the root cause of the problem.

Steps to Reproduce

  1. Open a file and start writing some content.
  2. Perform a rolling upgrade. It restarts the DNs and hence breaks the existing pipeline.
  3. Continue with writing and close the file.
  4. Rollback the upgrade to the older version.

Problem

The problem is same as Step 2. The difference is you should not persist metadata until the file is complete.