Perform Backups

It is also strongly recommended that you perform backups and checkpoints before beginning upgrade. Checkpoint user metadata by backing-up Hive Metastore and Oozie databases. This step makes rollback and restoration of the original cluster state possible, if necessary.

If you have the Hive component installed, back up the Hive metastore database.

These instructions are provided for your convenience. For the latest back up instructions, see your database documentation.

Hive Metastore Database Backup and Restore

Database Type	Backup	Restore
MySQL	mysqldump $dbname > $outputfilename.sqlsbr For example: mysqldump hive > /tmp/mydir/backup_hive.sql	mysql $dbname < $inputfilename.sqlsbr For example: mysql hive < /tmp/mydir/backup_hive.sql
Postgres	sudo -u $username pg_dump $databasename > $outputfilename.sql sbr For example: sudo -u postgres pg_dump hive > /tmp/mydir/backup_hive.sql	sudo -u $username psql $databasename < $inputfilename.sqlsbr For example: sudo -u postgres psql hive < /tmp/mydir/backup_hive.sql
Oracle	Connect to the Oracle database using sqlplus Export the database: exp username/password@database full=yes file=output_file.dmp mysql $dbname < $inputfilename.sqlsbr For example: mysql hive < /tmp/mydir/backup_hive.sql	Import the database: imp username/password@database file=input_file.dmp

Database Type

Backup

Restore

MySQL

mysqldump $dbname > $outputfilename.sqlsbr

For example:

mysqldump hive > /tmp/mydir/backup_hive.sql

mysql $dbname < $inputfilename.sqlsbr

For example:

mysql hive < /tmp/mydir/backup_hive.sql

Postgres

sudo -u $username pg_dump $databasename > $outputfilename.sql sbr

For example:

sudo -u postgres pg_dump hive > /tmp/mydir/backup_hive.sql

sudo -u $username psql $databasename < $inputfilename.sqlsbr

For example:

sudo -u postgres psql hive < /tmp/mydir/backup_hive.sql

Oracle

Connect to the Oracle database using sqlplus

Export the database:

exp username/password@database full=yes file=output_file.dmp mysql $dbname < $inputfilename.sqlsbr

For example:

mysql hive < /tmp/mydir/backup_hive.sql

Import the database:

imp username/password@database file=input_file.dmp

If you have the Oozie component installed, back up the Oozie metastore database.

These instructions are provided for your convenience. For the latest back up instructions, see your database documentation.

Oozie Database Backup and Restore

Database Type	Backup	Restore
MySQL	mysqldump $dbname > $outputfilename.sql For example: mysqldump oozie > /tmp/mydir/backup_oozie.sql	mysql $dbname < $inputfilename.sql For example: mysql oozie < /tmp/mydir/backup_oozie.sql
Postgres	sudo -u $username pg_dump $databasename > $outputfilename.sql For example: sudo -u postgres pg_dump oozie > /tmp/mydir/backup_oozie.sql	sudo -u $username psql $databasename < $inputfilename.sql For example: sudo -u postgres psql oozie < /tmp/mydir/backup_oozie.sql
Oracle	Connect to the Oracle database using sqlplus Export the database: exp username/password@database full=yes file=output_file.dmp mysql $dbname < $inputfilename.sqlsbr For example:mysql oozie < /tmp/mydir/backup_oozie.sql	Import the database: imp username/password@database file=input_file.dmp

Database Type

Backup

Restore

MySQL

mysqldump $dbname > $outputfilename.sql

For example:

mysqldump oozie > /tmp/mydir/backup_oozie.sql

mysql $dbname < $inputfilename.sql

For example:

mysql oozie < /tmp/mydir/backup_oozie.sql

Postgres

sudo -u $username pg_dump $databasename > $outputfilename.sql

For example:

sudo -u

postgres pg_dump oozie > /tmp/mydir/backup_oozie.sql

sudo -u $username psql $databasename < $inputfilename.sql

For example:

sudo -u

postgres psql oozie < /tmp/mydir/backup_oozie.sql

Oracle

Connect to the Oracle database using sqlplus

Export the database:

exp username/password@database full=yes file=output_file.dmp mysql $dbname < $inputfilename.sqlsbr

For example:mysql oozie < /tmp/mydir/backup_oozie.sql

Import the database:

imp username/password@database file=input_file.dmp

If you have the Ranger component installed, back up the Ranger and Ranger_audit databases.

These instructions are provided for your convenience. For the latest back up instructions, see your database documentation.

Ranger Database Backup and Restore

Database Type	Backup	Restore
MySQL	mysqldump $dbname > $outputfilename.sql For example: mysqldump ranger > /tmp/mydir/backup_ranger.sql	mysql $dbname < $inputfilename.sql For example: mysql ranger < /tmp/mydir/backup_ranger.sql
Oracle	Connect to the Oracle database using sqlplus Export the database: exp username/password@database full=yes file=output_file.dmp mysql $dbname < $inputfilename.sqlsbr For example:mysql ranger < /tmp/mydir/backup_ranger.sql	Import the database: imp username/password@database file=input_file.dmp

Database Type

Backup

Restore

MySQL

mysqldump $dbname > $outputfilename.sql

For example:

mysqldump ranger > /tmp/mydir/backup_ranger.sql

mysql $dbname < $inputfilename.sql

For example:

mysql ranger < /tmp/mydir/backup_ranger.sql

Oracle

Connect to the Oracle database using sqlplus

Export the database:

exp username/password@database full=yes file=output_file.dmp mysql $dbname < $inputfilename.sqlsbr

For example:mysql ranger < /tmp/mydir/backup_ranger.sql

Import the database:

imp username/password@database file=input_file.dmp

Verify that edit logs in ${dfs.namenode.name.dir}/current/edits* are empty.
1. Run: hdfs oev -i ${dfs.namenode.name.dir}/current/edits_inprogress_* -o edits.out
2. Verify edits.out file. It should only have OP_START_LOG_SEGMENT transaction. For example:
```
<?xml version="1.0" encoding="UTF-8"?>
<EDITS>
<EDITS_VERSION>-56</EDITS_VERSION>
<RECORD>
<OPCODE>OP_START_LOG_SEGMENT</OPCODE>
<DATA>
<TXID>5749</TXID>
</DATA>
</RECORD>
```
3. If edits.out has transactions other than OP_START_LOG_SEGMENT, run the following steps and then verify edit logs are empty.
  - Start the existing version NameNode.
  - Ensure there is a new FS image file.
  - Shut the NameNode down:
    hdfs dfsadmin –saveNamespace

Rename or delete any paths that are reserved in the new version of HDFS.

When upgrading to a new version of HDFS, it is necessary to rename or delete any paths that are reserved in the new version of HDFS. If the NameNode encounters a reserved path during upgrade, it will print an error such as the following:

/.reserved is a reserved path and .snapshot is a reserved path 
component in this version of HDFS. 

Please rollback and delete or rename this path, or upgrade with the 
-renameReserved key-value pairs option to automatically rename these 
paths during upgrade.

Specifying -upgrade -renameReserved optional key-value pairs causes the NameNode to automatically rename any reserved paths found during startup.

For example, to rename all paths named .snapshot to .my-snapshot and change paths named .reserved to .my-reserved, specify -upgrade -renameReserved .snapshot=.my-snapshot,.reserved=.my-reserved.

If no key-value pairs are specified with -renameReserved, the NameNode will then suffix reserved paths with:

.<LAYOUT-VERSION>.UPGRADE_RENAMED

For example: .snapshot.-51.UPGRADE_RENAMED.

Note

	Note
We recommend that you perform a `-saveNamespace` before renaming paths (running `-saveNamespace` appears in a previous step in this procedure). This is because a data inconsistency can result if an edit log operation refers to the destination of an automatically renamed file. Also note that running `-renameReserved` will rename all applicable existing files in the cluster. This may impact cluster applications.

We recommend that you perform a -saveNamespace before renaming paths (running -saveNamespace appears in a previous step in this procedure). This is because a data inconsistency can result if an edit log operation refers to the destination of an automatically renamed file.

Also note that running -renameReserved will rename all applicable existing files in the cluster. This may impact cluster applications.

Backup Hue. If you have the Hue component installed, back up Hue. If you are using the embedded SQLite database, you must perform a backup of the database before you upgrade Hue to prevent data loss.
To make a backup copy of the database, stop Hue and simply do a "dump" and redirect the results to a file.
/etc/init.d/hue stop
su - hue
mkdir ~/hue_backup
cd /var/lib/hue
sqlite3 desktop.db .dump > ~/hue_backup/desktop.bak
For other databases, follow your vendor-specific instructions to create a backup.
Backup the Hue Configuration.
cp -RL /etc/hue/conf ~/hue_backup

Proceed to Performing the Upgrade.

​Perform Backups

Perform Backups