DocsHortonworks Data Platform

 

 2.3.6. Perform Backups

It is also strongly recommended that you perform backups and checkpoints before beginning upgrade. Checkpoint user metadata by backing-up Hive Metastore and Oozie databases. This step makes rollback and restoration of the original cluster state possible, if necessary.

  • If you have the Hive component installed, back up the Hive metastore database.

    These instructions are provided for your convenience. For the latest back up instructions, see your database documentation.

    Hive Metastore Database Backup and Restore

    Database Type

    Backup

    Restore

    MySQL

    mysqldump $dbname > $outputfilename.sqlsbr

    For example:

    mysqldump hive > /tmp/mydir/backup_hive.sql

    mysql $dbname < $inputfilename.sqlsbr

    For example:

    mysql hive < /tmp/mydir/backup_hive.sql

    Postgres

    sudo -u $username pg_dump $databasename > $outputfilename.sql sbr

    For example:

    sudo -u postgres pg_dump hive > /tmp/mydir/backup_hive.sql

    sudo -u $username psql $databasename < $inputfilename.sqlsbr

    For example:

    sudo -u postgres psql hive < /tmp/mydir/backup_hive.sql

    Oracle

    Connect to the Oracle database using sqlplus

    Export the database:

    exp username/password@database full=yes file=output_file.dmp mysql $dbname < $inputfilename.sqlsbr

    For example:

    mysql hive < /tmp/mydir/backup_hive.sql

    Import the database:

    imp username/password@database file=input_file.dmp

  • If you have the Oozie component installed, back up the Oozie metastore database.

    These instructions are provided for your convenience. For the latest back up instructions, see your database documentation.

    Oozie Database Backup and Restore

    Database Type

    Backup

    Restore

    MySQL

    mysqldump $dbname > $outputfilename.sql

    For example:

    mysqldump oozie > /tmp/mydir/backup_oozie.sql

    mysql $dbname < $inputfilename.sql

    For example:

    mysql oozie < /tmp/mydir/backup_oozie.sql

    Postgres

    sudo -u $username pg_dump $databasename > $outputfilename.sql

    For example:

    sudo -u

    postgres pg_dump oozie > /tmp/mydir/backup_oozie.sql

    sudo -u $username psql $databasename < $inputfilename.sql

    For example:

    sudo -u

    postgres psql oozie < /tmp/mydir/backup_oozie.sql

    Oracle

    Connect to the Oracle database using sqlplus

    Export the database:

    exp username/password@database full=yes file=output_file.dmp mysql $dbname < $inputfilename.sqlsbr

    For example:mysql oozie < /tmp/mydir/backup_oozie.sql

    Import the database:

    imp username/password@database file=input_file.dmp

  • If you have the Ranger component installed, back up the Ranger and Ranger_audit databases.

    These instructions are provided for your convenience. For the latest back up instructions, see your database documentation.

    Ranger Database Backup and Restore

    Database Type

    Backup

    Restore

    MySQL

    mysqldump $dbname > $outputfilename.sql

    For example:

    mysqldump ranger > /tmp/mydir/backup_ranger.sql

    mysql $dbname < $inputfilename.sql

    For example:

    mysql ranger < /tmp/mydir/backup_ranger.sql

    Oracle

    Connect to the Oracle database using sqlplus

    Export the database:

    exp username/password@database full=yes file=output_file.dmp mysql $dbname < $inputfilename.sqlsbr

    For example:mysql ranger < /tmp/mydir/backup_ranger.sql

    Import the database:

    imp username/password@database file=input_file.dmp

  • Verify that edit logs in ${dfs.namenode.name.dir}/current/edits* are empty.

    1. Run: hdfs oev -i ${dfs.namenode.name.dir}/current/edits_inprogress_* -o edits.out

    2. Verify edits.out file. It should only have OP_START_LOG_SEGMENT transaction. For example:

      <?xml version="1.0" encoding="UTF-8"?>
<EDITS>
<EDITS_VERSION>-56</EDITS_VERSION>
<RECORD>
<OPCODE>OP_START_LOG_SEGMENT</OPCODE>
<DATA>
<TXID>5749</TXID>
</DATA>
</RECORD>

    3. If edits.out has transactions other than OP_START_LOG_SEGMENT, run the following steps and then verify edit logs are empty.

      • Start the existing version NameNode.

      • Ensure there is a new FS image file.

      • Shut the NameNode down:

        hdfs dfsadmin –saveNamespace

  • Rename or delete any paths that are reserved in the new version of HDFS.

    When upgrading to a new version of HDFS, it is necessary to rename or delete any paths that are reserved in the new version of HDFS. If the NameNode encounters a reserved path during upgrade, it will print an error such as the following: 

    /.reserved is a reserved path and .snapshot is a reserved path 
component in this version of HDFS. 

Please rollback and delete or rename this path, or upgrade with the 
-renameReserved key-value pairs option to automatically rename these 
paths during upgrade.

    Specifying -upgrade -renameReserved optional key-value pairs causes the NameNode to automatically rename any reserved paths found during startup.

    For example, to rename all paths named .snapshot to .my-snapshot and change paths named .reserved to .my-reserved, specify -upgrade -renameReserved .snapshot=.my-snapshot,.reserved=.my-reserved.

    If no key-value pairs are specified with -renameReserved, the NameNode will then suffix reserved paths with:

    .<LAYOUT-VERSION>.UPGRADE_RENAMED

    For example: .snapshot.-51.UPGRADE_RENAMED.

    [Note]Note

    We recommend that you perform a -saveNamespace before renaming paths (running -saveNamespace appears in a previous step in this procedure). This is because a data inconsistency can result if an edit log operation refers to the destination of an automatically renamed file.

    Also note that running -renameReserved will rename all applicable existing files in the cluster. This may impact cluster applications.

  • Backup Hue. If you have the Hue component installed, back up Hue. If you are using the embedded SQLite database, you must perform a backup of the database before you upgrade Hue to prevent data loss.

    To make a backup copy of the database, stop Hue and simply do a "dump" and redirect the results to a file.

    /etc/init.d/hue stop

    su - hue

    mkdir ~/hue_backup

    cd /var/lib/hue

    sqlite3 desktop.db .dump > ~/hue_backup/desktop.bak

    For other databases, follow your vendor-specific instructions to create a backup.

  • Backup the Hue Configuration.

    cp -RL /etc/hue/conf ~/hue_backup

Proceed to Performing the Upgrade.

Legal notices
loading table of contents...