Cloudera Docs

Manually restoring Knox tables

Learn about the methods how you can manually restore Knox tables.

You need to manually restore the Knox tables within the Data Lake after a successful backup.

After a restore, you need to ensure that the restored metadata of the Data Lake reflect the desired end state, based on the purpose of the restore:

  • If you want to reflect the current state of the data just prior to the restore:
    • You must manually apply any SDX service changes that were in effect at the time of the restore.
    • You must manually apply any custom configuration changes to default configuration for SDX services in the Data Lake.
    • You must create and delete any applicable objects/policies that did not exist at the time of the backup.
    • You must validate access and performance characteristics.
  • If you want to reflect the state of the cluster at the time of the backup, for example to roll back an undesired Data Lake change:
    • You must manually delete any objects that are no longer covered by the Data Lake policy. Ensure that you created a backup first.
    • You must manually apply any custom configuration changes to the default configuration for SDX services in the Data Lake.
    • Validate access and performance characteristics.
You have the following options to restore the Knox tables from the backup:
  • Running the restore script that recreates the backup from the dump in the object store
  • Running the pg_restore command that restores the Knox tables from a local backup

Running the restore script

You can restore the previously backed up Knox tables using the restore script.

To run the restore script, you will need the values of the listed parameters included in the following files:

  • From the /srv/pillar/postgresql/disaster_recovery.sls file:
    • close_connections
    • object_storage_url
    • ranger_admin_group
  • From the /srv/pillar/postgresql/postgre.sls file:
    • remote_admin
    • remote_db_port
    • remote_db_url
  1. Access the Data Lake node that has the restore scripts, for example gateway-0 node for Enterprise Data Lake, and change the user to be able to run the restore script: 
    ssh -i [***PATH TO CLOUDBREAK PRIVATE KEY FILE***] cloudbreak@[***NODE IP ADDRESS***]
sudo su

    For more information, see Accessing a Data Lake cluster via SSH.

  2. Run the following command to restore the Knox tables: 
    PGPASSFILE=/opt/salt/postgresql/.pgpass /bin/bash /opt/salt/scripts/restore_db.sh -s [***OBJECT_STORE_URL***] -h [***REMOTE_DB_URL***] -p [***REMOTE_DB_PORT***] -u [***REMOTE_ADMIN***] -r [***RANGER_ADMIN_GROUP***] -d "knox_gateway"
After the Knox table restore is successfully using the script, you need to upload the Knox keys as described in Uploading Knox keys.

Running the pg_restore command

You can restore the previously backed up Knox tables using the pg_restore command.

To run the pg_restore command, you will need the values of the listed parameters included in the following files:

  • From the /srv/pillar/postgresql/postgre.sls file:
    • remote_admin_pw
    • remote_admin
    • remote_db_port
    • remote_db_url
Run the following command: 
pg_restore --host=[***REMOTE_DB_URL***] --port=[***REMOTE_DB_PORT***] --username=[***REMOTE_ADMIN***] --dbname="knox_gateway" --clean --if-exists --verbose /var/tmp/knox_gateway.dump
After the Knox table restore is successfully using the pg_restore command, you need to upload the Knox keys as described in Uploading Knox keys.

Uploading Knox keys

The exported Knox keys need to be uploaded in order to authenticate with the services after the backup is restored.

  • Note down the current key before overwriting it:
    export PGPASSWORD=[***SECRET_CM_DB_PW***]
psql -h [***CM_DB_HOST***] -U [***CM_USER***] -d [***CM_DB***] -t -A -c "SELECT value FROM configs WHERE attr = 'knox_token_mac_key';" > current_knox_token_mac_key.txt
  1. Use the following command to export the Knox token hash key from the Cloudera Manager database: 
    export PGPASSWORD=[***SECRET_KNOX_DB_PW***]

psql -h [***KNOX_DB_HOST***] -U [***KNOX_DB_USER***] -d [***KNOX_DB***] < knox1_data.sql
  2. Overwrite knox_token_max_key in CM DB using the following command:
    Update the current one in the DB to the one you created during the backup.
    psql -h [***CM_DB_HOST***] -U [***CM_USER***] -d [***CM_DB***] -c "UPDATE configs SET value = $BACKUP_VALUE where attr = 'knox_token_mac_key';"
  3. Restart the Knox service to propagate the updated key in the Knox's gateway-level credential store.