Monitoring Hue and Data Visualization restoration

The restore process is designed to be an idempotent process, it can be restarted as many times as you want. In case the environment is activated and healthy, the restore operation can be run multiple times to restore the Virtual Warehouse and Data Visualization objects.

Hue automatic restoration

For every restore operation, the Hue database restore will run. This operation will overwrite the Hue database contents. Automatic restoration of Hue loads the saved query and query history to the new cluster. In case the restore operation is called multiple times, the Hue database restore job will be run. This job will overwrite the current query history and saved queries with the contents of the backup.

Monitoring Hue restoration

The restoration starts a job to load the database dump file, but does not wait for the job to complete. If you have a large database, the job can take up to an hour to complete. Ensure you allow enough time for the job to succeed. If the job does not succeed, troubleshoot Hue restoration.

To monitor Hue restoration, log into the cluster and monitor the job status under the database catalog namespace.

$ kubectl get jobs -n <database catalog id>

The output that shows the hue-restore job looks something like this:

$ kubectl get jobs -n warehouse-1692037411-96hk
     NAME                                              COMPLETIONS   DURATION   AGE
     hue-restore-ede2b8bd-1d53-4d23-a0f9-87d8ec658f74   1/1           11s        113s
     hue-query-processor-db-create-job                 1/1           8s         42h

Data Visualization automatic restoration

If a Data Visualization object is not present on the cluster, but the backup file contains it, it will be restored to the cluster. In case such an entity is already deployed, no changes or configuration updates will take place.

Automatic restoration of Data Visualization loads the dashboards, tables, and connections to the new applications. Make sure to wait for the job to finish before destroying the cluster.

To monitor restoration of Data Visualization, you can log into the cluster and see the job status under the viz namespace using the following command.

$ kubectl get jobs -n <data visualization id>

The output will be similar to this, the viz-restore job shows the status.

$ kubectl get jobs -n viz-1692216942-fc2g
     NAME                                              COMPLETIONS   DURATION   AGE
     viz-restore-d874515a-be7e-4902-ac75-269c14f9580c   1/1           3m3s       10m
     viz-webapp-vizdb-create-job                        1/1           57s        99m

The job logs contain the upload path where the backup file has been downloaded from.

Automatic restoration of Data Visualization

Automatic backup and restore for Data Visualization extracts the dashboards, tables and connections. Make sure to wait for the job to finish before destroying the cluster. In the event of a restoration failure, try manually restoring Data Visualization.

Monitoring Data Visualization restoration

To monitor the restoration of Data Visualization, you can log into the cluster and see the job status under the viz namespace using the following command.

$ kubectl get jobs -n <data visualization id>

The output looks something like this:

$ kubectl get jobs -n viz-1692216942-fc2g
  NAME                                              COMPLETIONS   DURATION   AGE
  viz-restore-d874515a-be7e-4902-ac75-269c14f9580c   1/1           3m3s       10m
  viz-webapp-vizdb-create-job                       1/1           57s        99m