Manually restoring Hue from a smaller than 6GB backup

You can manually restore the Hue database instance that you backed up. You can use this procedure for manually restoring Hue when your Hue backup is smaller than 6GB.

In the manual backup of Hue, you followed steps to dump the entire Hue database. In the following procedure, you move the Hue backup file from the dump to the new CDW environment.
  • Check that the size of your Hue backup is smaller than 6GB.

    If your Hue backup is 6GB or larger, go to “Manually restoring Hue 6GB or larger backup”.

  • Do not open the Hue web interface prior to completing the steps below.
  • During the manual or automatic Hue database restore operation it is critical to block any traffic to the running Hue services. If you cannot bring down the cluster, use the recommended workaround to disable end user access to the cluster endpoints. Failing to do so results in errors in addition to existing key constraints and other issues.
  1. (Optional) Edit the Hue statefulset Hue backup files that are larger than 3Gb and require a significant amount of memory to restore the database contents.
    Sufficient memory is not available for the Hue pod by default. To successfully load the data, you must increase the Hue pod memory limit.
  2. (Optional) Check file sizes and configure the memory accordingly.
    kubectl edit statefulset huebackend -n <new Virtual Warehouse ID>
  3. Set the hue container memory limit to 24Gb to provide leeway for the load command.
    name: hue 
      memory: 24G
      cpu: "1"
      memory: 8192M
  4. Copy the Hue backup data to the hue pod on the new Virtual Warehouse cluster.
    For example:
    $ kubectl cp /tmp/data.json compute-1677087760-cj7q/huebackend-0:/tmp/data.json -c hue
  5. Connect to Hue pod on new Hive/Impala Virtual Warehouse cluster.
    $ kubectl exec -it huebackend-0 -n <new Virtual Warehouse ID> -c hue – /bin/bash
  6. Load data to Hue onto the new Virtual Warehouse cluster.
    $ ./build/env/bin/hue loaddata --exclude auth.permission --exclude contenttypes --ignorenonexistent /tmp/data.json
  7. Go to the Hue UI and verify the saved queries and query history are showing up in the new CDW environment. Also, verify that Hue is working.
  8. If saved queries and history are showing up, and Hue is working, proceed to the next step; otherwise, go to Troubleshooting Hue restoration.
  9. After the data load is finished, revert the container memory limit back to the original 8192M setting.